# **OCT Ecosystem Roadmap (Curated & Prioritized)**

**Version:** 2.0

> **AI Review tracking:** Known bugs and correctness improvements identified by external AI
> review are tracked in three synthesis documents:
> - [`AI_REVIEW_SYNTHESIS V1.md`](AI_REVIEW_SYNTHESIS%20V1.md) — V1 reviews (v0.4.3), 23 OI items, **all resolved**.
> - [`AI_REVIEW_SYNTHESIS V2.md`](AI_REVIEW_SYNTHESIS%20V2.md) — V2 reviews (v0.6.1), 15 OI items, 8 resolved (v0.6.2–v0.7.0), 7 low-priority remaining.
> - [`AI_REVIEW_SYNTHESIS V3.md`](AI_REVIEW_SYNTHESIS%20V3.md) — V3 reviews (v0.7.4), 17 new OI items (2 medium, 12 low, 3 accepted) + 7 V2 carryovers. All 3 phases complete (v0.7.6, v0.7.7, v0.7.8) — **all 21 actionable OI items resolved**. v0.7.9: CLI help overhaul, linter `--dry-run`. v0.7.10: targeted linting, dry-run fix preview, safe header rewriting. **v0.8.0: FS-301 (`oct new --test`), FS-302 (`oct diag`), FS-303 (`oct lint --changed`).** v0.8.1: `.octrc.json` support in formatter/health, health rule profiles, colorized health output, shared terminal utility.
> - [`AI_REVIEW_SYNTHESIS V3_5.md`](AI_REVIEW_SYNTHESIS%20V3_5.md) — V3.5 intermediary reviews (v0.7.10), 9 OI items (2 medium, 4 low, 3 accepted). Phase 1+2 complete (v0.7.11): OI-351, OI-352, OI-353, OI-355 fixed. 2 low-priority items remain (OI-354, OI-356).
>
> Each phase below includes a **Known Issues (AI Review)** subsection cross-referencing
> the relevant OI items.

---

## **Phase 1 — Diagnostics System Extraction** ✅ COMPLETED — oc_diagnostics v1.5

`oc_diagnostics` v1.5 is now the standalone runtime diagnostics package for all Option C
projects.  It is project-agnostic, fully installable, and provides a complete public API.

Diagnostics is a foundational runtime subsystem. OCT is a development ecosystem.
They are separate, and OCT depends on `oc_diagnostics` for linting rules and scaffolding.

### **Goals** ✅ Achieved
- Turn diagnostics into a clean, reusable, installable package.
- Make it project-agnostic.
- Prepare it for integration into OCT scaffolding and linting.

### **Completed Tasks**
- ✅ Extracted diagnostics code from midi-bridge into standalone `oc_diagnostics` package.
- ✅ Normalized file structure (flat `oc_diagnostics/` layout).
- ✅ Added Option C headers and docstrings to all modules.
- ✅ Removed project-specific assumptions (Dash, FastAPI become optional extras).
- ✅ Added `pyproject.toml`.
- ✅ Added README, ARCHITECTURE, REFERENCE, INTEGRATION_GUIDE, CHANGELOG docs.
- ✅ Published locally; OCT scaffold and linting integrate with `oc_diagnostics`.

---

## **Phase 2 — OCT Scaffolding & Project Templates**
Once diagnostics is standalone, OCT can scaffold projects that use it.

### **Goals**
- Make `oct scaffold` produce a complete Option C project.
- Include diagnostics integration automatically.

### **Tasks**
- Create a full project template:
  - `src/<project>/...`
  - `tests/`
  - `docs/`
  - `oc_diagnostics/` config file (valid `oc_diagnostics` template) ✅ Done
  - Example module with Option C header + `_dbg` usage
- Add diagnostics config (`debug_config.json`) to template. ✅ Done
- Copy Doxygen template (`docs/doxygen_work_dir/Doxyfile`) with PROJECT_NAME set to project name. ✅ Done
- Doxyfile included as package data for pip installs. ✅ Done (v0.6.2)
- Add a minimal example test suite.
- Add a template README.
- Add a template architecture document.
- Add a template CLI entry point (optional).
- `oct new --test` — auto-generate companion test file alongside module. (FS-203)

### **Known Issues (AI Review)**

| OI | Title | Status |
|---|---|---|
| OI-02 | Test fixtures created legacy `diagnostics/` instead of `oc_diagnostics/` | ✅ FIXED v0.4.4 |
| OI-07 | Missing Doxyfile template causes scaffold warning on every new project | DISMISSED |
| OI-11 | No version handshake between OCT and `oc_diagnostics` schema | ✅ FIXED v0.6.0 |
| OI-14 | Deprecation timeline for legacy `diagnostics/` support undocumented | ✅ FIXED v0.6.1 |
| OI-18 | Scaffold stubs are minimal; require manual Option C setup | ✅ FIXED v0.6.1 |
| OI-204 | Doxyfile template not included as package data | ✅ FIXED v0.6.2 |
| OI-211 | `oct new` does not generate `__init__.py` in new directories | LOW |
| OI-212 | Scaffold creates `run_tests.py` with shebang but no execute permissions | LOW |

---

## **Phase 3 — OCT Documentation Tools** ✅ SUBSTANTIALLY COMPLETE
With scaffolding in place, documentation generation becomes meaningful.

### **Goals**
- Provide tools for generating and validating Option C documentation.

### **Completed Tasks**
- ✅ Implement `oct docs`:
  - Validate presence of required docs. ✅ Done
  - `oct docs --sphinx` — full Sphinx HTML pipeline (apidoc, markdown discovery, dependency graphs, priority-ordered toctree). ✅ Done
  - `oct docs --doxygen` — Doxygen integration with automatic INPUT path patching via temp Doxyfile copy. ✅ Done
  - `oct docs --all` — combined Sphinx + Doxygen generation. ✅ Done
  - `oct docs --clean` — remove generated artifacts (html/, doctrees/, _api/, doxygen_output/, index.rst, dependencies.rst) while preserving user-authored files. ✅ Done
  - `.env*` files excluded from Sphinx via conf.py.template exclude_patterns. ✅ Done
- ✅ Add diagnostics-aware validation:
  - Ensure Diagnostics block exists. ✅ Done (via linter)
  - Ensure Domain + L2/L3/L4 levels are present. ✅ Done (via linter)

### **Remaining Tasks**
- `oct docs --generate` — generate missing stub docs for existing projects. (FS-214)

---

## **Phase 4 — OCT Linter Enhancements**
Now that diagnostics is standalone and scaffolding is standardized, the linter can evolve.

### **Goals**
- Improve the linter to enforce Option C + diagnostics conventions.

### **Tasks**
- Add `_dbg` usage checks for non-trivial modules. ✅ Done
- Add diagnostics-aware linting rules:
  - Missing Diagnostics block. ✅ Done
  - Missing Domain. ✅ Done
  - Missing L2/L3/L4. ✅ Done
- Enforce `func = "function_name"` local variable pattern required by Option C specs. ✅ Done
- Add colorized terminal output. ✅ Done
- Add import-source check (`from oc_diagnostics import _dbg` vs bare `_dbg`). ✅ Done (v0.6.2)
- Add configurable rule profiles via `.octrc.json`. ✅ Done (v0.7.0)
- Shared exclusion lists between linter and exporter. ✅ Done (v0.7.0)
- `no_hardcoded_secrets` linter rule — AST-based detection of hardcoded secrets (§6b Secret Hygiene, DD-05). Blocking at compact/strict. ✅ Done
- `NEVER_EXPORT_FILE_PATTERNS` — file-level exclusion from exports and docs (Layer 2, DD-05). ✅ Done
- Update exclusion list: add `diagnostics.bk` and `diagnostics.bak` directories.
- Add `oc_diagnostics` (`_dbg`) instrumentation to the linter itself.
- Add plugin system for custom rules. (FS-205)
- Add better path handling (optional refinements).

### **Known Issues (AI Review V1)**
See `AI_REVIEW_SYNTHESIS V1.md` for full detail on each item.

| OI | Title | Status |
|---|---|---|
| OI-03 | Linter uses mutable module-level globals — not thread-safe | ✅ FIXED v0.5.0 |
| OI-04 | Heuristic regression test detection (`check_tests_exist`) produces false positives/negatives | ✅ FIXED v0.5.0 |
| OI-05 | Regex/indentation-based function parsing is fragile (AST migration needed) | ✅ FIXED v0.6.0 |
| OI-06 | `check_diagnostics_profile` uses substring matching — false positives possible | ✅ FIXED v0.5.0 |
| OI-10 | No machine-readable lint output (`--json` flag missing) | ✅ FIXED v0.5.0 |
| OI-12 | Hardcoded exclusion directories in linter; no per-project override | ✅ FIXED v0.5.0 |
| OI-15 | Linter log file path hardcoded to `logs/`; crashes if unwritable | ✅ FIXED v0.5.0 |
| OI-16 | `find_python_files` excludes any path containing `"oct_lint.py"` as substring | ✅ FIXED v0.6.1 |
| OI-19 | Raw ANSI escape codes may display as garbage on older Windows consoles | ✅ FIXED v0.6.1 |
| OI-23 | Linter log files accumulate without rotation | ✅ FIXED v0.6.1 |

### **Known Issues (AI Review V2)**
See `AI_REVIEW_SYNTHESIS V2.md` for full detail on each item.

| OI | Title | Status |
|---|---|---|
| OI-201 | `check_dbg_usage` still uses substring matching (not AST) | ✅ FIXED v0.6.2 |
| OI-202 | No enforcement of exact `from oc_diagnostics import _dbg` import source | ✅ FIXED v0.6.2 |
| OI-205 | `.octrc.json` malformed JSON is silently ignored | ✅ FIXED v0.6.2 |
| OI-206 | Hardcoded exclusion lists duplicated between linter and exporter | ✅ FIXED v0.7.0 |
| OI-208 | Linter rule configuration is hardcoded — no per-project toggle | ✅ FIXED v0.7.0 |
| OI-209 | `LOG_HANDLE` remains a module-level global (not in LinterContext) | LOW |
| OI-210 | `find_python_files` skips any file named `oct_lint.py` — may skip user files | LOW |
| OI-213 | No linter parallelisation for large codebases | LOW |

---

## **Phase 5 — OCT CLI Expansion**
Once the core ecosystem is stable, expand the CLI with additional developer tools.

### **Goals**
- Provide a full suite of development utilities.

### **Tasks**
- `oct test`:
  - Add coverage support.
  - Add flags for verbosity.
- ✅ `oct format` — Option C auto-formatter (headers, docstrings, `func = "..."`, imports). (FS-201, #1 feature request across all reviews) — **IMPLEMENTED v0.7.1**
- ✅ `oct health` — single-command project compliance dashboard (lint %, test status, docs, diagnostics coverage). (FS-202) — **IMPLEMENTED v0.7.2**
- ✅ Global `--root-dir` option — override project root for all commands. — **IMPLEMENTED v0.7.3**
- ✅ Syntax warnings lint check — capture `SyntaxWarning` from AST parsing as structured findings instead of stderr noise. — **IMPLEMENTED v0.7.4**
- ✅ `oct health` hardening — pre-flight pytest detection, actionable error details, test count always shown, clean summary output. — **IMPLEMENTED v0.7.5**
- ✅ `oct new --test` — auto-generate companion test file alongside module. (FS-203/FS-301) — **IMPLEMENTED v0.8.0**
- `oct deps` — dependency health checker, extends `oc_diagnostics` compat check. (FS-208)
- ✅ `oct diag` — diagnostics tools: `validate-config`, `list-domains`, `set-level`. (FS-204/FS-302) — **IMPLEMENTED v0.8.0**
- ✅ `oct lint --changed` — only lint git-modified files for CI efficiency. (FS-209/FS-303) — **IMPLEMENTED v0.8.0**
- ✅ `.octrc.json` support in `oct format` and `oct health` — formatter and health dashboard now load per-project exclusions and rule overrides from `.octrc.json`, matching the linter. — **IMPLEMENTED v0.8.1**
- ✅ `oct health` rule profiles — health dashboard respects `.octrc.json` `rules` and `profile` config; disabled rules are excluded from compliance counts and shown in report. — **IMPLEMENTED v0.8.1**
- ✅ `oct health` colorized output — health report uses ANSI colors (green/red/yellow/cyan) matching `oct lint` style; safe on Windows and piped output. — **IMPLEMENTED v0.8.1**
- ✅ Shared terminal color utility — extracted `oct/core/terminal.py` from linter for reuse across modules. — **IMPLEMENTED v0.8.1**
- Pre-commit hook integration — generate `.pre-commit-config.yaml` in `oct scaffold`. (FS-206)
- Improve help text and examples for all commands.

### **Known Issues (AI Review V1)**
See `AI_REVIEW_SYNTHESIS V1.md` for full detail on each item.

| OI | Title | Priority |
|---|---|---|
| OI-08 | `oct new` accepts any file extension but always injects Python content | ✅ FIXED v0.5.0 |
| OI-09 | No `--dry-run` mode for `scaffold`, `new`, `clean` | ✅ FIXED v0.6.0 |
| OI-17 | No `__version__` exposed from the `oct` package | ✅ FIXED v0.6.1 |
| OI-20 | Bare `print()` + `sys.exit(1)` error handling instead of Click exceptions | ✅ FIXED v0.6.1 |
| OI-21 | `oct test` does not verify pytest is installed before invoking it | ✅ FIXED v0.6.1 |
| OI-22 | Click pinned to `==8.1` (consider relaxing to `>=8.1,<10`) | ✅ FIXED v0.6.1 |

### **Known Issues (AI Review V2)**
See `AI_REVIEW_SYNTHESIS V2.md` for full detail on each item.

| OI | Title | Status |
|---|---|---|
| OI-203 | `.octrc.json` and `--single-dir` are undocumented | ✅ FIXED v0.6.2 |
| OI-207 | `oct new` does not validate target path is inside project root | ✅ FIXED v0.7.0 |
| OI-214 | Legacy `diagnostics/` deprecation is warning-only — no migration timeline | LOW |
| OI-215 | JSON lint output suppresses compat warnings — CI users may miss them | LOW |

---

## **Phase 6 — Source Exporter Enhancements**
The exporter is now integrated; this phase adds polish and power.

### **Goals**
- Make the exporter more robust and diagnostics-aware.

### **Tasks**
- Fixed double-counting of lines/files in summary totals. ✅ Done
- Per-project config file (`_oct_exporter_config.json`) — override include/exclude lists and
  format strings with add/remove/replace semantics. ✅ Done
- Shared exclusion lists with linter via `oct/core/exclusions.py`. ✅ Done (v0.7.0)
- Add `_dbg` instrumentation.
- Add Option C docstrings to helper functions.
- Add unit tests.
- Add dry-run mode.
- Add JSON summary output.
- Add diff mode (compare two exports). (FS-207)
- Add list-only mode.
- ✅ `oct export-skeleton` — AST-based structural skeleton export (headers, docstrings, signatures) with `--no-diagnostics`, `--single-dir`, `--verbose`, `--clean`. Done (v0.9.1)

### **Known Issues (AI Review)**

| OI | Title | Status |
|---|---|---|
| OI-01 | Exporter config listed `"gitignore"` (missing dot) — `.gitignore` silently excluded | ✅ FIXED v0.4.4 |
| OI-12 | Hardcoded exclusion directories in exporter; no per-project override | ✅ FIXED v0.7.0 |
| OI-13 | Exporter creates per-directory output folders, cluttering filesystem on large projects | ✅ FIXED v0.6.0 |

---

## **Phase 7 — Ecosystem Documentation**
Once the ecosystem is mature, document it thoroughly.

### **Goals**
- Provide a complete, authoritative reference for the entire OCT ecosystem.

### **Tasks**
- Write an "OCT Ecosystem Index":
  - Architecture
  - Philosophy
  - Tools
  - Diagnostics integration
  - Project lifecycle
- Write an OCT user guide.
- Write an OCT developer guide.

---

## **Phase 8 — Linter Packaging (Optional)**
Package `oct_lint` as a standalone, distributable tool.

### **Goals**
- Allow the linter to be used independently of the full OCT install.

### **Tasks**
- Create a standalone linter project directory.
- Generate README.md for the standalone linter.
- Generate pyproject.toml for the standalone linter.
- Generate a migration checklist for adopters.

---

## **Summary of Priorities**

### **Completed (Phase 1)**
`oc_diagnostics` v1.5 is live as a standalone package. OCT scaffold and linting integrate with it.

### **Substantially Complete (Phase 2)**
Core scaffolding is working: `oc_diagnostics/debug_config.json`, Doxygen Doxyfile, and directory
structure are all generated. Doxyfile template is now included as package data (v0.6.2).
Remaining gaps: full `src/<project>/` layout and example module.

### **AI Review Issues V1 — ALL RESOLVED**
All 23 actionable OI items from the V1 AI code reviews are now fixed across four releases
(v0.4.4, v0.5.0, v0.6.0, v0.6.1). OI-07 dismissed (intentional design), OI-24 no action
needed (intentional architecture). See `AI_REVIEW_SYNTHESIS V1.md` for full detail.

### **AI Review Issues V2 — 8 of 15 RESOLVED**
The V2 review round (6 AI reviewers against v0.6.1) identified 15 new OI items.
8 medium-priority items are now resolved across v0.6.2 and v0.7.0:
- **v0.6.2:** OI-201 (AST dbg check), OI-202 (import source), OI-203 (docs), OI-204 (packaging), OI-205 (config warning)
- **v0.7.0:** OI-206 (shared exclusions), OI-207 (path validation), OI-208 (rule profiles)

7 low-priority items (OI-209–OI-215) and 3 accepted limitations (OI-216–OI-218) remain.
See `AI_REVIEW_SYNTHESIS V2.md` for full detail.

### **Feature Suggestions (from V2 AI Reviews)**
The following high-value feature suggestions were consolidated from 6 AI reviewers.
See `AI_REVIEW_SYNTHESIS V2.md` section 6 for full detail and tiering.

| FS | Feature | Priority | Roadmap phase |
|---|---|---|---|
| FS-201 | ✅ `oct format` — Option C auto-formatter | **#1 request** (all 6 reviewers) | Phase 5 — **DONE v0.7.1** |
| FS-202 | ✅ `oct health` — project compliance dashboard | High (4+ reviewers) | Phase 5 — **DONE v0.7.2** |
| FS-203 | `oct new --test` — companion test file | High (3 reviewers) | Phase 2/5 |
| FS-204 | `oct diag` — diagnostics runtime integration | High (4 reviewers) | Phase 5 |
| FS-205 | Linter plugin system | Medium (4 reviewers) | Phase 4 |
| FS-206 | Pre-commit hook integration | Medium (2 reviewers) | Phase 5 |
| FS-207 | Export diff mode | Medium (2 reviewers) | Phase 6 |
| FS-208 | `oct deps` — dependency checker | Medium (2 reviewers) | Phase 5 |
| FS-209 | `oct lint --changed` (git-modified only) | Medium (1 reviewer) | Phase 5 |

### **Feature Suggestions (from V3 AI Reviews)**
See `AI_REVIEW_SYNTHESIS V3.md` section 6 for full detail and tiering.

| FS | Feature | Priority | Roadmap phase |
|---|---|---|---|
| FS-301 | `oct new --test` — companion test file (= FS-203) | **#1 request** (all 7 reviewers) | ✅ **DONE v0.8.0** |
| FS-302 | `oct diag` — diagnostics runtime integration (= FS-204) | High (5 reviewers) | ✅ **DONE v0.8.0** |
| FS-303 | `oct lint --changed` — git-modified files only (= FS-209) | High (4 reviewers) | ✅ **DONE v0.8.0** |
| FS-304 | Linter plugin system (= FS-205) | Medium (5 reviewers) | Phase 4 — PENDING |
| FS-305 | Pre-commit + CI templates in scaffold (= FS-206) | Medium (2 reviewers) | Phase 5 — PENDING |
| FS-306 | `oct upgrade` — automated project migration | Medium (3 reviewers) | Phase 5 — PENDING |
| FS-307 | `export-source --diff` — compare two exports (= FS-207) | Medium (2 reviewers) | Phase 6 — PENDING |

### **AI Review Issues V3.5 — 4 of 6 RESOLVED**
The V3.5 intermediary round (3 review files, 2 reviewers against v0.7.10) identified 9 OI items.
4 actionable items resolved in v0.7.11:
- **v0.7.11:** OI-351 (source exporter bare except), OI-352 (formatter lineno validation), OI-353 (regex word boundary), OI-355 (config validation)

2 low-priority items remain (OI-354 memory exhaustion, OI-356 legacy typing). 3 items accepted/dismissed.
See `AI_REVIEW_SYNTHESIS V3_5.md` for full detail.

### **Substantially Complete (Phase 3)**
Documentation tools: `oct docs --sphinx`, `--doxygen`, `--all`, `--clean` all implemented. Remaining: `--generate` for stub generation.

### **In Progress (Phase 4)**
Linter: `no_hardcoded_secrets` (§6b) and `NEVER_EXPORT_FILE_PATTERNS` (Layer 2) implemented. Remaining: plugin system, instrumentation.

### **Substantially Complete (Phase 5)**
CLI expansion: `oct format`, `oct health`, `oct diag`, `oct deps`, `oct new --test`, `oct lint --changed` all implemented. Remaining: `oct test` coverage, pre-commit hooks.

### **Finally (Phase 6-8)**
`oct export-skeleton` implemented (v0.9.1) with `--no-diagnostics`, `--single-dir`, `--verbose`, `--clean` flags. Remaining: exporter diff mode, ecosystem documentation, and optional linter packaging.