# oc_diagnostics — Unified Debugging & Instrumentation System
**Version:** 1.3
Authoritative Reference Guide
Option C Runtime Diagnostics

---

## Table of Contents

1. [Overview](#overview)
2. [Unified Debugging System](#unified-debugging-system)
   - [Goals](#goals)
   - [Debug Levels](#debug-levels)
   - [Domains](#domains)
   - [Global Settings](#global-settings)
   - [Environment Variables](#environment-variables)
   - [Output Routing](#output-routing)
   - [Color System](#color-system)
   - [Timestamps](#timestamps)
   - [Silent Mode](#silent-mode)
   - [Exception Hook](#exception-hook)
   - [Stdout/Stderr Interception](#stdoutstderr-interception)
   - [Using `_dbg`](#using-_dbg)
   - [Async `_adbg`](#async-_adbg)
   - [Runtime API](#runtime-api)
   - [Profiling and Timing](#profiling-and-timing)
   - [Production Mode](#production-mode)
   - [Config Hot-Reload](#config-hot-reload)
   - [Log Rotation](#log-rotation)
   - [JSONL Output Mode](#jsonl-output-mode)
   - [Python logging Integration](#python-logging-integration)
3. [Instrumentation System](#instrumentation-system)
   - [Purpose](#purpose)
   - [Global Toggle](#global-toggle)
   - [Instrumentation Levels](#instrumentation-levels)
   - [Structural ID Protection](#structural-id-protection)
   - [Instrumentation Rules](#instrumentation-rules)
   - [Instrumentation Helpers](#instrumentation-helpers)
   - [Recursion Depth Limit](#recursion-depth-limit)
   - [The ID Contract](#the-id-contract-critical)
   - [Two‑Phase Instrumentation Model](#two-phase-instrumentation-model)
4. [Instrumentation Middleware](#instrumentation-middleware)
   - [Purpose](#purpose-1)
   - [Static Layout Instrumentation](#static-layout-instrumentation)
   - [Dynamic Callback Instrumentation](#dynamic-callback-instrumentation)
   - [List and Tuple Return Handling](#list-and-tuple-return-handling)
   - [Middleware Limitations](#middleware-limitations)
   - [Middleware Activation Contract](#middleware-activation-contract)
5. [UI Event Logger](#ui-event-logger)
   - [Purpose](#purpose-2)
   - [ID Requirements](#id-requirements)
   - [Store Dependency](#store-dependency)
   - [Startup Store Validation](#startup-store-validation)
   - [Event Value Truncation](#event-value-truncation)
   - [Callback Contract](#callback-contract)
6. [FastAPI / ASGI Middleware](#fastapi--asgi-middleware)
   - [Purpose](#purpose-3)
   - [Activation](#activation)
   - [Log Levels](#log-levels)
   - [Contracts](#contracts)
7. [Self‑Test Suite](#self-test-suite)
   - [Purpose](#purpose-4)
   - [Automated Runner (tests/run\_tests.py)](#automated-runner-testsrun_testspy)
   - [CLI Tests](#cli-tests)
   - [Dash App Tests](#dash-app-tests)
   - [Self‑Test Lifecycle](#self-test-lifecycle)
   - [Verbosity Modes](#verbosity-modes)
8. [Migration Tool](#migration-tool)
9. [Recommended Debug Configurations](#recommended-debug-configurations)
10. [Extending the System](#extending-the-system)
11. [Appendix: File Map](#appendix-file-map)

---

## Overview

`oc_diagnostics` is the unified debugging, tracing, and instrumentation system for all Option C‑compliant projects.

It is designed to be:

- Authoritative
- Predictable
- Project‑agnostic
- Self‑contained
- Future‑proof
- Developer‑friendly

The system consists of:

1. Unified Debugging System
2. Instrumentation System
3. Instrumentation Middleware
4. UI Event Logger
5. FastAPI / ASGI Middleware
6. Self‑Test Suite

It is a **runtime** package.
OCT is a **development‑time** package.
Projects consume both.

---

## Unified Debugging System

### Goals

Provide a single, consistent, complete debugging pipeline across all Option C projects.

### Debug Levels

```
0 = Silent
1 = Errors only
2 = Lifecycle events
3 = Important details
4 = Verbose analysis
5–9 = Deep tracing
```

### Domains

Projects define their own domains in:

```
diagnostics/debug_config.json
```

Common domains include:

```
UI, DATA, ENGINE, SETTINGS, INSTRUMENTATION, HTTP, PYERR, STDOUT, STDERR
```

### Global Settings

Loaded from `debug_config.json` (`global_settings` object):

| Key | Type | Default | Description |
|---|---|---|---|
| `silent_mode` | bool | `false` | Hard global off switch — suppresses all output. |
| `enable_timestamps` | bool | `true` | Prefix each log line with `HH:MM:SS.mmm`. |
| `enable_colors` | bool | `true` | ANSI color codes in terminal output. |
| `include_filename` | bool | `true` | Include `filename:line::caller` in each log line. |
| `output_mode` | str | `"both"` | Routing target: `"terminal"`, `"file"`, `"both"`, `"json"`. |
| `instrumentation_enabled` | bool | `true` | Master toggle for all ID instrumentation. |
| `instrumentation_allow_all` | bool | `false` | Instrument non-Dash objects without requiring a high debug level (FS-13). |
| `max_log_files` | int | `0` | Keep at most N log files; `0` = unlimited (FS-06). |
| `max_log_size_mb` | float | `0` | Roll over to a new file when current file exceeds N MB; `0` = disabled (FS-06). |
| `ui_event_properties` | list[str] | `["n_clicks","value","data","figure","selectedData"]` | Properties the UI event logger listens to (FS-10). |
| `http_redact_headers` | list[str] | `["authorization","cookie","set-cookie","x-api-key"]` | Request/response headers replaced with `[REDACTED]` at L4 (FS-17). |
| `mode` | str | `""` | Set to `"prod"` / `"production"` / `"safe"` to activate production mode (FS-15). |
| `structural_id_prefixes` | list[str] | `[]` | Additional ID prefixes treated as structural (never instrumented) (FS-02). |
| `structural_id_exact` | list[str] | `[]` | Additional exact IDs treated as structural (never instrumented) (FS-02). |

### Environment Variables

Four environment variables allow the config path, log path, and initialization
behavior to be overridden without code changes.  All are optional; the default
behavior is unchanged when they are absent.

| Variable | Default | Description |
|---|---|---|
| `OC_DIAGNOSTICS_CONFIG` | `<cwd>/diagnostics/debug_config.json` | Absolute path to a `debug_config.json` file. Takes precedence over the project-root convention. Useful for embedded usage, subprocess invocation, or CI environments where the working directory is not the project root. |
| `OC_DIAGNOSTICS_LOG_DIR` | `<cwd>/logs` | Absolute path to the directory where log files are written. Replaces the default CWD-relative `logs/` directory. |
| `UNIFIED_DEBUG_DISABLE_EXCEPTION_HOOK` | *(unset)* | Set to `1` to prevent `init()` from installing the custom `sys.excepthook`. Useful when the host environment (pytest, pdb, ASGI framework) manages its own exception handler. |
| `UNIFIED_DEBUG_DISABLE_STDIO_INTERCEPT` | *(unset)* | Set to `1` to prevent `init()` from replacing `sys.stdout`/`sys.stderr` with `_StreamInterceptor`. |

**Resolution order for config:**

1. `OC_DIAGNOSTICS_CONFIG` env var (if set and non-empty)
2. `<cwd>/diagnostics/debug_config.json` (project root convention)
3. `debug_config.json` next to `unified_debug.py` (bundled fallback / tests)

`CONFIG_SOURCE` always records which path was used and how it was resolved.

**Example:**

```bash
OC_DIAGNOSTICS_CONFIG=/etc/myapp/debug_config.json \
OC_DIAGNOSTICS_LOG_DIR=/var/log/myapp \
python myapp.py
```

### Output Routing

| Mode | Terminal | Log file |
|---|---|---|
| `terminal` | Colored text | — |
| `file` | — | Plain text (ANSI stripped) |
| `both` | Colored text | Plain text (ANSI stripped) |
| `json` | Colored text | JSONL (one JSON object per line) |

See [JSONL Output Mode](#jsonl-output-mode) for structured log details.

### Color System

Each domain may define a color.
Errors (`level=1`) are always red.

### Timestamps

Format:

```
HH:MM:SS.mmm
```

### Silent Mode

Hard global off switch.

### Exception Hook

Uncaught exceptions are logged as `PYERR` (L1) and then delegated to the hook
that was active when `init()` was called.  This preserves the behavior of
debuggers (`pdb`), test runners (pytest), and framework-level formatters
(`rich.traceback.install()`) that install their own handlers.

Set `UNIFIED_DEBUG_DISABLE_EXCEPTION_HOOK=1` to prevent `init()` from
installing the hook at all.

### Stdout/Stderr Interception

All `print()` calls and server logs are captured and routed through `_dbg`.

### Using `_dbg`

```
_dbg(domain, func, message, level, override=False)
```

**Pre-`init()` warning:** If `_dbg()` is called before `init()`, a one-shot
`RuntimeWarning` is emitted via `warnings.warn()` (unless `SILENT_MODE` is true).
This surfaces forgotten `init()` calls at import time rather than silently producing
no log-file output.

Example:

```python
func = "load_data"
_dbg("DATA", func, "START", 2)
_dbg("DATA", func, f"rows={len(df)}", 3)
_dbg("DATA", func, "df_head=...", 4)
```

### Async `_adbg`

`_adbg()` is the async variant of `_dbg()`, designed for use inside `async def` functions
and coroutines (FS-16).

```python
from oc_diagnostics import _adbg

async def fetch_data():
    await _adbg("DATA", "fetch_data", "START", level=2)
    result = await some_async_call()
    await _adbg("DATA", "fetch_data", f"rows={len(result)}", level=3)
    return result
```

Key behaviors:

- The caller frame (filename, line number, function name) is captured **synchronously**
  before any `await`, so location information is always accurate.
- Terminal output is written synchronously (no latency).
- File I/O is offloaded to `asyncio.to_thread()` so it never blocks the event loop.
- JSONL records are serialized and written the same way.
- Signature mirrors `_dbg()` exactly; drop-in replacement inside `async` functions.

---

### Runtime API

Two functions allow debug levels and global settings to be changed at runtime without
editing `debug_config.json` or restarting the process (FS-03):

#### `set_debug_level(domain, level)`

```python
from oc_diagnostics import set_debug_level

set_debug_level("HTTP", 4)   # enable full header dumps for the current session
set_debug_level("UI", 0)     # silence UI events
```

- `domain`: any string (new domains are created if they do not exist in `DEBUG_LEVELS`)
- `level`: integer 0–9; raises `UserWarning` and returns without modifying anything if invalid
- Mutates `DEBUG_LEVELS` in place; change takes effect immediately

#### `set_global_setting(key, value)`

```python
from oc_diagnostics import set_global_setting

set_global_setting("silent_mode", True)
set_global_setting("output_mode", "json")
set_global_setting("enable_timestamps", False)
```

Valid keys: `silent_mode`, `enable_timestamps`, `enable_colors`, `include_filename`, `output_mode`.

Raises `UserWarning` for unknown keys and returns without modifying anything.

---

### Profiling and Timing

Three utilities cover lifecycle and timing tracing (FS-12, FS-19):

#### `dbg_scope(domain, func, label="", level=2)` — context manager (no timing)

```python
from oc_diagnostics import dbg_scope

with dbg_scope("DATA", "load_records"):
    result = load()
# Emits: [DATA][L2] ENTER
#        [DATA][L2] EXIT
```

Logs `ENTER` and `EXIT` at the specified level without any timing overhead.
Use this for pure lifecycle tracing where elapsed time is not needed.
For timed tracing, use `dbg_timed()` instead.

#### `dbg_timed(domain, func, label="", level=3)` — context manager

```python
from oc_diagnostics import dbg_timed

with dbg_timed("DATA", "load_records", label="CSV parse", level=3):
    df = pd.read_csv(path)
```

Emits a `ENTER` log line at the start and an `EXIT (N.N ms)` line at exit.

#### `dbg_profile(domain, func, label="", level=3)` — decorator

```python
from oc_diagnostics import dbg_profile

@dbg_profile(domain="ENGINE", level=3)
def compute_result(x, y):
    return x + y
```

Wraps the decorated function with `dbg_timed()`; the original return value is preserved.
The `label` defaults to the function name when not specified.

---

### Production Mode

When running in a production environment, stream interception and the exception hook
can be suppressed automatically without code changes (FS-15).

**Via environment variable (highest priority):**

```bash
OC_DIAGNOSTICS_MODE=prod python myapp.py
```

**Via `debug_config.json`:**

```json
"global_settings": {
  "mode": "prod"
}
```

Accepted values: `"prod"`, `"production"`, `"safe"`.

When production mode is active, `init()` behaves as if called with
`intercept_streams=False, intercept_exceptions=False`, regardless of the arguments
passed. All `_dbg()` / `_adbg()` calls continue to work normally; only the
stream/exception hook side-effects are suppressed.

---

### Config Hot-Reload

`init()` accepts an optional `watch_config=True` argument that starts a background
daemon thread polling the config file every 2 seconds (FS-09):

```python
from oc_diagnostics import init

init(watch_config=True)
```

When the file's modification time changes, all globals — `DEBUG_LEVELS`, `COLOR_CODES`,
`OUTPUT_MODE`, `HTTP_REDACT_HEADERS`, `UI_EVENT_PROPERTIES`, etc. — are updated in place.
The `DEBUG_LEVELS` dict object itself is preserved (`clear()` + `update()`), so any
downstream references to the dict continue to reflect the new values.

Hot-reload is disabled by default to avoid unnecessary file-system polling in
short-lived scripts and test runners.

---

### Log Rotation

Two complementary rotation strategies are available (FS-06):

**Count-based rotation** (`max_log_files`):

When the number of `unified_debug_logs_*.txt` files in the log directory exceeds
`max_log_files`, the oldest files are deleted before a new file is created.

**Size-based rotation** (`max_log_size_mb`):

When the current log file exceeds `max_log_size_mb` megabytes, `LOG_FILE` is reset and
a new timestamped file is created on the next write.

Both strategies can be active simultaneously.  Set either to `0` to disable it.

```json
"global_settings": {
  "max_log_files": 10,
  "max_log_size_mb": 5.0
}
```

---

### JSONL Output Mode

Setting `output_mode` to `"json"` switches file output to newline-delimited JSON
(JSONL), while terminal output continues to display colored human-readable text (FS-07).

```json
"global_settings": {
  "output_mode": "json"
}
```

Each log line in the file becomes a JSON object:

```json
{"ts":"2026-02-27T14:23:01.042","domain":"DATA","level":3,"func":"load_records","file":"data_loader.py","line":42,"caller":"main","msg":"rows=1500","override":false}
```

Fields:

| Field | Type | Description |
|---|---|---|
| `ts` | str | ISO 8601 timestamp with millisecond precision |
| `domain` | str | Debug domain |
| `level` | int | Effective debug level |
| `func` | str | Logical function name passed to `_dbg()` |
| `file` | str | Source filename |
| `line` | int | Source line number |
| `caller` | str | Python function name at the call site |
| `msg` | str | Message string |
| `override` | bool | `true` when `_dbg()` was called with `override=True` |

JSONL files are suitable for ingestion by log aggregation systems (Loki, Splunk, etc.)
and can be post-processed with `jq`.

---

### Python logging Integration

`OcDiagnosticsHandler` bridges the standard-library `logging` module with
`oc_diagnostics`, routing log records through `_dbg()` (FS-08):

```python
import logging
from oc_diagnostics import OcDiagnosticsHandler

logger = logging.getLogger("myapp")
logger.addHandler(OcDiagnosticsHandler(domain="GENERAL"))
logger.setLevel(logging.DEBUG)

logger.info("Server started")      # → _dbg("GENERAL", ..., level=2)
logger.debug("Detail here")        # → _dbg("GENERAL", ..., level=4)
logger.warning("Low disk space")   # → _dbg("GENERAL", ..., level=1)
```

Default level mapping:

| `logging` level | `_dbg` level |
|---|---|
| `DEBUG` (10) | 4 |
| `INFO` (20) | 2 |
| `WARNING` (30) | 1 |
| `ERROR` (40) | 1 |
| `CRITICAL` (50) | 1 |

Provide a custom `level_map` to override the defaults:

```python
import logging
from oc_diagnostics import OcDiagnosticsHandler

handler = OcDiagnosticsHandler(
    domain="REQUESTS",
    level_map={
        logging.DEBUG:   3,
        logging.INFO:    2,
        logging.WARNING: 2,
        logging.ERROR:   1,
    },
)
logging.getLogger("requests").addHandler(handler)
```

---

## Instrumentation System

### Purpose

Rewrite Dash component IDs so the UI event logger can capture all interactions.

### Global Toggle

```
instrumentation_enabled = true
```

### Instrumentation Levels

```
0     = Off (instrumentation_active() returns False)
1–4   = Standard Dash components only
          - 1–2: static layout instrumentation
          - 3:   static + dynamic (callback output) instrumentation
          - 4:   verbose logging (detailed recursion output)
5–9   = Standard Dash + custom components
          - 5–7: instrumentation_allow_custom_components() becomes True;
                 plain Python objects with a .id attribute are also rewritten
          - 8–9: maximum verbosity / full tracing
```

"Custom component" means any object that does **not** inherit from
`dash.development.base_component.Component`.  This includes plain Python
classes, third-party non-Dash objects, and mock objects used in tests.
Standard Dash components (all `dash.html`, `dash.dcc`, `dash_bootstrap_components`,
etc.) are always instrumented at level ≥ 1.

### Structural ID Protection

Never instrument:

- IDs starting with:
  `root-group-`, `tab-group-`, `pane-group-`, `tab-content-`
- Exact IDs such as:
  `theme-root`, `ui-theme-store`, `system-settings-store`, etc.

Projects may extend these lists without modifying package source by adding
`structural_id_prefixes` and `structural_id_exact` lists to `global_settings`
in `debug_config.json` (FS-02):

```json
"global_settings": {
  "structural_id_prefixes": ["my-group-", "app-root-"],
  "structural_id_exact": ["my-app-store", "shared-nav-store"]
}
```

These are loaded into `ADDITIONAL_STRUCTURAL_PREFIXES` and
`ADDITIONAL_STRUCTURAL_EXACT` in `unified_debug` and merged into
`is_structural_id()` checks at runtime.

### Instrumentation Rules

Instrument:

- Components with string IDs
- IDs in `.id`, `_props`, `_prop_ids`
- Nested children

Skip:

- Structural IDs
- Dict IDs
- Components without IDs
- Custom components unless allowed

### Instrumentation Helpers

- `instrumentation_active()` — returns `True` when level ≥ 1
- `instrumentation_verbose()` — returns `True` when level ≥ 4
- `instrumentation_allow_custom_components()` — returns `True` when level ≥ 5
- `instrumentation_allow_all()` — returns `True` when `instrumentation_allow_all: true` in config (FS-13);
  instruments non-Dash objects regardless of debug level

### Recursion Depth Limit

`instrument_ids()` enforces a maximum recursion depth:

```python
from oc_diagnostics.id_instrumentation import MAX_INSTRUMENT_DEPTH
# MAX_INSTRUMENT_DEPTH == 500
```

When the limit is exceeded, a L1 diagnostic is emitted and the subtree below
that point is returned uninstrumented.  Real-world Dash applications rarely
exceed 30–50 levels of nesting; the 500-level limit exists as a safety guard
against pathological or circular component structures.

### The ID Contract (Critical)

Dash callback decorators match **string IDs**.
Instrumentation rewrites IDs into **dicts**.

Therefore:

**Static layout must not be instrumented before callback registration.**

If static layout is instrumented too early:

- Callbacks will not fire
- Dynamic content will not appear
- UI event logger will not receive events

### Two‑Phase Instrumentation Model

**Phase 1 — Static instrumentation**
Performed *after* callback registration.

**Phase 2 — Dynamic instrumentation**
Performed explicitly inside callbacks:

```python
return apply_instrumentation(dynamic_tree)
```

Middleware supplements but does not replace explicit calls.

---

## Instrumentation Middleware

### Purpose

Provide a central integration point for dynamic instrumentation.

### Static Layout Instrumentation

Performed in `build_layout`:

```python
full_layout = apply_instrumentation(full_layout)
```

Only safe **after** callbacks are registered.

### Dynamic Callback Instrumentation

Authoritative mechanism:

```python
return apply_instrumentation(dynamic_tree)
```

### List and Tuple Return Handling

`apply_instrumentation()` accepts single components, lists, and tuples.
When given a list or tuple it instruments each element individually and
returns a container of the same type:

```python
# Callback returning multiple components as a list
return apply_instrumentation([header_div, content_div, footer_div])

# Callback returning a tuple
return apply_instrumentation((header, body))
```

### Middleware Limitations

Dash 3.x callback_map entries:

- may contain wrappers
- may not expose original callbacks
- may not be complete until after first layout request
- are not guaranteed stable

Middleware is opportunistic.
Explicit instrumentation is authoritative.

### Middleware Activation Contract

Must run:

- after app creation
- before layout building
- after UI event logger registration

---

## UI Event Logger

### Purpose

Capture all user interactions across the entire UI.

### ID Requirements

Listens only to IDs of the form:

```
{"event": "<original-id>"}
```

### Store Dependency

Requires a `dcc.Store` with a specific ID in the Dash layout:

```python
from oc_diagnostics.ui_event_logger import REQUIRED_STORE_ID
# REQUIRED_STORE_ID == "debug-settings-data-store"

dcc.Store(id=REQUIRED_STORE_ID, storage_type="memory")
```

Use the exported constant rather than the hard-coded string to guarantee
consistency across all call sites.

### Startup Store Validation

`register_callbacks(app)` inspects `app.layout` at registration time and
emits a **L1 diagnostic warning** when `REQUIRED_STORE_ID` is not found:

```
[UI][L1] WARNING: 'debug-settings-data-store' not found in app.layout —
the UI event logger callback will fail at runtime.
```

The check is skipped when `app.layout` is `None` or a callable (resolved at
request time).

### Event Value Truncation

Event values are truncated before logging:

- All properties: `repr()` is capped at **500 characters**; values exceeding
  this are annotated with the original length.
- `figure` and `data` properties: logged as `<type len=N>` summary when
  `DEBUG_LEVELS["UI"] < 9`; full repr is logged at level 9.

```
EVENT: id=my-graph, property=figure, value=<dict len=5> (set UI level=9 for full value)
```

### Callback Contract

The set of properties listened to is configurable via `ui_event_properties` in
`global_settings` (FS-10):

```json
"global_settings": {
  "ui_event_properties": ["n_clicks", "value", "data", "figure", "selectedData"]
}
```

Default properties: `n_clicks`, `value`, `data`, `figure`, `selectedData`.

Returns `no_update` — the callback never modifies the store.

> **Note:** Changing `ui_event_properties` requires restarting the application;
> the property list is read at Dash callback registration time.

---

## FastAPI / ASGI Middleware

### Purpose

Log every HTTP request and response through `_dbg()` as a pure ASGI middleware
callable.  Works with FastAPI, Starlette, or any ASGI-compatible framework.

### Activation

```python
from fastapi import FastAPI
from oc_diagnostics import init, DiagnosticsMiddleware

app = FastAPI()

# ... register routes ...

app.add_middleware(DiagnosticsMiddleware)
init()  # activate log file and stream interception
```

With a custom domain name:

```python
app.add_middleware(DiagnosticsMiddleware, domain="API")
```

Enable HTTP logging by adding the `HTTP` domain to `diagnostics/debug_config.json`:

```json
{
  "domains": {
    "HTTP": { "level": 2, "color": "blue" }
  }
}
```

### Log Levels

| Level | Output |
|---|---|
| 2 | Request start + response end (method, path, status, elapsed ms) |
| 3 | User-Agent header; 4xx/5xx status class commentary |
| 4 | Full request and response header dumps |

### Header Redaction

At L4, sensitive headers are replaced with `[REDACTED]` (FS-17).  The default
redacted headers are `authorization`, `cookie`, `set-cookie`, and `x-api-key`.

Customize the list in `debug_config.json`:

```json
"global_settings": {
  "http_redact_headers": ["authorization", "cookie", "set-cookie", "x-api-key", "x-auth-token"]
}
```

### Contracts

- Non-HTTP scopes (WebSocket, lifespan) pass through unchanged.
- Application exceptions are logged at L2 and then re-raised; the middleware
  never swallows errors.
- Request and response bodies are never buffered or modified.
- Logging output only appears when `DEBUG_LEVELS["HTTP"] >= 2`; setting the
  level to `0` disables HTTP logging entirely.
- No dependency on `starlette.BaseHTTPMiddleware`; pure ASGI callable.
- Header redaction is always active at L4; it cannot be disabled without
  removing the headers from `http_redact_headers`.

Install the optional extra:

```bash
pip install oc_diagnostics[fastapi]
```

---

## Self‑Test Suite

### Purpose

Validate the entire instrumentation pipeline.

### Automated Runner (tests/run_tests.py)

The primary automated test runner for CI/CD integration.

Covers:

- **Core tests** (no Dash or optional dependencies required):
  - config loading and `CONFIG_SOURCE`
  - `_dbg` silent mode
  - `_StreamInterceptor` interface
  - `is_structural_id()` classification
  - `DEBUG_LEVELS` mutability
- **FastAPI ASGI contract tests** (no optional dependencies required):
  - HTTP scope: status code pass-through
  - WebSocket scope: non-HTTP pass-through
  - Application exceptions: logged and re-raised
- **Dash tests** (auto-skip when Dash is not installed):
  - `apply_instrumentation` no-op when disabled
  - `instrument_ids` basic ID rewriting
  - `ui_event_logger` module import
  - list and tuple callback output instrumentation
  - `instrumentation_selftest_cli` subprocess

Exit codes: `0` on full pass, `1` on any failure.

### CLI Tests

`instrumentation_selftest_cli.py` validates:

- static instrumentation
- nested recursion
- structural ID protection
- custom components
- debug‑level transitions

All assertions use explicit `if not condition: raise AssertionError(message)` so
checks survive `python -O` optimization.

Modes:

- default: FORCE VERBOSE
- optional: `--use-global`

### Dash App Tests

`instrumentation_selftest_app.py` validates:

- dynamic instrumentation
- UI event logger
- dispatcher patching
- ID rewriting
- logging behavior

### Self‑Test Lifecycle

1. Build static layout (uninstrumented)
2. Register UI event logger
3. Register callbacks
4. Enable middleware
5. Run server
6. Instrument dynamic content explicitly

### Verbosity Modes

- default: FORCE VERBOSE
- optional: `--use-global`

---

## Migration Tool

`tools/migrate_to_oc_diagnostics.py` migrates existing projects from legacy
per-file `debug.py` / `unified_debug.py` to the `oc_diagnostics` package.

### Usage

```bash
# Preview changes without modifying any files
python tools/migrate_to_oc_diagnostics.py /path/to/project --dry-run

# Migrate and rename diagnostics/ → diagnostics.bak/ (default — backup preserved)
python tools/migrate_to_oc_diagnostics.py /path/to/project

# Migrate and delete the original diagnostics/ directory (no backup)
python tools/migrate_to_oc_diagnostics.py /path/to/project --delete-original
```

### Behavior

- Rewrites all `from debug import _dbg` and related import patterns to
  `from oc_diagnostics import _dbg`.
- Excludes common non-project directories (`.git`, `__pycache__`, `venv`,
  `env`, `.venv`, `node_modules`, etc.).
- By default, renames the existing `diagnostics/` directory to `diagnostics.bak/`
  so the project config is preserved and recoverable.
- `--delete-original` deletes `diagnostics/` without creating a backup.
- `--dry-run` previews all changes without modifying any files.

---

## Recommended Debug Configurations

Development:

```
INSTRUMENTATION = 4
UI = 4
```

Deep debugging:

```
INSTRUMENTATION = 8–9
UI = 8–9
```

FastAPI / HTTP tracing:

```
HTTP = 2   # request/response lines
HTTP = 3   # + User-Agent and status class
HTTP = 4   # + full header dumps
```

---

## Extending the System

- Add new domains to `diagnostics/debug_config.json`.
- Override levels at runtime with `set_debug_level("DOMAIN", N)` (preferred)
  or by mutating `DEBUG_LEVELS["DOMAIN"] = N` directly.
- Override global settings at runtime with `set_global_setting(key, value)`.
- Use `override=True` in `_dbg()` to bypass level filtering for critical
  messages (e.g., always-visible lifecycle events).
- Use `dbg_timed()` / `dbg_profile()` to measure and log elapsed time inline.
- Use `_adbg()` inside `async def` functions for coroutine-safe logging.
- Use `OcDiagnosticsHandler` to route third-party `logging` output through
  the `oc_diagnostics` domain-and-level system.
- Use `OC_DIAGNOSTICS_CONFIG` to point multiple projects at a shared config.
- Use `UNIFIED_DEBUG_DISABLE_EXCEPTION_HOOK=1` or
  `UNIFIED_DEBUG_DISABLE_STDIO_INTERCEPT=1` when the host environment
  already manages those channels.
- Use `OC_DIAGNOSTICS_MODE=prod` or `"mode": "prod"` in config to suppress
  stream interception and the exception hook in production environments.

---

## Appendix: File Map

```
oc_diagnostics/
    oc_diagnostics/
        __init__.py
        unified_debug.py
        logging_handler.py
        id_instrumentation.py
        instrumentation_middleware.py
        ui_event_logger.py
        fastapi_middleware.py
        instrumentation_selftest.py
        instrumentation_selftest_cli.py
        instrumentation_selftest_app.py
    tests/
        run_tests.py
        Tests_README.md
    tools/
        migrate_to_oc_diagnostics.py
    docs/
        ARCHITECTURE.md
        CHANGELOG.md
        REFERENCE.md
        INTEGRATION_GUIDE.md
        debug_config.example.json
    README.md
    pyproject.toml
```