providers.openai_compat.conformance Namespace Reference

Classes
class	RoundResult
class	ConformanceResult

Functions
RoundResult	_chat_round (Any client, str model)
RoundResult	_tools_round (Any client, str model)
ConformanceResult	run_conformance (Any client, Lane lane, str|None model=None)
Any	build_live_client (Lane lane)
str	format_report (ConformanceResult result, Lane lane, str model)
argparse.ArgumentParser	build_arg_parser ()
int	main (Sequence[str]|None argv=None)

Variables
dict	PROBE_TOOL
str	_CHAT_PROMPT = "Reply with exactly the word: pong"
str	_TOOLS_PROMPT = "What is the current temperature in Rotterdam? Use the tool."

Detailed Description

Lane conformance runner — the evidence behind a ``verification`` promotion.

Purpose
-------
``python -m providers.openai_compat.conformance <lane> --live`` exercises
one streamed chat round and one single-tool ReAct round against a lane's
real endpoint and prints a verdict. A lane's catalogue ``verification``
field is promoted from ``"chat"`` to ``"tools"`` only by a HUMAN who ran
this and recorded the date in that row's ``notes`` (plan 215 P4) — the
runner never edits the catalogue.

Live network access requires the explicit ``--live`` flag. Without it the
runner refuses to touch the network and exits non-zero, which is what
makes it safe to exercise from the test suite: the suite drives it with an
injected mock client and never opens a socket.

Responsibilities
----------------
- :func:`run_conformance` — the two rounds against an injected client.
- :func:`build_live_client` — construct a real client for a lane, or fail
  with the env-var NAME the operator must set.
- :func:`format_report` — render a :class:`ConformanceResult` as ASCII.
- :func:`main` — argv parsing, the ``--live`` gate, and the exit code.

Diagnostics
-----------
Domain: OC_CHAT.CLIENT
Levels:
    L2 — lifecycle events
    L3 — semantic details
    L4 — deep tracing

Contracts
---------
- Without ``--live`` the runner NEVER constructs a network client and
  exits 2. This is a refusal, not a silent skip.
- ``--live`` with an unset key env var exits 2 naming the variable; it
  never falls back to another lane's key.
- Exit codes: 0 both rounds passed; 1 a round failed; 2 refused to run
  (no ``--live``, unknown lane, or missing key).
- The runner reports ONLY what it observed. A tools round that produced
  no tool call is reported as a FAIL with the reason, never softened into
  a pass, and the runner never writes the catalogue.
- No key value is printed, logged, or included in any report line.

Dependencies
------------
- oc_diagnostics (structured debug logging via _dbg)
- providers.openai_compat.catalogue (LANES, Lane and lane_by_name to find the lane under test)
- providers.openai_compat.client (lazy; OpenAICompatClient built for a live run)

Function Documentation

_chat_round()

RoundResult providers.openai_compat.conformance._chat_round ( Any client, str model )

protected

One streamed chat round: assert content arrived.

Definition at line 125 of file conformance.py.

Here is the caller graph for this function:

_tools_round()

RoundResult providers.openai_compat.conformance._tools_round ( Any client, str model )

protected

One single-tool ReAct round: assert a well-formed tool call arrived.

Definition at line 149 of file conformance.py.

Here is the caller graph for this function:

build_arg_parser()

argparse.ArgumentParser providers.openai_compat.conformance.build_arg_parser

(

)

Build the runner's argument parser.

Definition at line 271 of file conformance.py.

Here is the caller graph for this function:

build_live_client()

Any providers.openai_compat.conformance.build_live_client

(

Lane

lane

)

Construct a real client for *lane*, or raise :exc:`SystemExit(2)`.

Reads the key from the env-var NAME the catalogue row advertises. The
key value is never logged, echoed, or stored anywhere in this module.

Definition at line 215 of file conformance.py.

Here is the caller graph for this function:

format_report()

str providers.openai_compat.conformance.format_report	(	ConformanceResult	result,
		Lane	lane,
		str	model )

Render *result* as an ASCII report block.

Definition at line 242 of file conformance.py.

Here is the caller graph for this function:

main()

int providers.openai_compat.conformance.main

(

Sequence[str] | None

argv = None

)

Entry point. Returns the process exit code (0 pass, 1 fail, 2 refused).

Definition at line 295 of file conformance.py.

Here is the call graph for this function:

Here is the caller graph for this function:

run_conformance()

ConformanceResult providers.openai_compat.conformance.run_conformance	(	Any	client,
		Lane	lane,
		str | None	model = None )

Run both rounds for *lane* against an already-built *client*.

The client is INJECTED so the test suite can drive this with a mock and
stay off the network entirely (plan 215 W5).

Definition at line 200 of file conformance.py.

Here is the call graph for this function:

Here is the caller graph for this function:

Variable Documentation

_CHAT_PROMPT

str providers.openai_compat.conformance._CHAT_PROMPT = "Reply with exactly the word: pong"

protected

Definition at line 88 of file conformance.py.

_TOOLS_PROMPT

str providers.openai_compat.conformance._TOOLS_PROMPT = "What is the current temperature in Rotterdam? Use the tool."

protected

Definition at line 89 of file conformance.py.

PROBE_TOOL

dict providers.openai_compat.conformance.PROBE_TOOL

Initial value:

=  {
    "type": "function",
    "function": {
        "name": "get_current_temperature",
        "description": "Return the current temperature for a named city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name, e.g. 'Rotterdam'"},
            },
            "required": ["city"],
        },
    },
}

Definition at line 73 of file conformance.py.