oct.git.commit_observer Namespace Reference

Functions
str	_hash_file (Path path)
list[tuple[str, str]]	_tracked_entries (Path repo_root)
set[str]	_dirty_paths (Path repo_root)
	_make_escape_check (Path repo_root)
dict[str, str]	snapshot_unstaged_tracked (Path repo_root, set[str]|None staged_set=None, *, str method=METHOD_HASH, set[str]|None also_hash=None)
list[str]	diff_snapshots (dict[str, str] before, dict[str, str] after)

Variables
int	_MAX_HASH_BYTES = 10 * 1024 * 1024
str	_OVERSIZE_SENTINEL = "OVERSIZE"
str	_MISSING_SENTINEL = ""
str	_CLEAN_SENTINEL = "CLEAN"
str	METHOD_GIT = "git"
str	METHOD_HASH = "hash"
tuple	OBSERVE_METHODS = (METHOD_GIT, METHOD_HASH)
str	_GITLINK_MODE = "160000"
str	_SYMLINK_MODE = "120000"

Detailed Description

Purpose
-------
Detect silent mutations of tracked-but-unstaged files during the
``oct git commit`` pipeline (B1.0). Snapshots the working-tree state
of every tracked file outside the staged set at the start of the
commit, then compares again right before and right after the
underlying ``git commit`` invocation. Any byte-level change is
reported as either a warning or a hard abort, depending on the
``git.observe_unstaged_mutation`` octrc setting.

Two detection methods produce those snapshots (plan 242). They agree
on every verdict the control exists to produce; they differ only in
what they read to reach it.

``git`` (:data:`METHOD_GIT`)
    Ask git which tracked files differ from the index
    (``git diff --name-only -z``, answered from git's stat cache with
    the racy-git re-hash every ``git commit`` already trusts) and
    content-hash only those, plus the paths named in *also_hash*.
    Every other tracked file carries :data:`_CLEAN_SENTINEL`. About
    0.2 s per pass on a 13k-file tree.

``hash`` (:data:`METHOD_HASH`)
    SHA-256 every tracked-but-unstaged file. Byte-identical to the
    pre-242 behaviour apart from the two enumeration fixes below.
    188 s per pass on the same tree with the OS file cache cold.

The commit flow resolves the method from the octrc key
``git.observe_unstaged_mutation_method`` and always passes it
explicitly; ``git`` is the default when the key is absent. This
function's own parameter defaults to ``hash``, the more thorough of
the two, so that a caller who names no method never silently gets
less observation than it did before plan 242.

Responsibilities
----------------
- Provide :func:`snapshot_unstaged_tracked` - returns a mapping of
  repo-relative path -> sha256 hex digest or sentinel for every
  tracked file *not* in the staged set.
- Provide :func:`diff_snapshots` - returns the list of paths whose
  digest changed (new file, deleted file, or content drift) between
  two snapshots. Unchanged by plan 242; both methods feed it.
- Be defensive: large files (> :data:`_MAX_HASH_BYTES`) are recorded
  with the sentinel digest ``"OVERSIZE"`` so they don't dominate
  commit latency, and a missing/unreadable file maps to ``""`` so the
  diff path treats it consistently.
- Enumerate the index with ``git ls-files -z --stage``, which fixes
  two observation blind spots that were silent before plan 242:

  1. Without ``-z``, ``git ls-files`` quotes and octal-escapes any
     name containing a non-ASCII character (``core.quotepath``
     defaults to true). The observer stat-failed on the quoted
     string, recorded the missing sentinel, and never observed the
     file - a mutation of it was invisible to the control.
  2. Gitlink entries (mode ``160000``, a nested repository) came out
     of ``git ls-files`` looking like files. The observer tried to
     open the directory and logged a permission-denied L1 override on
     every snapshot. They are now skipped by mode, not by luck.

Diagnostics
-----------
Domain: GIT
Levels:
    L1 - errors: file IO failures during snapshot, unparsable index
         records, path escapes
    L2 - lifecycle: snapshot start / snapshot complete
    L3 - details: file count, hashed count, gitlinks skipped
    L4 - deep trace: per-file hash + size

Contracts
---------
- This module is pure: no Click, no subprocess of its own. It uses
  :func:`oct.core.git.git_run` to enumerate tracked files and, under
  :data:`METHOD_GIT`, to ask for the dirty set.
- Snapshot calls must never raise on individual-file IO errors -
  they record the file as unreadable and continue. A failure of git
  *itself* does propagate: the caller's fail-soft wrapper turns the
  observer off for that commit rather than silently switching method,
  which would reintroduce the cost this module exists to remove.
- An unknown *method* is a programming error and raises
  :class:`ValueError`; there is no permissive fallback in either
  direction.
- The hash digest is content-only (no metadata). Renames are detected
  via the path key, not the digest.
- A tracked path that resolves outside *repo_root* is dropped from
  the snapshot rather than hashed, under both methods. Plan 242
  computes that predicate once per distinct parent directory instead
  of once per file (:func:`_make_escape_check`) - the same answer,
  and the difference between a git-native pass costing 0.2 s and one
  costing 2.4 s on a 13k-entry index.
- Residual of :data:`METHOD_GIT`, accepted when the method was ruled
  (FACTORY-01 D3 option (a)): a writer that restores a file's size
  and mtime after writing evades git's stat cache where a SHA-256
  walk would not, and a change visible only through a content filter
  (``core.autocrlf``, a ``.gitattributes`` clean filter) is likewise
  not a difference in git's eyes. Such a writer is adversarial, and
  an adversarial writer can already disable the observer through
  octrc, so it was never in this control's threat model. Run under
  :data:`METHOD_HASH` for a paranoid pass.
- One documented verdict difference between the methods: an oversize
  file that is CLEAN at the pre snapshot is recorded as
  :data:`_CLEAN_SENTINEL` under ``git`` and as
  :data:`_OVERSIZE_SENTINEL` under ``hash``, so a mutation of it is
  flagged under ``git`` and not under ``hash`` (``diff_snapshots``
  cannot compare an OVERSIZE pair). ``git`` is strictly stronger;
  ``hash`` keeps the pre-242 answer by design.

Dependencies
------------
- oct.core.diagnostics (_dbg structured logger)
- oct.core.git (lazy; git_run - the only subprocess path out of this module)
- hashlib (SHA-256 content digests of tracked files)
- pathlib (Path for the repo root and tracked-file paths)

Function Documentation

_dirty_paths()

set[str] oct.git.commit_observer._dirty_paths ( Path repo_root )

protected

Return the repo-relative paths git reports as differing from the index.

``git diff --name-only -z`` compares the working tree against the
index - the same question ``git commit`` itself asks - and
includes deletions. Untracked files are absent by construction.

Definition at line 234 of file commit_observer.py.

Here is the caller graph for this function:

_hash_file()

str oct.git.commit_observer._hash_file ( Path path )

protected

Return the sha256 hex digest of *path*, or a sentinel.

Returns :data:`_OVERSIZE_SENTINEL` when the file is larger than
:data:`_MAX_HASH_BYTES`, and :data:`_MISSING_SENTINEL` when the
file is missing or unreadable. Never raises.

Definition at line 168 of file commit_observer.py.

Here is the caller graph for this function:

_make_escape_check()

oct.git.commit_observer._make_escape_check ( Path repo_root )

protected

Return ``is_inside(mode, rel) -> bool`` for index paths.

Answers exactly the question the pre-242 code asked per file -
"does this tracked path resolve inside *repo_root*?" - without
paying a ``Path.resolve()`` syscall per file. A tracked path can
only land outside the repository if one of its parent directories
is a symlink out, or if its own leaf is one; git names the second
case in the index (mode :data:`_SYMLINK_MODE`), so the first is
resolved once per distinct parent directory and cached, and the
second per entry.

Measured on this monorepo's 13,394-entry index: 2.142 s of
per-file resolves collapses to 0.048 s across 306 directories,
which is what brings a git-native pass inside its budget. Only
:class:`ValueError` is caught, matching the pre-242 behaviour -
an :class:`OSError` still propagates to the caller's fail-soft.

Definition at line 250 of file commit_observer.py.

Here is the caller graph for this function:

_tracked_entries()

list[tuple[str, str]] oct.git.commit_observer._tracked_entries ( Path repo_root )

protected

Return ``[(mode, rel_path)]`` for every entry in the index.

Enumerates with ``git ls-files -z --stage``, whose records are
``<mode> <object> <stage>\\t<path>`` separated by NUL. ``-z`` is
what keeps a non-ASCII path out of ``core.quotepath``'s quoting
and octal-escaping, and ``--stage`` is what makes a gitlink
distinguishable from a blob. Raises whatever
:func:`oct.core.git.git_run` raises - a git failure is the
caller's fail-soft business, not this function's.

Definition at line 202 of file commit_observer.py.

Here is the caller graph for this function:

diff_snapshots()

list[str] oct.git.commit_observer.diff_snapshots	(	dict[str, str]	before,
		dict[str, str]	after )

Return the list of paths whose digest changed between snapshots.

Includes:
  - Paths present in both with different digests (mutation).
  - Paths present in *before* but missing from *after* (deletion or
    unreadable).
  - Paths newly tracked in *after* (rare; would only appear if
    ``git add`` ran during the commit pipeline).

A path with the same :data:`_OVERSIZE_SENTINEL` on both sides is
NOT flagged - we cannot tell whether the content of an oversize
file changed without a full hash. Document this caveat to the
caller via the per-file payload (omitted from the simple list
return; see :func:`diff_snapshots_detail` for the verbose form).

:data:`_CLEAN_SENTINEL` needs no special case: it compares as an
ordinary value, so ``CLEAN`` on both sides is no mutation, and
``CLEAN`` against a digest (a file that entered or left the dirty
set between the snapshots) is one. Plan 242 left this function
unchanged on purpose - both detection methods feed it.

Definition at line 401 of file commit_observer.py.

snapshot_unstaged_tracked()

dict[str, str] oct.git.commit_observer.snapshot_unstaged_tracked	(	Path	repo_root,
		set[str] | None	staged_set = None,
		*	,
		str	method = METHOD_HASH,
		set[str] | None	also_hash = None )

Return ``{rel_path: digest_or_sentinel}`` for tracked-but-unstaged files.

*repo_root* must be the actual git repository root (not the OCT
project root in monorepo layouts). *staged_set* is a set of
repo-relative path strings already in the index; those files are
excluded from the snapshot because their mutations are caller
intent, not silent regression.

*method* selects how each recorded value is produced -
:data:`METHOD_HASH` content-hashes every entry, :data:`METHOD_GIT`
content-hashes only what git reports as dirty (plus *also_hash*)
and records :data:`_CLEAN_SENTINEL` for the rest. See the module
docstring for the trade-off and the accepted residual. An unknown
*method* raises :class:`ValueError`.

*also_hash* is the pre snapshot's set of paths that carried
anything other than :data:`_CLEAN_SENTINEL`. Passing it on the
post snapshot is what keeps a hash-to-hash comparison exact for a
file that was already dirty when the commit started: without it, a
file reverted to its index content between the two snapshots would
leave the dirty set and read as unchanged. Ignored under
:data:`METHOD_HASH`, which hashes everything anyway.

The snapshot includes:
  - Tracked files that are clean (working tree == index).
  - Tracked files modified but not staged (``M`` in
    ``git status -s`` second column).

The snapshot excludes:
  - Files in *staged_set*.
  - Untracked files (``??``).
  - Gitlink entries (mode :data:`_GITLINK_MODE`) and worktree
    symlinks pointing outside *repo_root*.

Definition at line 303 of file commit_observer.py.

Here is the call graph for this function:

Variable Documentation

_CLEAN_SENTINEL

str oct.git.commit_observer._CLEAN_SENTINEL = "CLEAN"

protected

Definition at line 144 of file commit_observer.py.

_GITLINK_MODE

str oct.git.commit_observer._GITLINK_MODE = "160000"

protected

Definition at line 160 of file commit_observer.py.

_MAX_HASH_BYTES

int oct.git.commit_observer._MAX_HASH_BYTES = 10 * 1024 * 1024

protected

Definition at line 132 of file commit_observer.py.

_MISSING_SENTINEL

str oct.git.commit_observer._MISSING_SENTINEL = ""

protected

Definition at line 139 of file commit_observer.py.

_OVERSIZE_SENTINEL

str oct.git.commit_observer._OVERSIZE_SENTINEL = "OVERSIZE"

protected

Definition at line 135 of file commit_observer.py.

_SYMLINK_MODE

str oct.git.commit_observer._SYMLINK_MODE = "120000"

protected

Definition at line 165 of file commit_observer.py.

METHOD_GIT

str oct.git.commit_observer.METHOD_GIT = "git"

Definition at line 148 of file commit_observer.py.

METHOD_HASH

str oct.git.commit_observer.METHOD_HASH = "hash"

Definition at line 151 of file commit_observer.py.

OBSERVE_METHODS

tuple oct.git.commit_observer.OBSERVE_METHODS = (METHOD_GIT, METHOD_HASH)

Definition at line 156 of file commit_observer.py.