Retro

Instruction files grow by accretion. Every rule was added after someone watched an agent do the wrong thing, and nobody measures whether the rule works afterwards. Retro closes that loop. It records when the guardrails get broken, measures what each session costs in tokens, and proposes concrete edits to the instructions, hooks, and skills as an OpenSpec you approve or reject line by line.

Nothing leaves the machine. Nothing is applied automatically.

What it records

A Claude Code hook parses the transcript Claude Code already keeps on disk and appends metadata to .coograph/signals.jsonl (gitignored). It runs at SessionEnd, and also every time the session compacts. That second trigger is what makes a long session legible: a session that runs for a day reaches its end once, late, or never if it is killed, and compaction is the only event it fires repeatedly. Re-capturing is safe, since a session’s records are replaced rather than appended. warn-scope.py and block-generated.py append a violation each time they fire. Every rule hook also appends a decision: what it did (warned, blocked or suppressed) and the id of the tool call it did it to.

Each violation carries the moment it happened, taken from the message that caused it, not the moment the capture ran.

Each record holds tool names, counts, rule ids, repo-relative file paths, a hash and program name for shell commands, timestamps, and token usage. It never holds prompt text, assistant text, tool output, file contents, or full commands. The allow-list is enforced in code, and a sentinel test fails if a marker string planted in every part of a transcript reaches the signals file.

Detectors

Heuristic signals are never sole evidence for a change.

The defect detector is the one that watches for bugs rather than broken rules. A change that type-checks, passes review and ships is invisible to every other detector; git is the only place it surfaces, as a fix landing on something recently changed.

Decisions and outcomes

A count of violations says a rule was broken. It does not say whether the warning did anything. The first retro on coograph itself found scope at 23 warnings in one session with no way to tell the warnings that were acted on from the ones that were noise, and no-new-deps reading “escalate to a blocking hook” on one real event plus two verification probes.

So every rule hook now records what it decided, and the capture records what followed. The decision is written at the moment the hook fires, with the id of the tool call. At session end the transcript is joined to it by that id, never by timestamp, and an outcome is written with four fields: proceeded (the call ran), corrected (the next thing the user said matched a correction pattern), reconciled (something later answered the warning; rule-specific, and null when the answer is not observable, as with a dependency approval), and repeated (the same rule fired again later in the session).

That evidence is what moves a rule from a warning hook to a blocking one. A hook-warn rule over its threshold escalates only when it has at least deterministic_events outcomes and at least escalate_ignored_rate of them were ignored, meaning the rule fired again and nothing reconciled it. Otherwise the report shows hold: no_outcomes (the hook has not run under decision records yet) or hold: warnings_change_behaviour (warnings are followed by a fix, or never repeat). A held rule is watched, not changed. corrected comes from heuristic patterns and is shown, never used as a gate.

Outcomes only count from a rule’s outcomes_since date (optional; new projects get it from the seed, existing ones add it to rules.json by hand, see MIGRATION.md), the last change to the hook’s behaviour. Until 2026-10-06 the warning hooks printed to stderr, which the model never sees, so outcomes from before that date describe warnings nobody read; they must not push a rule toward blocking. last_changed still marks any edit to a rule, a rewording included, and never discards evidence.

The report gains a Decisions table: per rule, how often the hook warned, blocked or suppressed, how many outcomes exist, and the proceeded, corrected, reconciled and ignored rates.

Verification probes, the nested claude -p sessions the retro skill starts to prove a hook fires, are real sessions and used to count as real violations. rules.json now carries ignore_session_prefixes, seeded with the probe prefix. A session whose id starts with one of them is captured but left out of every count, and the report says how many it left out.

The OpenSpec gate gets its first enforcement in the same change. openspec-gate-warn.py warns once per session at the moment the second distinct source file is about to be edited with nothing in the session having touched openspec/changes and no active change directory present. It uses the detector’s own classification and the same check for an active change, so the two cannot drift. It warns, it does not block, and its outcome records whether an OpenSpec followed.

Per-tool support

Only Claude Code exposes transcripts and lifecycle hooks. Other tools get the analyzer and the skill but no capture.

How you find out

You do not have to remember a step. Every Claude Code session start prints one line, in the same place the code-graph status already appears:

[retro] 12 sessions captured, 3 rules over threshold (graph-first 11x), run /coograph-retro

When the instruction files are over budget a second line follows, whether or not any session was captured: [layout] N instruction file(s) over budget, see python3 .github/layout/layout.py --budget, or, when small edits cannot close the gap:

[layout] STRUCTURAL: apps/web/AGENTS.md over twice its budget (+5 more), run /coograph-docs-restructure

When nothing crossed a threshold it says so. When Retro is not enabled but the project has ten or more archived OpenSpec changes, it says run /coograph-retro to bootstrap.

coograph-archive and coograph-apply also ask, once, at the end of a change: Run /coograph-retro now? (n sessions since last retro). Default is no. The agent never runs a retro unprompted.

Day one, not day twenty

Claude Code keeps every past transcript under ~/.claude/projects/<slug>/, where <slug> is the absolute project path with every character outside A-Z a-z 0-9 replaced by - (for example C--paul-code-app or -home-paul-app). Init asks whether to read them, so the first report has real sessions in it immediately:

python3 .claude/hooks/capture-signals.py --backfill ~/.claude/projects/<slug> --cwd .

Projects that never enabled Retro but have at least ten archived changes can bootstrap by running /coograph-retro. The skill seeds the registry, asks for the transcript directory, backfills, and continues as a normal run.

The report

python3 .github/retro/retro.py --report

writes .coograph/retro/report.md. It opens with a plain-language paragraph, then tables: every rule against its threshold with an “escalate to” column (a rung, or a hold: reason when a warning hook has no evidence of being ignored), a Decisions table, path clusters, build retries, tokens per session with a before / after split around the most recent rule change, workflow adherence (editing sessions that also ran a review or verify skill), the instruction layout per tier with a structural verdict, gotcha activity (how often each entry was shown, and whether its command still failed and was retried), and archive statistics. Sessions matching ignore_session_prefixes are left out and counted once as ignored. Deterministic Python, standard library only, no model involved.

The proposal

/coograph-retro reads the report and writes openspec/changes/<date>-retro-<n>/ with a proposal, a spec, tasks, and ready patches. The change types:

• edit rule: reword an existing rule when the evidence shows ambiguity
• add rule: a recurring pattern no rule covers
• new instruction file: a .github/instructions/<name>.instructions.md scoped with applyTo to the directory where the violations cluster
• new hook: a prose rule that is still violated becomes a warning hook; a warning hook whose warnings are ignored becomes a blocking one
• prune rule: a non-hard prose rule nobody has tripped in ten sessions
• structural: the instruction files are too far over budget for small edits; the one proposal is to run /coograph-docs-restructure
• add / prune gotcha: a build that kept failing on the same command becomes a GOTCHAS.md entry; an entry whose paths match no file is removed
• escalate gotcha: a gotcha that keeps being shown while its command still fails and is retried gets a prose proposal for a lint rule or hook that makes the trap impossible

Every change opens with three plain sentences, what happened, why it matters, and what changes, followed by an evidence block whose numbers come only from the report. You approve, then /coograph-apply applies it like any other change.

Rules of the loop

• A prose rule that is still being violated is escalated to a hook. It is never reworded louder; the data says that does not work.
• A warning hook becomes a blocking hook on ignored outcomes, never on counts alone. A warning followed by a fix, or never repeated, is a warning that works.
• Heuristic signals are never sole evidence.
• Over the instruction token budget, every added rule is paired with a pruned one.
• Retro may target its own skill, its detectors, and its hooks, but cannot change thresholds or disable a detector without an explicit Loosen: task in the proposal.
• Every generated hook carries a provenance header. Every retro proposal ends with a rollback task.

Episodes

Thresholds count episodes, not sessions. An episode is one session on one calendar day.

A session id cannot carry a threshold on its own. Someone who works in long sessions produces one id per several days, so a rule needing “three sessions” is unreachable no matter how often it breaks; someone who restarts constantly crosses it on noise. Counting session-days makes the two styles comparable, and means a rule broken on three different days inside one long session reaches its threshold, which is exactly the case the session count used to hide. The report shows both figures, so three short sessions still look different from one long one.

Thresholds

All of them live in .github/retro/rules.json and can be tuned per project. The session-named keys count episodes.

What it is not

Retro is not recursive self-improvement in the model sense. No weights change, no model trains itself. It is the instruction-layer version: the system observes its own failures and proposes changes to its own scaffolding, with a human at the gate. Whether rules keep improving across many cycles is unproven; the first retro usually finds most of the value. The full detector reference, registry format, and tests are in the repo at .github/retro/README.md.