Architecture

How the system coordinates agents, enforces quality, and learns

System Overview

The system is configured through six layers, loaded in order of precedence.

Layer Location Loaded Purpose
CLAUDE.md Project root Every session Project identity, folder structure, commands, output organization
Rules .claude/rules/ 6 every session; 7 when a matching file is read Quality gates, invariants, permissions, lifecycle, workflow
References .claude/references/ On demand Domain profile, journal profiles, coding standards
Skills .claude/skills/ When invoked Rich folders — routing logic, templates, gotchas, config. The knowledge layer.
Agents .claude/agents/ When dispatched Lean identity files — role, voice, constraints. The execution layer.
Hooks .claude/hooks/ Event-driven File protection, session guards, post-edit lint, compaction save and restore, hook health

CLAUDE.md and six rule files are always in context: 679 lines, under the 700-line budget that scripts/check_scaffold.py enforces. The other seven rules carry paths: frontmatter and load when Claude reads a matching file (see How Rules Load). Everything else loads on demand. The pre-commit gate is a git hook in .githooks/, not a Claude Code hook; both are covered in Hooks & Automation. For the full layer-by-layer breakdown, see the Customization Guide.

The Skill-Folder Architecture

Skills used to be thin dispatchers — a single SKILL.md file (~100 lines) that routed a command to an agent. All the real knowledge lived inside agent files: checklists, templates, scoring rubrics, reference protocols, packed into monolithic prompts that ran 200–474 lines. Adding knowledge meant editing a massive agent file. Knowledge was locked inside agents, not reusable across contexts. Loading an agent meant loading all its templates even when we needed one.

The architecture inverted. Skills are now rich folders. Agents are lean identity files.

Three-level progressive loading

The system loads context in three stages, each gated by need:

Level Trigger Contents Budget
1 — Metadata Always (skill index) Name, description, argument-hint ~100 tokens
2 — SKILL.md body User invokes the skill Workflow steps, dispatch logic, modes < 5k tokens
3 — Bundled files Agent reads on demand Templates, gotchas, checklists, references Unlimited

Level 1 costs almost nothing — every skill’s frontmatter is visible in the skill index, so the system knows what commands exist without loading their bodies. Level 2 activates only when the user types the slash command. Level 3 loads only the specific files the agent needs for the task at hand.

Skill folder structure

Skills share one layout:

skill-name/
├── SKILL.md          # Routing + dispatch logic (Level 2)
├── gotchas.md        # Known failure points, edge cases
├── templates/        # Reusable artifacts the agent loads
├── references/       # Domain-specific reference material
└── config/           # Machine-readable configuration

Not every skill uses every subfolder. The /write skill has templates/ (paragraph moves, section templates, cleanup patterns) and references/ (notation protocol). The /review skill has templates/ (review checklists, referee report template, disposition pool) and config/ (scoring rubrics). The /strategize skill has templates/ (strategy memo structure, PAP templates, theory memo) and references/ (PAP interview flow). Each skill carries exactly the knowledge its agents need.

The wiring

When a skill dispatches an agent, it tells the agent which templates to read. The agent file says “read the templates provided by the invoking skill.” This separation means:

  • Skills own the WHAT — what to produce, what structure to follow, what pitfalls to avoid.
  • Agents own the HOW — voice, reasoning approach, quality standards.

The same agent can serve multiple skills. The coder-critic reviews output from both /analyze and /review --code, loading different checklists each time. The strategist produces both strategy memos and pre-analysis plans, guided by different templates from the same skill folder.

Gotchas as first-class content

Thirteen of the 18 skills have a gotchas.md — the highest-signal file in the folder. (/coherence, /dashboard, /deepread, /explain, and /paper-format have none.) These are known failure modes, accumulated from real pipeline runs. The strategist’s gotchas warn that “staggered adoption does not automatically mean DiD is the right choice.” The writer’s gotchas note that “the cleanup pass can over-correct domain-specific hedging that’s actually appropriate.” The review skill’s gotchas document when cold-read critics produce false positives.

Gotchas are the fastest path to quality improvement. When a failure recurs, it gets a line in gotchas.md. The agent reads it before execution. The failure stops recurring.

What this means in practice

Adding a new template no longer requires editing a 400-line agent prompt. We create a file in templates/, reference it from SKILL.md, and the agent picks it up on next invocation. Knowledge accumulates in skill folders without bloating agent context. Agents stay lean enough to load fast and focused enough to execute well.

The Permission Registry

Every agent’s capabilities, dependencies, and routing are declared in one registry: .claude/rules/permissions.md. The Orchestrator reads it, and scripts/pipeline.py parses it to check each dispatch. Two other files hold part of the picture. Quality weights live in .claude/skills/new-project/config/quality-gates.json; the registry’s QUALITY_WEIGHT fields mirror them, and scripts/check_scaffold.py fails if the two disagree. The orchestrator’s tools: Agent(...) roster lists the agents it is allowed to dispatch.

Reading an entry

Each entry declares seven fields: PHASE, PARALLEL_GROUP, REQUIRES, PRODUCES, CRITIC, ESCALATION_TARGET, and QUALITY_WEIGHT.

Here is the strategist entry, annotated:

## strategist
- PHASE: Strategy
- PARALLEL_GROUP: strategy
- REQUIRES: quality_reports/literature/{project}/
             OR quality_reports/data-assessment/{project}/
- PRODUCES: quality_reports/strategy/{project}/
    - Required files: strategy_memo.md
    - Required sections: Estimand, Specification, Assumptions,
                         Robustness Plan, Threats
- CRITIC: strategist-critic
- ESCALATION_TARGET: User --- fundamental design question
- QUALITY_WEIGHT: 25% (identification validity)

The REQUIRES field uses OR logic — the strategist can activate after literature review alone, data assessment alone, or both. The PRODUCES field specifies not just the output file but the sections that must appear in it. pipeline.py check-post (see Lifecycle Validation) checks that these sections exist before the pipeline advances.

Adding an agent

Create the agent file in .claude/agents/ and add its entry to permissions.md. Two more edits when they apply: add the agent to the orchestrator’s Agent(...) roster if the orchestrator dispatches it, and to quality-gates.json if it carries a weight. scripts/check_scaffold.py fails on a roster name with no agent file and on a weight that differs between the two files. Step-by-step in the Customization Guide.

The Orchestrator Loop

After a plan is approved, the Orchestrator takes over autonomously. It runs a 5-step loop until quality gates are met or limits are reached.

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#f0eee6', 'primaryTextColor': '#141413', 'primaryBorderColor': '#d1cfc5', 'lineColor': '#d97757', 'secondaryColor': '#faf9f5', 'tertiaryColor': '#e6e3da', 'edgeLabelBackground': '#faf9f5', 'clusterBkg': '#f0eee6', 'clusterBorder': '#d1cfc5'}}}%%
flowchart TD
    START([Plan approved]):::start --> ID
    ID[1. IDENTIFY\nCheck dependency graph]:::step --> DISP[2. DISPATCH\nLaunch worker-critic pairs]:::step
    DISP --> REV[3. REVIEW\nCritic scores artifact]:::step
    REV -->|">= 80, or SOUND / MINOR"| VER[4. VERIFY\nCompile, render, run]:::step
    REV -->|"< 80, or MAJOR / CRITICAL"| FIX[Worker fixes]:::worker
    FIX --> REV
    VER --> SC[5. SCORE\nWeighted aggregate]:::step
    SC -->|>= threshold| DONE([Present summary]):::approved
    SC -->|< threshold| ID
    SC -.->|max 5 rounds| DONE

    classDef start fill:#f0eee6,stroke:#d97757,color:#141413,stroke-width:2px
    classDef step fill:#f0eee6,stroke:#87867f,color:#141413,stroke-width:2px
    classDef worker fill:#faf9f5,stroke:#d97757,color:#141413,stroke-width:1px
    classDef approved fill:#f0eee6,stroke:#4d7c5a,color:#141413,stroke-width:2px

Step 1: IDENTIFY

The Orchestrator reads permissions.md and checks which agents have their REQUIRES satisfied. Agents activate when dependencies are met, not by phase number. If you already have data and a draft paper, you can enter at Strategy or Peer Review — the Orchestrator checks REQUIRES, not sequence.

Step 2: DISPATCH

Worker-critic pairs launch for all agents whose REQUIRES are satisfied. Agents in the same PARALLEL_GROUP run concurrently — for example, the librarian and explorer both belong to the discovery group and run simultaneously. Before dispatch, the Orchestrator runs pipeline.py check-pre (see Lifecycle Validation).

Step 3: REVIEW

The paired critic evaluates the worker’s artifact against its rubric. If the score is below 80, the worker revises and the critic re-reviews. The strategist-critic gives a verdict instead of a score: SOUND or MINOR ISSUES passes, MAJOR ISSUES or CRITICAL ERRORS sends the memo back. This inner loop runs up to 3 rounds per pair (see Worker-Critic Protocol).

Step 4: VERIFY

Compile LaTeX, run scripts, check that output files exist at expected paths. If verification fails, the worker fixes and re-verifies (max 2 attempts).

Step 5: SCORE

Run scripts/score.py, which computes the weighted aggregate from the research journal (see Scoring Protocol). If the score meets the gate threshold, present the summary to the user. If not, loop back to Step 1 to identify the blocking components.

Limits

Limit Value What happens
Worker-critic rounds 3 Escalate per ESCALATION_TARGET
Overall loop rounds 5 Present with remaining issues
Verification retries 2 Report failure to user

The system never loops indefinitely.

Modes

Pipeline mode (/new-project): Full orchestration with dependency graph, parallel dispatch, and quality gates.

Standalone mode (/review, /write, /analyze, etc.): Skip dependency checks, dispatch the requested agent(s) directly, return results.

Simplified mode (R scripts, explorations): Plan, implement, run, verify quality >= 80. No multi-agent reviews.

Lifecycle Validation

Validation runs at two points: before an agent is dispatched and after it completes. The checks are a script, scripts/pipeline.py, which reads each agent’s permissions.md entry and the research journal; the Orchestrator calls it and acts on the result. This implements a fail-fast principle — missing inputs are caught before work begins, missing outputs are caught before advancement.

PRE-dispatch checks

python3 scripts/pipeline.py check-pre <agent> verifies the agent’s REQUIRES: paths exist and directories are non-empty, critic scores and verdicts in the research journal meet their thresholds, and OR / AND clauses combine as written. Exit 1 means the agent is not dispatched.

POST-completion checks

python3 scripts/pipeline.py check-post <agent> verifies that the PRODUCES artifacts exist and contain their required sections. Exit 1 means the pipeline does not advance; the Orchestrator re-dispatches the agent with the gaps noted. A critic’s result gates the next agent, so it is checked by that agent’s check-pre.

Each condition prints PASS, FAIL, or MANUAL. MANUAL marks a condition only judgment can settle (for example, “Research idea or research spec”); the Orchestrator decides it and says how.

What failure looks like

check-pre coder
  FAIL   `quality_reports/strategy/minwage/strategy_memo.md` missing or empty
  FAIL   strategist-critic has no verdict in the research journal
Cannot dispatch coder: `quality_reports/strategy/minwage/strategy_memo.md` missing or empty

No agent launches with missing inputs. No agent advances with missing outputs. The Orchestrator reports the failure to the user along with the skill that produces the missing input (here, /strategize).

Worker-Critic Protocol

Every agent that creates an artifact has a paired critic that evaluates it. For the full list of pairs and what each reviews, see Agents.

The 3-round loop

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#f0eee6', 'primaryTextColor': '#141413', 'primaryBorderColor': '#d1cfc5', 'lineColor': '#d97757', 'secondaryColor': '#faf9f5', 'tertiaryColor': '#e6e3da', 'edgeLabelBackground': '#faf9f5', 'clusterBkg': '#f0eee6', 'clusterBorder': '#d1cfc5'}}}%%
flowchart TD
    W[Worker creates artifact]:::worker --> C[Critic reviews cold-read]:::critic
    C -->|score >= 80| A([APPROVED]):::approved
    C -->|score < 80| F[Worker fixes issues]:::worker
    F --> C2[Critic re-reviews cold-read]:::critic
    C2 -->|score >= 80| A
    C2 -->|score < 80, round < 3| F
    C2 -->|score < 80, round = 3| ESC([ESCALATE]):::escalate

    classDef worker fill:#f0eee6,stroke:#d97757,color:#141413,stroke-width:2px
    classDef critic fill:#f0eee6,stroke:#788c5d,color:#141413,stroke-width:2px
    classDef approved fill:#f0eee6,stroke:#4d7c5a,color:#141413,stroke-width:2px
    classDef escalate fill:#f0eee6,stroke:#b33d3d,color:#141413,stroke-width:2px

The strategist-critic rules by verdict rather than score: SOUND or MINOR ISSUES approves the memo, MAJOR ISSUES or CRITICAL ERRORS sends it back.

Cold-read enforcement

On every round, the critic receives only: the artifact, its scoring rubric, the severity level, and the content invariants from Rules & Invariants. It does not receive what round this is, what the worker struggled with, prior critic reports, or the research journal. This prevents the critic from softening its evaluation because it knows the worker has already tried twice.

Round-aware feedback (what to fix and how) flows from the Orchestrator to the worker, never through the critics.

Separation of powers

Critics never create artifacts. Creators never self-score. Every critic’s and referee’s tool list is read-only (Read, Grep, Glob), so a critic cannot write to scripts/ or paper/ in the first place. If a creator reports its own score, the Orchestrator discards it and dispatches the critic.

A critic who fixes its own findings has incentive to find only fixable issues. Separation keeps criticism honest.

Escalation routing

When a pair hits 3 rounds without converging, the Orchestrator reads the agent’s ESCALATION_TARGET from permissions.md. Some examples from the registry:

Pair Escalation Target Rationale
coder + coder-critic strategist-critic Strategy memo may not be implementable
writer + writer-critic Orchestrator Needs structural rewrite, not polish
strategist + strategist-critic User Fundamental design question
theorist + theorist-critic User Proof-level disagreement

Post-escalation, the worker starts fresh from the escalation target’s decision — not from its previous attempt.

Dual-Critic Dispatch

For artifacts that gate major phase transitions, the Orchestrator dispatches two critics with different evaluation lenses. This catches blind spots that a single-dimension review misses.

Gate artifacts

Artifact Primary Critic Secondary Lens
Strategy memo strategist-critic (identification) coder-critic (implementability)
Main results (code) coder-critic (code quality) strategist-critic (strategy alignment)
Final manuscript writer-critic (prose quality) strategist-critic (claims-strategy match)

Synthesis rules

A critic passes the artifact with a score of 80 or more; the strategist-critic passes it with a verdict of SOUND or MINOR ISSUES.

  • Both pass: Artifact advances.
  • One fails: Worker fixes issues from the failing critic. Both critics re-evaluate.
  • Both fail: Escalate per the pair’s ESCALATION_TARGET.

Independence

Neither critic sees the other’s report. The Orchestrator synthesizes both scores independently. This is the same cold-read principle applied across critics, not just across rounds.

Scope

Dual-critic dispatch activates only for gate artifacts in the full pipeline (/new-project). Standalone skill invocations (/review, /write, etc.) use single critics.

Pipeline State and Session Recovery

The state file

Nobody maintains quality_reports/pipeline_state.json by hand. python3 scripts/pipeline.py status derives it from the research journal and the registry: which agents have run, with rounds and latest result; which can be dispatched now; which are waiting and on what; the overall score; and the first gate that fails. The Orchestrator reruns it after each agent completes and before each phase transition. A trimmed example:

{
  "project": "minwage",
  "phase": "Strategy",
  "last_updated": "2026-10-15T14:15:35",
  "agents_completed": [
    { "agent": "strategist-critic", "score": null, "verdict": "MINOR ISSUES",
      "rounds": 1, "report": "quality_reports/strategy/minwage/strategy_review.md",
      "timestamp": "2026-10-14 11:20" }
  ],
  "agents_in_progress": [],
  "agents_pending": ["librarian", "explorer", "theorist", "data-engineer", "coder"],
  "agents_waiting": [
    { "agent": "writer", "missing": "coder-critic has no score in the research journal" }
  ],
  "overall_score": 85.0,
  "components": { "identification": 85.0, "code": null, "manuscript": null },
  "blocked_by": "pr gate: overall 85.0 < 90",
  "derived_from": "quality_reports/research_journal.md via scripts/pipeline.py status"
}

Session recovery

After compaction or a resume, the post-compact-restore.py SessionStart hook prints the active plan, the pipeline state, and the last SESSION_REPORT.md entry into the new context (see Hooks & Automation). Claude reads the files it names and checks git status before continuing. In a fresh session, start from pipeline_state.json and the most recent plan in quality_reports/plans/.

Execution traces

After a multi-agent skill (/new-project, /analyze, /write full), python3 scripts/pipeline.py traces tabulates rounds, first-pass and final score per agent, and flags HIGH-PERF, FRICTION, and USER ESCALATION. The Orchestrator saves that table with a short reading of the patterns to quality_reports/traces/YYYY-MM-DD_trace.md.

Relationship to the research journal

The research journal is the record: append-only, one entry per agent invocation, and the input to both score.py and pipeline.py. A critic result that isn’t logged there doesn’t count. pipeline_state.json is a view of the journal, regenerated whenever it is needed.

Scoring Protocol

Weighted aggregation

scripts/score.py computes the score; the Orchestrator reports its output and never recomputes it. The script reads each critic’s latest entry in the research journal and the weights in .claude/skills/new-project/config/quality-gates.json, the single source for weights and thresholds:

\[\text{Overall} = \frac{\sum_{i \in \text{scored}} w_i \times \text{score}_i}{\sum_{i \in \text{scored}} w_i}\]

where \(w_i\) is the component’s weight and \(\text{score}_i\) is the critic’s latest score (0–100). Current weights:

Component Weight Source
Literature coverage 10% librarian-critic
Data quality 10% explorer-critic
Identification validity 25% strategist-critic (verdict: SOUND 95, MINOR ISSUES 85, MAJOR ISSUES 65, CRITICAL ERRORS 40)
Theory (when present) 20% theorist-critic
Code quality 15% coder-critic
Referee: domain 12.5% domain-referee
Referee: methods 12.5% methods-referee
Manuscript polish 10% writer-critic
Replication readiness 5% verifier (pass/fail: 100 or 0)

The strategist-critic’s verdict maps to a number for this aggregate only; everywhere else it gates by verdict.

Missing components

If a component has not been scored, score.py excludes it, renormalizes the remaining weights, and lists what it excluded. The theory weight applies only when the theorist ran (papers with formal theory sections), so an applied paper’s weights sum to 100% and a theory paper’s are renormalized from 120%.

Gate thresholds

Gate Overall Per component Also required Enforced by
Commit >= 80 — Passes with a notice while nothing is scored /tools commit runs score.py --gate commit
PR >= 90 — — score.py --gate pr before opening a PR
Submission >= 95 >= 80 Every non-conditional component scored; understanding gate (INV-26) passes /submit final runs score.py --gate submission and verify.py --mode submission

The understanding gate is pass/fail and never enters the score: score.py runs explainer_status.py --gate as part of the submission gate. The git pre-commit gate does not check the score; a new project has no critic scores, and a gate that fails every early commit gets bypassed. See Quality Gates for the coherence gate and the full rules.

Severity gradient

Critics calibrate harshness by phase. The Orchestrator includes the severity level in the critic’s prompt. For the full deduction scaling table, see Rules & Invariants.

Phase Severity Stance
Discovery Low Encouraging — early ideas need space
Strategy Medium Constructive — sound identification, but suggest alternatives
Execution High Strict — bugs are costly at this stage
Peer Review Maximum Adversarial — simulates real referees
Presentation Medium-high Professional — polished but advisory

Scripts Enforce the Gates

The model used to do this arithmetic itself, working from prose that disagreed with itself about weights and thresholds: it added up weighted scores, globbed for required files, and estimated hedge counts by reading. Wherever the inputs fully determine the output, the check is now a script in scripts/. Agents call it and report the result; they keep only the checks that need judgment.

Script What it decides Called by
score.py Weighted aggregate and the commit / PR / submission gates, from the research journal and quality-gates.json Orchestrator, /tools commit, /submit final
pipeline.py check-pre / check-post against the registry; status writes the pipeline state; traces flags patterns Orchestrator
verify.py The verifier’s mechanical checks: compile, file integrity, output freshness, script inventory, dependency files, master run, exhibit cross-reference, INV-26 verifier, /submit final
paper_coherence.py Cross-references, citation keys, claim numbers against table cells, abstract length (INV-5), Table 1 columns (INV-24) /coherence, pre-commit gate (advisory)
style_metrics.py The writer-critic’s countable deductions: AI vocabulary, hedging, em dashes, sentence length /review, which hands the counts to the writer-critic
check_scaffold.py Broken references, missing hook files, weight disagreements between permissions.md and quality-gates.json, malformed frontmatter, guards without "onFailure": "block", the 700-line always-loaded budget pre-commit gate, /tools upgrade

The git pre-commit gate. .githooks/pre-commit, turned on once per clone with scripts/install-hooks.sh, runs on every commit, including commits made outside Claude. It blocks on check_scaffold.py errors, on failing tests when hooks, scripts, or the registry changed, and on HIGH lint findings in staged analysis scripts. Paper-coherence findings are printed as advice. It does not check the quality score. To bypass it, run SKIP_GATE=1 git commit ... and say why in the commit message.

The scripts are tested. scripts/test_scripts.py runs each enforcement script against a throwaway fixture project; scripts/test_hooks.py covers the Claude Code hooks, which are documented in Hooks & Automation.

Plans and Approval

Plans and approval are reserved for work that is expensive to get wrong: a multi-agent pipeline run (/new-project, /analyze, /write full, /revise) or a change across many files. These cost real time and tokens. Medium and small tasks proceed without a written plan.

The protocol

  1. Write the plan — approach, files to change, how it will be verified; save it to quality_reports/plans/YYYY-MM-DD_description.md
  2. Get approval — the user approves before work starts
  3. Implement — for pipelines, the Orchestrator takes over via the dependency-driven loop

/new-project keeps plan mode as its pipeline gate. For a large, ambiguous task, record the answers to clarifying questions as a requirements spec (MUST / SHOULD / MAY; CLEAR / ASSUMED / BLOCKED). Plans survive context compression because they are on disk, and the SessionStart hook points back to the active one.

Simplified mode

For standalone R scripts, simulations, and explorations: plan, implement, run, verify quality >= 80. No multi-agent reviews.

“Just do it” mode

When the user says “just do it” or “handle it”: skip the final approval pause, auto-commit if python3 scripts/score.py --gate commit passes, but still run the full verify-review-fix loop.

Learning Loop

After completing a multi-agent skill, the Orchestrator reviews the execution trace for three pattern types.

Pattern detection

Pattern Signal Example
HIGH-PERF Agent-critic pair scored >= 90 on first pass Strategist produced a clean memo that passed on round 1
FRICTION Pair hit 3 strikes Coder-critic kept failing on numerical discipline — prompt may need refinement
ESCALATION Question escalated to user Could the system have resolved it with better context or rules?

How it works

The Orchestrator surfaces findings as “Suggested Learnings” at the end of the pipeline summary:

### Suggested Learnings
- [HIGH-PERF] Strategist memo structure matched coder needs ---
  consider templating the robustness section
- [FRICTION] Coder failed numerical discipline 3 rounds ---
  add float-comparison examples to coding standards
- [ESCALATION] User resolved data-source ambiguity ---
  add data source preferences to domain profile

The Orchestrator never auto-appends to memory. It presents suggestions. The user approves or rejects. Approved learnings are saved via the auto-memory system.

Learning promotion

When a pattern appears across 3 or more projects, it is a candidate for promotion to a template or rule file — moving from project-specific memory to the system’s permanent configuration. This is how the architecture improves over time without accumulating noise.