Handoff — Closing the Requirements -> Code -> Tests -> Proof Loop📋

Pitch📋

In a TypeScript project (custom SSG), we built a system that proves at every commit that every business requirement is implemented, tested, and covered — with no manual declaration whatsoever. It rests on 3 pillars: a requirements DSL in TypeScript, test decorators that bind each assertion to a requirement, and an AST scanner that automatically infers the traceability graph. Vitest v8 coverage closes the loop by verifying that the code actually traversed is the code claimed.

The problem we solve📋

The classic requirements -> code -> tests cycle has a gap: who verifies that the tests actually test what they claim to test? And who verifies that every requirement has at least one test?

The usual approaches:

• Manual traceability matrices (Excel, Jira links) -> drift, nobody maintains them
• Tags in tests (@requirement REQ-123) -> declarative, unchecked, untyped
• Coverage alone -> tells you the code is executed, not that it verifies the right requirement

We want a system where:

1. Every requirement is typed (the compiler rejects typos)
2. Every test declares which requirement it verifies (via typed decorator)
3. A scanner proves that the test actually calls code linked to the requirement
4. Coverage confirms that this code is actually executed
5. A quality gate blocks if a requirement is not covered

System architecture📋

Layer 1 — Requirements DSL (native TypeScript)📋

// requirements/features/navigation.ts
export abstract class NavigationFeature extends Feature {
  readonly id = 'NAV';
  readonly title = 'SPA Navigation + Deep Links';
  readonly priority = Priority.High;

  /** Clicking a TOC item loads the page with a fade transition. */
  abstract tocClickLoadsPage(): ACResult;

  /** Browser back button restores previous page. */
  abstract backButtonRestores(): ACResult;

  // ... 6 more ACs
}

Every Feature is an abstract class. Every AC (Acceptance Criterion) is an abstract method. The TypeScript compiler guarantees:

• The AC exists (otherwise @Verifies<F>('typo') won't compile — keyof F)
• The feature is exported and registered

No YAML, no JSON, no Jira. The requirement IS TypeScript.

Layer 2 — Test decorators📋

// test/unit/spa-nav-state.test.ts
@FeatureTest(NavigationFeature)
class SpaNavTests {
  @Verifies<NavigationFeature>('tocClickLoadsPage')
  'navigating to a page transitions to loading state'() {
    const machine = createSpaNavMachine(fakeDeps);
    machine.navigate('/blog/foo.html');
    expect(machine.getState().phase).toBe('loading');
  }
}

@FeatureTest(F) — binds the test class to a Feature. Auto-registers methods with vitest (no manual describe/it).

@Verifies<F>('acName') — binds a test method to a specific AC. The keyof F type guarantees at compile-time that the AC exists.

@Exclude() — marks an internal helper (setup, teardown) so the scanner ignores it.

The runtime registry collects all { feature, ac, testClass, testMethod } at execution. The AST scanner reads them statically.

Layer 3 — AST Scanner (the centrepiece)📋

scripts/lib/test-bindings-scanner.ts — a TypeScript walker that:

1. Parses every *.test.ts and *.spec.ts
2. Finds @FeatureTest(F) classes and @Verifies<F>('ac') methods
3. Walks the body of each @Verifies method and collects all identifiers referenced in value position (not types, not declaration names, not .name in PropertyAccess)
4. Resolves each identifier via the file's import map -> repo-relative path
5. Follows transitively local helpers (function setup() { ... }) and class helpers (this.makeFixture()) via a visited set (cycle-safe)
6. Filters to files in src/** or scripts/** (the code under test, not external deps)
7. Aggregates into a BindingsManifest: { featureId -> { acName -> [{ file, symbol }] } }

Result: for every AC of every Feature, we know which src symbols the test actually calls. Not what a human declared — what the AST proves.

The transitive walker📋

The key case that made naive scanners useless:

// The test calls setup(), not createMachine() directly
function setup() {
  return createSpaNavMachine(fakeDeps);  // <- imported symbol
}

@Verifies<NavigationFeature>('tocClickLoadsPage')
'navigates correctly'() {
  const m = setup();  // <- local call, not an import
  m.navigate('/page');
  expect(m.getState().phase).toBe('loading');
}

A CallExpression-only scanner sees setup() (local, not bindable) and expect() (vitest, filtered out). It misses createSpaNavMachine.

Our scanner:

1. Collects top-level helpers (function setup() { ... }) into a map name -> body
2. Collects class helpers (this.makeFixture()) into a map name -> body
3. When a referenced identifier matches a helper, walks the helper's body recursively
4. A visited set prevents infinite loops

Result: 485 "empty" methods (first naive scanner) -> 11 (transitive scanner). The remaining 11 are artefact tests (CSS regex, HTML inspection) that don't traverse any TypeScript symbol — legitimate.

Layer 4 — Compliance Report📋

scripts/compliance-report.ts consumes the inferred manifest and vitest v8 coverage to produce a report:

✓ NAV       SPA Navigation + Deep Links    8/8 ACs (100%)  src 100% (1 file)  impl 4/8 TU + 4/8 E2E
✓ THEME     Theme Switching                5/5 ACs (100%)  src 100% (1 file)  impl 5/5 TU
✓ VIS       Visual Regression              4/4 ACs (100%)  src 100% (1 file)  impl 1/4 TU + 3/4 E2E

Features: 97 active
Acceptance criteria: 829/829 ACs covered (100%)
Total tests linked to ACs: 2812 (2757 unit + 55 e2e)
Runtime coverage warnings: 0
Unbound features: 0
Orphan source files: 0
Quality gate: PASS

Three columns:

• ACs — percentage of ACs with at least one @Verifies test (green = 100%)
• src — min of v8 line coverage% on the files claimed by the feature (green = 100%)
• impl — TU / E2E split: how many ACs are bound to src symbols (TU, green) vs only verified by Playwright (E2E, yellow)

Layer 5 — Quality Gate📋

The quality gate FAILs if:

• A critical AC is not covered by at least one @Verifies test
• Overall coverage drops below 80%

The vitest threshold enforces separately:

• src/lib/**/*.ts: 98% statements, 95% branches, 98% functions, 99% lines
• scripts/lib/compliance-core.ts: 99% lines

The closed loop📋

Feature (TypeScript)
   |
   | abstract methods = ACs
   v
@Verifies<Feature>('ac') on test method
   |
   | the test calls src symbols
   v
AST Scanner walks the body + transitive helpers
   |
   | resolves imports -> src files
   v
BindingsManifest: feature -> ac -> [file, symbol]
   |
   | cross-referenced with vitest v8 coverage
   v
Compliance Report: each AC is
   (a) verified by a test ✓
   (b) the test calls src code ✓
   (c) that code is covered by v8 ✓
   |
   v
Quality Gate: PASS / FAIL

No link is declarative-only. Every link is verified:

• Feature -> AC: TypeScript type system (keyof F)
• AC -> Test: @Verifies decorator (scanned statically)
• Test -> Code: AST call graph (transitive import resolution)
• Code -> Execution: vitest v8 line coverage

Hexagonal + SOLID: why it matters📋

Coverage is worthless if the code under test has dependencies on the real world (fs, DOM, network). A test that imports createMachine() but createMachine() internally calls fs.readFileSync() -> the test is no longer a unit test.

Solution: rigorous hexagonal architecture.

// src/lib/external.ts — all ports
export interface FileSystem {
  readFile(path: string, encoding: 'utf8'): Promise<string>;
  writeFile(path: string, content: string): Promise<void>;
  exists(path: string): Promise<boolean>;
  // ...
}

export interface Logger {
  info(msg: string, ...args: unknown[]): void;
  warn(msg: string, ...args: unknown[]): void;
  error(msg: string, ...args: unknown[]): void;
}

// scripts/lib/validate-md-links-core.ts — pure core
export function createLinkValidator(deps: { fs: FileSystem; logger: Logger }) {
  // ... zero imports from 'fs' or 'console'
}

// scripts/validate-md-links.ts — thin shell (~30 lines)
import { promises as fsp } from 'fs';
const realFs: FileSystem = {
  readFile: (p, enc) => fsp.readFile(p, enc),
  // ...
};
createLinkValidator({ fs: realFs, logger: console }).run();

The coverage instrumented by vitest only covers the cores (scripts/lib/**, src/lib/**). The shells are excluded. Result: 99.75% coverage on pure code, not IO wrappers.

Numbers📋

Friction points resolved📋

1. "The scanner doesn't see helpers"📋

Problem: 523 @Verifies methods "empty" because the scanner only followed direct CallExpressions. Solution: transitive walker + class helpers + bare identifier references. 523 -> 11.

2. "Coverage doesn't write"📋

Problem: vitest v8 doesn't flush the coverage report when stdout is saturated. Cause: build-js.ts was printing ✓ src/foo.ts -> js/foo.js for every entry compiled during tests. Solution: inject a Logger port -> tests pass a silent logger -> no more flooding -> coverage flushes.

3. "Manual bindings over-declare"📋

Problem: a human writes rightClickOpensPalette: createPaletteMachine in a .bindings.ts. The test @Verifies('rightClickOpensPalette') actually calls routeThemeButtonClick(), not createPaletteMachine(). The manual binding is conceptually true but factually wrong — the test does NOT verify the palette machine. Solution: delete all manual bindings. The AST is more honest.

4. "E2E tests don't bind to anything"📋

Problem: a Playwright test @Verifies<NavigationFeature>('tocClickLoadsPage') does page.goto('/') + page.click('.toc-item'). No src symbol in the call graph. Solution: distinguish TU (symbol resolved) and E2E (test exists but no symbol binding) in the report. impl 4/8 TU + 4/8 E2E — honest, not a failure.

5. "Who tests the tester?"📋

The AST scanner itself is a feature (TEST-BINDINGS-INF) with 23 ACs and 52 tests. The compliance report is a feature (REQ-TRACK) with 14 ACs. The system tests itself via its own decorators. No infinite meta-regress: at some point the TypeScript compiler is the final judge (if @Verifies<F>('typo') doesn't compile, it's over).

Going further📋

What the system does NOT do (yet)📋

• Mutation testing — coverage proves the code is executed, not that the assertions are relevant. A test expect(true).toBe(true) passes coverage. Stryker would be the next stage.
• AC completeness — the system verifies that every declared AC is tested, but not that the declared ACs cover all behaviour. That's a human judgement.
• Cross-feature dependencies — the scanner treats each feature independently. A change in THEME can break NAV if both share a state machine, but the system doesn't know that. The event topology scanner (scripts/scan-event-topology.ts) partially covers this case.

Reference files for the next Claude📋

• HANDOFF-BIG-BANG.md — technical summary of the refactor session
• scripts/lib/test-bindings-scanner.ts — the AST scanner (500 lines, commented)
• scripts/lib/compliance-report-core.ts — the compliance report
• requirements/decorators.ts — the 3 decorators (@FeatureTest, @Verifies, @Exclude)
• requirements/base.ts — the base Feature class
• src/lib/external.ts — the hexagonal ports
• test/unit/CLAUDE.md — test conventions
• vitest.config.js — coverage include/exclude + thresholds

Article tone📋

Continental, technical, first person. Not a tutorial — a field report on an architectural choice and its consequences. Show the problem (the gap in the cycle), the solution (the AST scanner), the friction (the 5 points above), and the result (the numbers table). Cite real code, not toy examples.