Skip to main content
Welcome. This site supports keyboard navigation and screen readers. Press ? at any time for keyboard shortcuts. Press [ to focus the sidebar, ] to focus the content. High-contrast themes are available via the toolbar.
serard@dev00:~/cv

Part 08: requirements-test-smells

What it reifies

@frenchexdev/requirements-test-smells is the leanest Tier-1 sibling and the one whose existence is the most opinionated. Its job is to catch what the compliance reporter cannot: tests that exist and declare a @Verifies link, but whose bodies do not actually verify anything. An empty method, a method whose only statement is a console.log, a method that calls only mocks, a method whose expect clauses are tautologies — these are honest gaps the AST can detect without running the tests, and this package is where that detection lives.

The package's package.json description: "Test-smells detector — AST-level honest detection of declarative ↔ behavior gaps in @FeatureTest/@Verifies test methods." The word honest is load-bearing. The package does not claim to catch every dishonest test; it claims to catch the ones a syntactic walk over the AST can identify without ambiguity.

The public surface

src/index.ts is a single line:

// @frenchexdev/requirements-test-smells — Tier-1 analyzer: AST-level honest detection
// of declarative ↔ behavior gaps in @FeatureTest/@Verifies test methods.
export * from './test-smells';

The implementation file exports two entry points and a vocabulary. The function signatures:

import type { FileSystem } from '@frenchexdev/requirements-core/ports';
import type { BindingsManifest } from '@frenchexdev/requirements-scanner';

export type SmellKind =
  | 'empty-body'
  | 'no-assertions'
  | 'mock-only'
  | 'tautological-expect'
  | 'commented-out-only';

export interface TestSmell {
  file: string;
  className: string;
  methodName: string;
  ac: string;
  kind: SmellKind;
  reason: string;
}

export interface ScanTestSmellsOptions {
  testDir: string;
  manifest?: BindingsManifest;
}

export declare function detectTestSmells(
  fs: FileSystem,
  options: ScanTestSmellsOptions,
): Promise<TestSmell[]>;

export declare function scanTestSmells(
  fs: FileSystem,
  options: ScanTestSmellsOptions,
): Promise<TestSmell[]>;

Five smell kinds today, each with its own AST-pattern detector. empty-body matches any decorated method whose body is {}; no-assertions matches any body where the call graph contains no expect(...), assert(...), or framework-equivalent; mock-only matches a body that calls only methods on vi.fn() returns; tautological-expect matches expect(true).toBe(true) and its trivial variants; commented-out-only matches a body whose only non-whitespace tokens are comments.

The function names are duplicated by intent: detectTestSmells returns the raw findings, scanTestSmells (named for parallelism with scanTestBindings) walks the file set and applies detectTestSmells to each. Same behaviour, different aggregation level; the duplication is preserved because callers of either name find what they expect.

Where it sits

Tier 1, sibling of the scanner. Depends on requirements-scanner (it consumes the same AST parser to walk method bodies) and on requirements-requirements (it cross-references findings against the Feature registry). It does not depend on requirements-compliance — the compliance reporter aggregates test-smells into its larger report, but a smell scan can run standalone.

Three things the package must not do:

  • Run the tests. That is requirements-behavioral-check's territory (Part 09). Test-smells is static — it walks the AST and never executes a single line of test code.
  • Pick patterns the AST cannot prove. A test that calls only one helper function that happens to assert internally is not a smell — the AST cannot tell the difference between a helper that asserts and a helper that does not without running the code. Test-smells stays in the narrow zone of structural patterns that are unambiguous.
  • Render anything but typed findings. The output is a TestSmell[] array; the compliance reporter or the CLI decides how to display them.

A concrete call-site

The CLI's compliance subcommand wires test-smells into the larger report:

import { detectTestSmells, type TestSmell } from '@frenchexdev/requirements-test-smells';
import { scanTestBindings } from '@frenchexdev/requirements-scanner';
import { generateReport } from '@frenchexdev/requirements-compliance';
import { fs } from '@frenchexdev/requirements-core/ports';

const manifest = await scanTestBindings(fs, { testDir: 'test', srcDir: 'src' });
const smells: TestSmell[] = await detectTestSmells(fs, {
  testDir: 'test',
  manifest,
});

const report = generateReport({
  bindings: manifest,
  smells,
  features: getFeatureRegistry(),
  requirements: getRequirementRegistry(),
  satisfactions: getSatisfactionLinks(),
});

The compliance reporter accepts the smells array as an optional field; if present, the rendered table flags every method with a smell glyph and the strict mode treats each smell as a violation. The shape stays the same: pure functions, pure data, the CLI is the only place that decides what is fatal.

Why it is its own package

The case is the subtlest in the family.

First, the package admits its own limit. It is named honest detection on purpose, because the maintainer is on record (in feedback_ast_vs_behavioral_grounding.md) that AST inspection moves the declarative lie one level deeper — a smell scan can confirm a test has an expect clause, but cannot confirm that the expect clause is testing the right thing. The package's promise is bounded: it catches only the structural patterns where the AST is definitive. Anything subtler — a test that asserts the wrong invariant, a test that asserts the right invariant against the wrong fixture, a test that asserts on a value the implementation always returns regardless of input — requires running the code. That is what mutation testing is for, and that is the next package.

Keeping test-smells and behavioural-check in separate packages mirrors this epistemic distinction. The user reading a compliance report can tell, looking at the columns, whether a finding came from a static analyzer (cheap, fast, deterministic) or from a mutation run (expensive, slow, statistical). Conflating them would obscure the distinction; separating them keeps the reader honest about which guarantee they have.

Second, the package is consumed independently by IDEs. A future LSP extension wants to flash a red squiggle under an empty @Verifies method as the user types — that requires synchronous AST analysis, no test runner, no Stryker, no mutation budget. Mounting test-smells as a separate package means the LSP depends on it directly without pulling in the mutation runner's @stryker-mutator/* dependencies. The smell-detection workflow is editor-time; the mutation-verdict workflow is CI-time. Two consumers, two latencies, two packages.

Third, the registry of smell kinds is an extension point. The five smell kinds today (empty-body, no-assertions, mock-only, tautological-expect, commented-out-only) are AST-pattern recognisers, but the package exposes the SmellKind discriminator as a union type that downstream packages can extend through declaration merging. A consumer with project-specific patterns — say, "a test that calls our deprecated singleton is a smell" — can register a sixth kind without forking the package. That openness is only possible because the package is small enough to host the union without coupling it to the wider compliance machinery.

The next page covers the sibling that runs the tests: requirements-behavioral-check, the mutation-testing harness that confirms a passing test would have caught the bug.

⬇ Download