Part 06 — Atomic commit: all-or-nothing disk safety📋

The previous two articles walked the in-memory invariants — strict additivity and deterministic banner output. This article walks the third invariant, the one that turns the in-memory virtFS into the on-disk reality: the commit is atomic. Every emission is written to a .tmp sibling first, every .tmp is renamed to its final path only after every other .tmp has been written, stale generated files are purged only after every rename succeeds. If any single step fails — write, rename, or purge — the run aborts with the disk in a recoverable state. The contract is all-or-nothing: either every emission lands or no emission lands.

The full mechanism is in packages/ts-codegen-pipeline/src/virtfs/commit.ts — three phases, roughly fifty lines, no third-party dependencies. After this article the reader has the complete safety story: the engine cannot mutate user code (Part 04), the engine produces deterministic output (Part 05), the engine commits or aborts (this article). The three properties together are what distinguish multi-stage feedback codegen from the disk-write-per-stage shape that templating tools and one-shot scaffolders use.

Why disk-write-per-stage is wrong for this shape📋

A naive multi-stage pipeline would write each generator's output to disk as it finishes. Stage 0 writes its files; stage 1 reads from disk and writes its files; stage 2 reads from disk and writes; and so on. This works fine when every stage is a one-shot — nx generators and plop essentially do this — but it has three failure modes that become structural problems for a feedback codegen pipeline.

The first failure mode is partial commit on error. If stage 7 (out of ten) throws after stages 0–6 have already written to disk, the operator is left with a half-generated outDir containing files from stages 0–6 but missing files from stages 7–9. There is no clean rollback; the operator either deletes the half-output and re-runs, or hand-reconciles. The disk-write-per-stage model has no concept of transaction; the failure mode is the default.

The second failure mode is stale-file accumulation. If the user deletes an entity, then runs the pipeline, the new pipeline run does not emit the entity's children. But those children are still on disk from the previous run. With disk-write-per-stage, there is no obvious moment to purge — the runner doesn't know which generators would have emitted what. The inevitable workaround is to ask the user to rm -rf outDir/ before each run, which defeats the convergence-in-three-iterations property by re-running every iteration from scratch. Stale-file accumulation is structural to the disk-write-per-stage model.

The third failure mode is observability of intermediate state. With disk-write-per-stage, an external process (a tsc --watch daemon, a file-system watcher, a build cache) can observe the disk during the pipeline run. It will see stage 0's output, then stage 1's, then stage 2's; if any of those intermediate states are inconsistent — for example, a dto.generated.ts exists but the mapper.generated.ts that depends on it has not yet been written — the observer will react to a transient state that should not have been visible. The observer either fights this with debouncing (slow, fragile) or sees flapping errors. The atomic-commit model never lets the observer see the intermediate state.

The library deliberately avoids all three failure modes by deferring every disk write until fixpoint, and then committing in a single atomic phase.

Phase 1 — write every emission to a .tmp sibling📋

The implementation is in commit.ts:89-103:

// Phase 1 — write all .tmp siblings. If any throws, clean up and rethrow.
try {
  for (const file of opts.emissions) {
    const finalAbs = path.join(opts.outDir, file.relPath);
    const tmpAbs = finalAbs + '.tmp';
    await fs.mkdir(path.dirname(finalAbs), { recursive: true });
    await fs.writeFile(tmpAbs, file.contents);
    tmpFiles.push(tmpAbs);
  }
} catch (err) {
  for (const tmp of tmpFiles) {
    await fs.unlink!(tmp).catch(() => undefined);
  }
  throw err;
}

Three observations.

The first is the choice of suffix: .tmp appended to the full final path, not a separate temporary directory. The reason is atomic rename semantics: fs.rename(tmpAbs, finalAbs) is atomic at the filesystem level only when source and destination are on the same filesystem. By placing .tmp next to the final file, the rename is always within the same directory, always within the same filesystem, always atomic. A separate tmp/ directory could, in principle, span filesystems on a poorly-configured host, and the rename would degenerate to a copy-and-delete that an external observer could see in flight.

The second is the failure recovery. If any single write throws — disk full, permission denied, ENOSPC — the catch block iterates tmpFiles (every .tmp written so far) and unlinks them all, swallowing any error from the unlink itself (commit.ts:99-101: await fs.unlink!(tmp).catch(() => undefined)). The disk is left in its prior state — none of the .tmp siblings linger as orphans. The original error is rethrown; the runner will surface it as part of the run's diagnostics.

The third is the absence of writes to the final paths in this phase. After phase 1 succeeds, every emission exists as a .tmp sibling on disk, and zero final paths have been touched. An external observer at this point sees the previous run's outputs untouched, plus a set of .tmp files that are not recognised as generated outputs (the verifier's isGeneratedFileName regex requires .generated. not .generated.ts.tmp). The intermediate state is invisible to consumers.

Phase 2 — atomic rename, all .tmp → final📋

Phase 2 is the commit proper (commit.ts:107-120):

let renamedCount = 0;
try {
  for (const tmpAbs of tmpFiles) {
    const finalAbs = tmpAbs.slice(0, -'.tmp'.length);
    await fs.rename(tmpAbs, finalAbs);
    written.push(finalAbs);
    renamedCount++;
  }
} catch (err) {
  for (let i = renamedCount; i < tmpFiles.length; i++) {
    await fs.unlink!(tmpFiles[i] as string).catch(() => undefined);
  }
  throw err;
}

This is the part that requires the most careful reading.

The renames happen sequentially, in the order of tmpFiles (which is in turn the order of emissions in virtFS). For every tmp, the final path is computed by stripping the .tmp suffix. The rename is atomic at the filesystem level: a consumer of the file system either sees the old finalAbs (if the rename has not happened yet) or the new finalAbs (after the rename completes). The .tmp is never visible as the final file; the final file always either contains the previous run's content or this run's content.

Failure handling here is more delicate than phase 1. If rename i succeeds but rename i+1 fails, the disk is in a mixed state: emissions 0..i are already at their final paths (committed), emission i+1 is still a .tmp, emissions i+2..N do not exist. The recovery code unlinks every .tmp not yet renamed (for (let i = renamedCount; i < tmpFiles.length; i++)), leaving the partial commit in place. This is a deliberate trade-off: a fully-atomic recovery would require copying the previous finalAbs files back, which would itself be a non-atomic multi-step operation susceptible to its own failures. Instead the library accepts that a phase-2 failure leaves the disk in a partially-committed state, with the precondition that the partial state is identical to a state that could have arisen from a successful run on a slightly-different input — every committed file is an emission of the current run, every uncommitted file is the prior run's emission. The disk is consistent (no file contains a half-write) even though it is partially updated.

In practice, phase-2 failures are rare. Renames within a single directory on a single filesystem do not fail except for catastrophic conditions (disk full during rename, which is extraordinarily unlikely after writes already succeeded; permission changes between phase 1 and phase 2; a parallel process deleting the directory). The library handles them defensively but does not optimise for them.

Phase 3 — prune stale generated files📋

Phase 3 is the cleanup of files left over from prior runs whose inputs have been removed (commit.ts:122-133):

const writtenSet = new Set(written);
const purged: string[] = [];
for (const stale of opts.preCommitFiles) {
  if (writtenSet.has(stale)) continue;
  try {
    await fs.unlink!(stale);
    purged.push(stale);
  } catch {
    // already gone — fine
  }
}

The preCommitFiles argument was computed at the start of the run by scanExistingGeneratedFiles — the list of every *.generated.ts file currently on disk under outDir. The written set is the list of every absolute path renamed in phase 2. The diff is the prune set: every pre-existing generated file not covered by a new emission. Those files are unlinked, with the unlink errors swallowed (a stale file that is already gone is a no-op).

The phase ordering matters. The prune happens after phase 2 succeeds, not before. If phase 2 failed, phase 3 does not run — the stale files stay on disk, alongside whatever partial commit phase 2 left, and the next run's scanExistingGeneratedFiles will find them and prune them. The operator's recovery story is "re-run", and the engine is designed to make re-run idempotent at the disk level (the next run will write the same files to the same paths, atomically, and prune whatever is now stale).

The isGeneratedFileName predicate (commit.ts:9-11) is conservative: a file is considered generated if its name contains .generated. (not at the start) and ends in .ts. This catches both <name>.generated.ts and the <name>.scaffold.generated.test.ts shapes used by the TestStubGenerator shipped in the library. It does not catch arbitrary user-named files in outDir; a hand-written outDir/notes.md would not be pruned.

What the operator sees on success and on failure📋

On success, the operator sees: every emission landed at its final path; stale generated files are gone; the run returns { ok: true, iterations: N, diagnostics: [] } to whatever called runFixpoint. The CLI prints a one-line summary; CI captures the exit code zero.

On failure, the operator sees a non-zero exit code and a diagnostic list. There are five distinct failure shapes, each with its own treatment:

• Iteration-level error (SG0020 EmissionDivergence, SG0040 InvalidEmissionPath, SG0050 GeneratorThrew). Phase 1 / 2 / 3 do not run. The runner (run-fixpoint.ts:82-85) returns ok: false before reaching commit. Disk untouched.
• Convergence failure (SG0030 MaxIterationsReached). Same — the loop exits with an error, commit is skipped, disk untouched.
• Phase 1 failure (write to .tmp throws). All .tmps written so far are unlinked. Disk untouched.
• Phase 2 failure (rename throws partway through). The renames already done are not undone (see above); the .tmps not yet renamed are unlinked. Disk in a partially-committed but consistent state — recoverable by re-running.
• Phase 3 failure (unlink throws). Errors are silently swallowed; the run is reported as successful. The stale file on disk will be picked up by the next run's scanExistingGeneratedFiles and pruned then.

The strongest guarantee — disk untouched on iteration-level errors — covers the common case. A bug in a generator does not corrupt the disk. A path-escape attempt does not corrupt the disk. A divergence between two producers does not corrupt the disk. The runner is designed so that the operator's recovery story for these failures is "fix the bug and re-run", with no manual filesystem cleanup.

What atomic-commit does not provide📋

Three properties a reader might expect, and the reasons they are not provided.

First, the commit is not crash-safe across an unexpected process kill. If kill -9 is sent to the runner during phase 2, the renames done so far are committed and the .tmps not yet renamed are leaked. Cleanup-on-startup of the runner does not sweep stale .tmp files — the assumption is that a re-run will rewrite the same .tmps and atomically rename them, overwriting any orphan. A truly crash-safe design would log the intent of phase 2 to a journal before performing it; the library does not, on the grounds that kill -9 mid-rename is rare and that the recovery story (re-run) is fast enough. A consumer that needs crash-safety should add their own journal or filesystem-level transaction layer; the library does not pretend to provide it.

Second, the commit does not coordinate with concurrent runners. Two sourcegen run processes against the same outDir will race; the outcome depends on the OS's filesystem semantics for concurrent renames. The library is single-runner; a consumer running multiple pipelines (in a monorepo, say) is expected to scope each pipeline's outDir to a non-overlapping directory tree. A future revision could add a lockfile-based coordination layer; the current version does not.

Third, the commit is not a transaction across outDirs. If a consumer's pipeline writes to multiple outDirs through multiple runFixpoint calls, each commit is atomic within its outDir, but the combination is not. The all-or-nothing property is per-run, not per-pipeline-of-runs. A consumer who needs multi-outDir atomicity should either fold them into a single runFixpoint call (one configuration, one commit) or accept that intermediate states between runs are visible.

Bridge📋

Part 07 opens the example as curriculum. The next four articles walk the ten generators stage by stage: Part 07 walks the @Module × @Entity × @StateMachine decorator families and the three shared scanners; Part 08 walks the entity-scaffolding quadruple (Repo + DTO + Mapper + Validator); Part 09 walks the FSM-typed dispatch generator pair; Part 10 walks the module-wiring generator pair and closes the registry backward-edge loop. Every behaviour those articles describe assumes the three engine invariants articulated in Parts 04, 05, and this one.

The Feature for this article is FEAT-TSGEN-06 in assets/features.ts. Acceptance criteria: .tmp-rename-purge sequence explained; mid-loop failure leaves disk untouched; pruning of stale generated files explained; contrast with disk-write-per-stage hazards. Each section above maps to one of those ACs.