3. Collecting Versioned CLI Evidence📋

Scraping creates evidence about a specific binary version's invocation surface. The shared pipeline separates version discovery, binary preparation, help collection and serialization so a consumer supplies tool-specific rules without duplicating orchestration.

Its purpose is reproducibility: know which version was inspected, preserve what it printed and be able to repair interpretation independently.

Discovering and selecting versions📋

Consumers use different authorities. Docker and Git inspect GitHub tags; Compose and Podman use release collectors; Packer and Vagrant have specialized collectors; GitLab CLI uses GitLab releases. WinGet inspects an installed Windows executable.

The collector's result is a candidate list. The runner applies selection such as a minimum version, requested versions, missing outputs or known unavailable versions before it sends work to the pipeline. A release tag is still an acquisition identifier: tag normalization and semantic-version ordering must agree with the binary's naming scheme.

A fetched version list and a help snapshot are separate artifacts. A version appearing in release metadata does not prove that an installation recipe can build it on the selected platform.

Resolving recipes before acquisition📋

An image recipe may change across binary versions. A tool can move between source builds and packaged downloads, change archive layout or require a newer compiler. IDesignImagePlanResolver exists to describe those changes explicitly.

A DesignImagePlan models dependency bases and version images. The runner rejects configuring a fixed plan and a resolver simultaneously. It resolves selected recipes and prepares required dependency bases before the worker phase. Preparation and image commands have their own cache and ownership rules.

Recipe selection is a Design-time operation. Runtime IBinaryResolver chooses the executable used by an application; it does not resolve a scraping image recipe.

Composing the middleware📋

Vagrant's current Design entry point composes a version-image stage, container stage and scraper. This is an excerpt of the existing pipeline construction; surrounding collector, logger and output-directory configuration belongs to its Program.

var pipeline = new DesignPipeline()
    .UseVersionImage()
    .UseContainer()
    .UseScraper("vagrant", parserFactory: parser, helpFlag: "-h")
    .Build();

var reparsePipeline = new DesignPipeline()
    .UseCachedHelp()
    .UseScraper("vagrant", parserFactory: parser, helpFlag: "-h")
    .Build();

Middleware changes a VersionContext and then calls the next stage. The container stage installs a RunHelp callback that executes inside the selected container, using a snapshot of the Design environment. The scraper stage uses that callback and a parser factory to create the tree.

The same scraper can consume cached help because its dependency is the help callback, rather than a concrete container API.

Two concurrency dimensions📋

Version-level parallelism determines how many versions enter acquisition concurrently. Scrape-level parallelism determines help-query concurrency within a version. Both default to four in the current runner, and both must be positive.

The runner uses an unbounded channel for the version work list with a configured number of workers. That is distinct from the bounded runtime process-output channel described in execution. An unbounded work queue does not imply unlimited worker creation.

Container process throttling adds another pressure boundary. Effective resource demand depends on version workers, concurrent help queries and the recipe's own compiler or package manager. Multiplying two concurrency settings can produce more work than expected.

Start a real collection with explicit conservative limits and measure CPU, memory and completion time on that environment. No benchmark in this documentation establishes a universal optimal worker count.

Cache identity and ownership📋

The system tracks image plans, version images, active containers and help paths. Reusing a dependency base avoids repeated installation work; cached help avoids binary preparation entirely.

Image locks protect shared acquisition resources from simultaneous builds. A lock must be held through its actual resource lifetime; merely seeing a lock filename is insufficient evidence of ownership. WindowsBinary uses its own executable locking rather than container image locking.

Cache validity still depends on recipe identity and inputs. A new installation recipe should not be described as equivalent merely because the binary version string stayed the same. Consult the image resolver tests when changing how recipes are identified or reused.

Failure, cancellation and cleanup📋

Middleware uses try/finally to release acquired resources, and the runner tracks active resources for cleanup. Cancellation must reach workers and process execution while cleanup has enough independence to release owned resources.

Fail-fast controls scheduling after a failure. It does not turn unfinished versions into successful snapshots. Failure logs and known-missing tracking should remain distinguishable from complete command trees.

WinGet deliberately disables marking failed versions as known missing: a local failure to execute the installed binary is not evidence that a historical release does not exist.

Reparse is another acquisition path📋

In reparse mode the runner discovers cached help versions and uses the replay pipeline. Image resolution, base preparation and live help execution are bypassed. WinGet's offline replay also bypasses executable resolution, version detection and install/uninstall lifecycle callbacks.

This is the normal loop for help-parser maintenance: change interpretation, replay identical evidence, inspect the JSON diff and test generated commands. Reacquire only when the evidence itself is incomplete or a new binary version is needed. The next chapter describes the tree and cache formats.

Source snapshot📋

Source inspection: 2 October 2026. These implementation paths are relative to Net/FrenchExDev in the local ecosystem checkout. The v2 changes are largely uncommitted; these paths identify inspected code, without claiming that public branch links contain it.

• BinaryWrapper/src/FrenchExDev.Net.BinaryWrapper.Design.Lib/DesignPipelineRunner.cs
• BinaryWrapper/src/FrenchExDev.Net.BinaryWrapper.Design.Lib/DesignImagePlan.cs
• BinaryWrapper/src/FrenchExDev.Net.BinaryWrapper.Design.Lib/ResolvedDesignImagePlans.cs
• BinaryWrapper/src/FrenchExDev.Net.BinaryWrapper.Design.Lib/DesignPipelineExtensions.cs
• Vagrant/src/FrenchExDev.Net.Vagrant.Design/Program.cs

Previous · Series index · Next