Streaming Long-Running Jobs
A long-running program can produce thousands of progress events before reporting a useful result. Capturing the entire transcript and interpreting it afterward can consume unnecessary memory and agent context. QuietCli's streaming mode lets a companion update local state as records arrive, then construct its domain answer when native execution has finished.
The supplied Generated-NoisyJob.ps1 (ouvre dans une nouvelle fenêtre) demonstrates this with a Python fixture. It runs two stages, prepare and validate, within one task. Each stage produces progress records and a completion event. The companion counts observed progress, reads the reported completion total, and checks that they agree. Its final answer contains the two stage totals and selected metadata, rather than the full event sequence.
Run the supplied example
From an extracted module or source checkout, with a working Python interpreter:
$moduleRoot = 'C:/tools/QuietCli'
& (Join-Path $moduleRoot 'examples/Generated-NoisyJob.ps1') `
-Python python3 -Records 2000 -MaxOutputBytes 2048-Python accepts an executable name or path. On Windows, a python3 application alias can exist even when Python is unavailable. Point the parameter to an installed interpreter if needed. The verification report identifies the environment actually used; the presence of an alias alone is not evidence that this example ran.
The fixture is synthetic. It exercises the execution contract and parser behavior without claiming to represent a production build's distribution of events, diagnostics or processing cost. Its 2,000-record parameter is a test input, not a performance benchmark.
The published run used the offline container adapter (ouvre dans une nouvelle fenêtre), because host Python was unavailable. It runs the original Python fixture in a pre-existing Podman image and uses the supplied companion's event reduction and completion rules in the host runtime. No image is pulled and networking is disabled. The report labels this adaptation; it does not claim the unchanged native Python companion was executed.
State, records and completion
Invoke-QuietCli has a streaming parameter set with three callbacks or values that matter here:
| Parameter | Responsibility |
|---|---|
-State |
Mutable local state chosen by the companion. |
-OnRecord |
Interpret each record and update that state. |
-Complete |
Validate the final state and return one domain value. |
For each stage, the example keeps the stage name, observed progress count, reported total, completion flag and diagnostic count. It uses the stream identity to distinguish stderr records from stdout events. An stderr record increments a diagnostic counter; a stdout record is parsed as a JSON event.
The event parser is specific to the fixture. A progress event increments Observed. A completion event sets Reported and Done. An unknown event type causes analysis to fail. The callback updates state without emitting arbitrary pipeline values, because the runtime requires OnRecord to remain an internal reduction step.
After execution, -Complete checks the crucial relationship:
if (!$state.Done -or $state.Reported -ne $state.Observed) {
throw 'Missing or inconsistent completion evidence.'
}Only then does it return the stage name, processed total and diagnostic count. This is stronger evidence than assuming that every observed progress event represents a completed operation. The meaning comes from the fixture's event contract and the companion's validation.
Keep execution and preview limits separate
QuietCli reads native stdout and stderr internally. Its record handling allows the companion to keep counters and bounded state instead of retaining all records. That does not make every possible workload constant-memory: the author's state can still grow if it appends every event to an unbounded collection.
There are also explicit runtime limits. MaxRecordCharacters limits an individual streaming record, while document mode uses MaxCaptureCharacters for captured text. The units are characters at those internal boundaries. The final output budget uses UTF-8 bytes. Treating all three numbers as interchangeable would obscure what can be analyzed and what can be transmitted.
An oversized record cannot silently establish an exact count. The runtime records lost records and returns a failed result with RecordLimit when applicable. The companion also avoids analyzing a truncated record as if its prefix were complete JSON. The verification runner exercises this condition separately from final-budget overflow.
A final preview or JSON budget should not stop the intended native analysis prematurely. Conversely, a timeout is an execution condition, not a harmless omission. A partial stream with no completion proof must not become a successful empty result.
Compose stages without hiding failure
Each stage runs through Invoke-QuietStep inside Invoke-QuietTask. The script checks the stage outcome and analysis status before adding its domain data to the completed-stage collection. If preparation fails, it returns from the task body instead of computing a success-shaped result from missing values.
The enclosing task has already recorded the step, so a locally consumed failure remains visible in the final envelope. When both stages complete, the final projection exposes Prepared, Validated, Stages and AnswerComplete. One final budget applies to the resulting task document, including its status information.
The example shows how local reduction and domain validation work together. It does not prove that a particular real-world tool emits the same completion evidence. Adapting the method means inspecting that tool's event format, deciding which records matter, and testing the failure conditions before trusting its totals.
Previous: A Practical Git Companion · Series overview · Next: Compose Tasks and Reuse Scripts