Skip to main content
Welcome. This site supports keyboard navigation and screen readers. Press ? at any time for keyboard shortcuts. Press [ to focus the sidebar, ] to focus the content. High-contrast themes are available via the toolbar.
serard@dev00:~/cv

The Problem with Docker from .NET

"I counted 47 distinct Process.Start() calls across 3 projects before I started this. Every single one was a string-concatenation liability waiting to detonate."


The Breaking Point

I want to show you the exact code that broke. Not a contrived example -- the actual pattern I found in a deployment tool that had been running in production for two years:

public async Task<string> RunContainer(string image, string name, int memoryMb)
{
    var args = $"run -d --name {name} --memory {memoryMb}m --restart always {image}";

    var process = Process.Start(new ProcessStartInfo
    {
        FileName = "docker",
        Arguments = args,
        RedirectStandardOutput = true,
        RedirectStandardError = true,
        UseShellExecute = false
    });

    await process!.WaitForExitAsync();
    return (await process.StandardOutput.ReadToEndAsync()).Trim();
}

This worked for two years. Then someone upgraded Docker from 20.10 to 23.0 on the CI server. The --memory flag still exists -- that's not what broke. What broke was the return value. Docker 23 changed the output format for docker run in certain terminal modes, and the code downstream that expected a bare container ID now got a container ID followed by a warning about cgroup v2 migration. The Trim() call didn't help. The downstream code tried to docker exec into a1b2c3d4\nWARNING: ... and got:

Error response from daemon: No such container: a1b2c3d4\nWARNING:

Nobody caught it until a deploy failed at 2am on a Saturday. The fix was a one-liner -- split on newline, take the first line. But the debugging took three hours because the error message pointed at docker exec, not at docker run. The root cause was invisible: unstructured text flowing between unrelated code paths with no type to constrain it.

That was the last straw, but it wasn't the only straw. I've orchestrated enough infrastructure from .NET -- Docker, Compose, Packer, Vagrant, kubectl -- to know that this class of failure is endemic to the "call CLI tools with string arguments" approach. The Docker case is just the most painful because Docker has the most flags (over 2,400 across all commands and versions), the most version churn (6 major releases in 4 years with breaking changes), and the most complex output formats (legacy builder vs BuildKit, table vs JSON vs custom Go templates).

Let me walk you through the full catalog of pain.


Four Categories of Pain

I went through every Process.Start("docker", ...) call in the monorepo and categorized the failures. Not just my code -- all of it. Helper scripts, deployment tools, CI utilities, test infrastructure. Every place where C# code constructs a Docker command string and hands it to a process.

The 47 calls broke down as follows:

docker run / docker container run:     14 calls
docker build / docker image build:      9 calls
docker ps / docker container ls:        6 calls
docker exec:                            5 calls
docker inspect:                         4 calls
docker compose up:                      3 calls
docker compose down:                    2 calls
docker stop / docker rm:                2 calls
docker network create/connect:          2 calls
                                       ──
Total:                                 47 calls

Of those 47 calls, 31 had at least one latent bug: a flag that could be misspelled without detection, a version-sensitive flag used without version checking, stdout parsed with assumptions about format, or a string value that should have been validated against a known set.

They fall into four buckets, and every .NET developer who has ever called Docker from code will recognize at least three of them.

Diagram
The raw Process.Start pipeline every untyped Docker call follows — strings go in, strings come out, and in between sits a process boundary where the compiler cannot help.

This is the pipeline every untyped Docker interaction follows. Strings go in, strings come out, and in between there's a process boundary where the compiler can't help you. Every failure I'm about to describe lives somewhere in this pipeline.


Category 1: Flag Typos Discovered at Runtime

This is the one that should embarrass us. We write --detatch instead of --detach, and we don't find out until the code runs:

public async Task RunDetached(string image)
{
    var args = $"run --detatch --name myapp {image}";

    var process = Process.Start(new ProcessStartInfo
    {
        FileName = "docker",
        Arguments = args,
        RedirectStandardOutput = true,
        RedirectStandardError = true,
        UseShellExecute = false
    });

    await process!.WaitForExitAsync();

    if (process.ExitCode != 0)
    {
        var stderr = await process.StandardError.ReadToEndAsync();
        // stderr: "unknown flag: --detatch"
        // But this only happens at RUNTIME
        throw new Exception($"Docker run failed: {stderr}");
    }
}

The error message from Docker is clear enough:

unknown flag: --detatch
Did you mean --detach?

But here's the thing -- this code compiled. It passed code review (the reviewer didn't notice the typo either). It passed unit tests (because who mocks Process.Start for flag spelling?). It only failed when it ran against an actual Docker daemon. In CI if you're lucky. In production if you're not.

And --detatch is the easy case. Docker catches it and gives you a suggestion. Here are the typos I found across real codebases:

// All of these compile. All of them fail at runtime.
var detached  = "--detatch";       // --detach
var interact  = "--interative";    // --interactive
var privilege = "--priviliged";    // --privileged
var environ   = "--enviroment";    // --env or --environment (neither exists -- it's --env)
var network   = "--net";           // works, but deprecated alias for --network
var memory    = "--mem";           // doesn't exist -- it's --memory
var publish   = "--publish-port";  // doesn't exist -- it's --publish or -p

Seven distinct typos, all found in production code across three projects. The --net one is particularly insidious: it works today because Docker maintains backward compatibility, but it's a deprecated alias. One day it will emit a warning. Eventually it might be removed. You'll never know until it happens.

Now look at what this becomes with a typed API:

public async Task RunDetached(string image)
{
    var command = new DockerContainerRunCommandBuilder()
        .WithDetach(true)      // Compiler knows this exists
        .WithName("myapp")     // Compiler knows this exists
        .WithImage(image)      // Compiler knows this exists
        .Build();

    // .WithDetatch(true)  <-- CS1061: 'DockerContainerRunCommandBuilder'
    //                         does not contain a definition for 'WithDetatch'
}

The typo becomes a compile error. Not a runtime error, not a CI error, not a 2am page -- a red squiggle in your editor before you even save the file. IntelliSense shows you the valid options. The compiler enforces them.

And it goes deeper than just flag names. Consider flag values:

// Raw: any string is accepted. All of these compile and run.
var args1 = "run --restart always nginx";       // Valid
var args2 = "run --restart unless-stopped nginx"; // Valid
var args3 = "run --restart on-fail nginx";       // INVALID -- it's "on-failure"
var args4 = "run --restart allways nginx";       // INVALID -- typo

// Typed: the value is an enum.
var command = new DockerContainerRunCommandBuilder()
    .WithRestart(RestartPolicy.Always)         // Valid
    .WithRestart(RestartPolicy.UnlessStopped)  // Valid
    // RestartPolicy.OnFail doesn't exist -- compiler error
    // RestartPolicy.Allways doesn't exist -- compiler error
    .Build();

The same pattern applies to --log-driver (json-file, syslog, journald, gelf, fluentd, awslogs, splunk, etwlogs, gcplogs, logentries, local, none), --network mode (bridge, host, none, container:{id}), --isolation (default, process, hyperv), and every other flag that accepts a constrained set of values. In the raw approach, each one is a string. In the typed approach, each one is an enum or a typed union. The compiler knows the valid values. You don't have to memorize them.

This is not a revolutionary idea. We've had this for decades in every other domain: database columns, HTTP methods, configuration keys. But somehow, when it comes to infrastructure tooling, we collectively decided that string concatenation was good enough. It's not.


Category 2: Version Drift

This one is worse than typos because the failures are silent.

I had a deployment script that used --platform linux/amd64 to force multi-arch builds. It worked on every developer machine because everyone was running Docker Desktop 4.x (Docker Engine 24+). Then CI ran it against Docker 18.09 -- the version pinned in the company's base image:

public async Task BuildImage(string dockerfile, string tag)
{
    var args = $"build --platform linux/amd64 -t {tag} -f {dockerfile} .";

    var process = Process.Start(new ProcessStartInfo
    {
        FileName = "docker",
        Arguments = args,
        RedirectStandardOutput = true,
        RedirectStandardError = true,
        UseShellExecute = false
    });

    await process!.WaitForExitAsync();

    if (process.ExitCode == 0)
    {
        Console.WriteLine($"Built {tag} successfully");
        // But was --platform actually applied?
        // On Docker 18.09: no. Silently ignored.
        // On Docker 19.03+: yes.
        // On Docker 20.10+: yes, and BuildKit handles it natively.
    }
}

The truly dangerous behavior: on some Docker versions, unknown flags are silently ignored. The command succeeds. The exit code is 0. The image builds. But the flag you thought was controlling platform selection did nothing. You've just built an amd64 image on your arm64 CI runner, tagged it, pushed it, and deployed it. The crash happens 30 minutes later when the container starts on an amd64 node and the binary is actually arm64.

Here's a timeline of flags that appeared, changed behavior, or were deprecated across Docker versions:

// Docker 18.09 - baseline
// --platform does NOT exist for 'docker build'
// --cgroupns does NOT exist
// --pull accepts "always" but not "missing" or "never"

// Docker 19.03
// --platform added to 'docker build' (experimental)
// --gpus added (requires nvidia-container-toolkit)
// BuildKit becomes opt-in via DOCKER_BUILDKIT=1

// Docker 20.10
// --cgroupns added (host/private)
// BuildKit becomes default
// --platform is no longer experimental
// --ssh added for build secrets

// Docker 23.0
// Compose V2 becomes default (docker compose vs docker-compose)
// --provenance and --sbom added to build
// Some warning formats change

// Docker 24.0
// Legacy builder deprecated
// --build-arg behavior changes with BuildKit
// --load and --push stabilized

// Docker 25.0
// --annotation added to build
// --attest added
// containerd image store becomes option

That's six major versions with breaking or silent changes. If your code uses any of these flags, it behaves differently depending on which Docker version is installed. And you have no way to know at compile time.

The typed alternative:

public async Task BuildImage(string dockerfile, string tag)
{
    var command = new DockerImageBuildCommandBuilder()
        .WithPlatform("linux/amd64")   // [SinceVersion("19.03.0")]
        .WithTag(tag)
        .WithFile(dockerfile)
        .WithPath(".")
        .Build();

    // If the detected Docker version is 18.09:
    // throws OptionNotSupportedException:
    //   "WithPlatform() requires Docker >= 19.03.0, but the
    //    detected version is 18.09.0. The --platform flag was
    //    added in Docker 19.03.0."
}

The [SinceVersion("19.03.0")] attribute is generated by the source generator based on the scraped help output across 40+ Docker versions. When Docker 19.02 doesn't have --platform in its docker build --help output but 19.03 does, the generator emits the annotation. At runtime, the builder checks the detected Docker version before adding the flag. The failure is explicit, immediate, and actionable -- not silent and delayed.

The same mechanism handles deprecated flags:

var command = new DockerContainerRunCommandBuilder()
    .WithNet("bridge")  // [DeprecatedSince("25.0.0", Replacement = "WithNetwork")]
    // Emits compiler warning:
    //   "WithNet() is deprecated since Docker 25.0.0.
    //    Use WithNetwork() instead."
    .Build();

Every flag in the generated API carries its version metadata. The compiler and the runtime work together to prevent version drift from becoming silent failures.

The version metadata isn't hand-maintained. It's computed automatically by the source generator. During the design phase, I scrape docker build --help from Docker 18.09, 19.03, 20.10, 23.0, 24.0, 25.0 (and every minor version in between -- 40+ versions total). The generator diffs the command trees: if --platform appears in version 19.03 but not in 18.09, the generated WithPlatform() method gets [SinceVersion("19.03.0")]. If --squash disappears in version 25.0, WithSquash() gets [UntilVersion("24.0.0")]. No manual tracking. No changelog reading. The binary is the source of truth.

// Generated code (simplified) -- you never write this by hand
public sealed class DockerImageBuildCommandBuilder
{
    [SinceVersion("19.03.0")]
    public DockerImageBuildCommandBuilder WithPlatform(string platform)
    {
        VersionGuard.Require("19.03.0", _detectedVersion, "--platform");
        _platform = platform;
        return this;
    }

    [SinceVersion("20.10.0")]
    [UntilVersion("25.0.0")]
    public DockerImageBuildCommandBuilder WithSquash(bool squash = true)
    {
        VersionGuard.RequireRange("20.10.0", "25.0.0", _detectedVersion, "--squash");
        _squash = squash;
        return this;
    }

    [SinceVersion("24.0.0")]
    public DockerImageBuildCommandBuilder WithProvenance(string mode)
    {
        VersionGuard.Require("24.0.0", _detectedVersion, "--provenance");
        _provenance = mode;
        return this;
    }
}

The VersionGuard checks happen at build time (when you call .Build() on the builder), not at process execution time. If your Docker is 18.09 and you call WithPlatform(), you get a clear exception before any process is spawned. The error message tells you exactly which version introduced the flag, what version you have, and what to do about it.


Category 3: Unstructured Stdout Parsing

This is the category where I found the most code and the most bugs. Parsing Docker's text output is a rite of passage for .NET infrastructure tooling, and it's a rite that never stops hurting.

Here's what docker ps output looks like:

CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS                    NAMES
a1b2c3d4e5f6   nginx:latest   "/docker-entrypoint.…"   2 hours ago      Up 2 hours      0.0.0.0:80->80/tcp       web
f6e5d4c3b2a1   postgres:16    "docker-entrypoint.s…"   3 hours ago      Up 3 hours      0.0.0.0:5432->5432/tcp   db

And here's the code I found parsing it:

public List<ContainerInfo> ParseDockerPs(string stdout)
{
    var lines = stdout.Split('\n', StringSplitOptions.RemoveEmptyEntries);
    var containers = new List<ContainerInfo>();

    // Skip header line
    for (var i = 1; i < lines.Length; i++)
    {
        var line = lines[i];

        // Parse fixed-width columns by position
        // CONTAINER ID is 0-14, IMAGE is 17-31, ...
        // Wait, the widths change based on content length.

        // Fine, use regex instead
        var match = Regex.Match(line,
            @"^(\S+)\s+(\S+)\s+""([^""]*)""\s+(.*?)\s+(Up|Exited|Created)[\s\S]*?\s+(\S+)$");

        if (match.Success)
        {
            containers.Add(new ContainerInfo
            {
                Id = match.Groups[1].Value,
                Image = match.Groups[2].Value,
                Command = match.Groups[3].Value,
                Status = match.Groups[5].Value,
                Name = match.Groups[6].Value
            });
        }
        // If match fails: silently skip the container.
        // No logging. No error. Just gone.
    }

    return containers;
}

This code has at least four bugs:

  1. The column widths are not fixed. Docker formats output based on terminal width. A container with a long name shifts every column to the right. The regex assumes a specific layout that changes based on data.

  2. The "COMMAND" column is truncated. Docker truncates long commands with . If the command contains a double quote, the regex breaks because it expects "..." boundaries.

  3. The STATUS column has multiple formats. Up 2 hours, Up 2 hours (healthy), Exited (0) 3 hours ago, Created, Restarting (1) 5 seconds ago. The regex captures Up or Exited or Created but not Restarting, and it loses the health status and exit code.

  4. The PORTS column is variable width and can be empty. A container with no published ports has nothing in that column. A container with multiple port mappings has a comma-separated list. The regex doesn't account for either.

And this was the working version. The previous version used string.Split(' ') and positional indexing. It broke every time someone used an image name with a space-containing tag.

Here's the build output parsing, which is even worse:

public async IAsyncEnumerable<string> StreamBuildLayers(Process buildProcess)
{
    string? line;
    while ((line = await buildProcess.StandardOutput.ReadLineAsync()) != null)
    {
        // Legacy builder format:
        // Step 1/12 : FROM mcr.microsoft.com/dotnet/sdk:9.0
        // ---> a1b2c3d4e5f6
        // Step 2/12 : WORKDIR /src

        // BuildKit format:
        // #1 [internal] load build definition from Dockerfile
        // #2 [internal] load .dockerignore
        // #3 [1/8] FROM mcr.microsoft.com/dotnet/sdk:9.0@sha256:abc...
        // #3 sha256:abc... 52.43MB / 112.84MB 23.1s

        // Which format are we getting? Depends on:
        // - Docker version
        // - Whether DOCKER_BUILDKIT=1 is set
        // - Whether the Dockerfile has syntax directive
        // - Terminal type (progress=plain vs progress=auto vs progress=tty)

        if (line.StartsWith("Step "))
        {
            // Legacy format
            var stepMatch = Regex.Match(line, @"Step (\d+)/(\d+) : (.+)");
            if (stepMatch.Success)
            {
                yield return $"[{stepMatch.Groups[1]}/{stepMatch.Groups[2]}] {stepMatch.Groups[3]}";
            }
        }
        else if (line.StartsWith("#"))
        {
            // BuildKit format -- maybe
            // But '#' could also be a comment in certain output modes
            var bkMatch = Regex.Match(line, @"#(\d+) \[(.+?)\] (.+)");
            if (bkMatch.Success)
            {
                yield return $"[BK-{bkMatch.Groups[1]}] {bkMatch.Groups[3]}";
            }
            // Silently drops progress lines, error lines, cache hit lines...
        }
        // Lines that match neither pattern: silently dropped.
    }
}

This is the kind of code that works 80% of the time and fails in ways that are impossible to debug. When a build fails and the error message was on a line that didn't match either regex, the developer sees "build failed" with no output. The error was captured by Docker, written to stdout, and then silently discarded by our parsing code.

The docker inspect case is almost comical. Docker returns perfectly structured JSON, but because the rest of the codebase was already using raw stdout parsing, I found code that was regex-parsing JSON:

public string GetContainerIp(string containerId)
{
    var process = Process.Start(new ProcessStartInfo
    {
        FileName = "docker",
        Arguments = $"inspect {containerId}",
        RedirectStandardOutput = true,
        UseShellExecute = false
    });

    var json = process!.StandardOutput.ReadToEnd();

    // Yes, someone regex-parsed JSON.
    var match = Regex.Match(json, @"""IPAddress"":\s*""(\d+\.\d+\.\d+\.\d+)""");
    if (match.Success)
        return match.Groups[1].Value;

    // Falls back to... an empty string. Not null. Not an exception.
    // An empty string that propagates through three more method calls
    // before causing a "cannot connect to host: " error with no host.
    return "";
}

I won't even enumerate all the ways this breaks. The IPAddress field appears multiple times in docker inspect output (once per network the container is attached to). The regex returns the first one, which may or may not be the one you want. If the container has no IP (it's using host networking), the field is "" and the regex doesn't match because there are no digits.

And then there's the failure propagation. That empty string return becomes the input to another method:

public async Task WaitForService(string containerId)
{
    var ip = GetContainerIp(containerId);  // Returns "" on failure
    var url = $"http://{ip}:8080/health";  // url = "http://:8080/health"

    // HttpClient.GetAsync("http://:8080/health") throws:
    //   "An invalid request URI was provided. Either the request URI
    //    must be an absolute URI or BaseAddress must be set."
    //
    // The error message says nothing about Docker, nothing about
    // the container IP, nothing about the regex. It's a URL parsing
    // error three method calls away from the root cause.

    var client = new HttpClient();
    var response = await client.GetAsync(url);
    // Never reaches here
}

The empty string propagates silently through string interpolation, producing a malformed URL, which produces an HTTP error that has no connection to the original problem. I've debugged this exact chain. It took 45 minutes to trace the HTTP error back to the regex failure back to the container not having an IP on the expected network. With typed code, the failure would have been a null or an Option<IPAddress>.None at the point where the IP was retrieved -- not three layers of indirection later.

Now the typed alternative:

// Typed command execution with structured parsing
var result = await executor.ExecuteAsync<ContainerListEvent, ContainerListResult>(
    new DockerContainerListCommandBuilder()
        .WithAll(true)
        .WithFormat("{{json .}}")
        .Build(),
    new JsonLineOutputParser<ContainerListEvent>(),
    new ContainerListResultCollector());

// result.Containers is List<ContainerListEntry>
foreach (var container in result.Containers)
{
    Console.WriteLine($"{container.Id}: {container.Image} [{container.State}]");
    Console.WriteLine($"  Ports: {string.Join(", ", container.Ports)}");
    Console.WriteLine($"  Networks: {string.Join(", ", container.Networks.Keys)}");
}
// Build output as typed events
await foreach (var evt in executor.StreamAsync<BuildEvent>(
    new DockerImageBuildCommandBuilder()
        .WithTag("myapp:latest")
        .WithFile("Dockerfile")
        .WithProgress(ProgressType.Plain)
        .WithPath(".")
        .Build(),
    new DockerBuildOutputParser()))
{
    switch (evt)
    {
        case BuildStepEvent step:
            logger.LogInformation("Step {Current}/{Total}: {Instruction}",
                step.Current, step.Total, step.Instruction);
            break;

        case BuildLayerCachedEvent cached:
            logger.LogDebug("Cache hit: {Layer}", cached.LayerId);
            break;

        case BuildErrorEvent error:
            logger.LogError("Build failed at step {Step}: {Message}",
                error.Step, error.Message);
            break;

        case BuildProgressEvent progress:
            progressBar.Update(progress.Current, progress.Total);
            break;
    }
}

No regex. No silent drops. Every line of stdout is parsed into a typed event. Events that don't match any known pattern become UnrecognizedOutputEvent -- captured, logged, never silently discarded. The parser knows which output format to expect based on the command configuration (BuildKit vs legacy, plain vs TTY).


Category 4: Hand-Written YAML for Docker Compose

This is the category that affects the most people, because even developers who never call Docker from code still write docker-compose.yml by hand. And YAML is where infrastructure typos go to hide.

Here's a compose file with three bugs. See if you can spot them all:

version: "3.8"

services:
  web:
    image: myapp:latest
    ports:
      - "8080:80"
    depends_on:
      - database
    environment:
      - CONNECTION_STRING=Host=db;Port=5432;Database=myapp;
    networks:
      - frontend
      - backend

  db:
    image: postgres:16
    enviroment:
      POSTGRES_DB: myapp
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: secret
    volumes:
      - pgdata:/var/lib/postgresql/data
    networks:
      - backend
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U admin"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    networks:
      - backend

volumes:
  pgdata:

networks:
  frontend:
  backend:

Bug 1: depends_on: database but the service is named db. Docker Compose won't start the web service because it's waiting for a service called database that doesn't exist. The error message is: service "web" depends on undefined service "database". But you only see this when you run docker compose up. Not when you write the file. Not when you commit. Not when CI checks syntax.

Bug 2: enviroment instead of environment on the db service. This is the YAML equivalent of the --detatch typo. Docker Compose doesn't error on unknown top-level keys within a service definition -- it silently ignores them. So db starts with no environment variables. PostgreSQL creates a database called postgres with user postgres, not myapp with user admin. The web service connects with the wrong credentials and fails. The error message points at the connection string, not at the YAML.

Bug 3: The CONNECTION_STRING references Host=db, which is correct -- but the web service uses depends_on without condition: service_healthy. This means web starts as soon as db's process starts, not when PostgreSQL is ready. The healthcheck is defined but unused. The first few requests crash with "connection refused."

Three bugs, three different failure modes, all invisible at write time:

Bug 1: docker compose up    → error: undefined service "database"
Bug 2: docker compose up    → db starts without env vars (SILENT)
Bug 3: docker compose up    → web crashes connecting to unready db

The silent failure is the worst. I've seen teams debug "wrong credentials" errors for hours before discovering that the environment variables were on a misspelled key. The mental model is "I set the variables" but the reality is "YAML parsed them as a custom extension and threw them away."

And it gets worse with compose file version drift. The develop key (for Docker Compose Watch) was added in Compose spec 2.22.0. The include top-level key was added in 2.20.0. The deploy.resources.reservations.devices path was added for GPU support. Using any of these on an older Docker Compose gives you one of three behaviors depending on the version:

# Using 'develop' on Docker Compose < 2.22.0:
services:
  web:
    build: .
    develop:
      watch:
        - path: ./src
          action: rebuild

# Compose 2.17: silently ignores 'develop' entirely. No warning.
# Compose 2.20: prints a warning but starts the service anyway.
# Compose 2.22+: works as intended.
# Using 'include' on Docker Compose < 2.20.0:
include:
  - path: ./monitoring/compose.yaml

# Compose 2.17: ERROR: yaml: unmarshal errors
# Compose 2.20+: works as intended

The failure modes are inconsistent: sometimes silent, sometimes a warning, sometimes an error. You can't predict which behavior you'll get without knowing the exact Compose version on the target machine. And Compose versions aren't always tied to Docker versions in an obvious way -- Docker Desktop bundles its own Compose version that may differ from what's installed on your Linux CI runner.

Now look at the typed alternative:

var composeFile = new ComposeFileBuilder()
    .WithService("web", web => web
        .WithImage("myapp:latest")
        .WithPorts(p => p.WithTarget(80).WithPublished("8080"))
        .WithDependsOn("db", d => d
            .WithCondition(DependsOnCondition.ServiceHealthy))  // Bug 3: caught by design
        .WithEnvironment("CONNECTION_STRING", "Host=db;Port=5432;Database=myapp;")
        .WithNetworks("frontend", "backend"))
    .WithService("db", db => db
        .WithImage("postgres:16")
        .WithEnvironment(env => env                  // Bug 2: impossible -- no WithEnviroment()
            .Add("POSTGRES_DB", "myapp")
            .Add("POSTGRES_USER", "admin")
            .Add("POSTGRES_PASSWORD", "secret"))
        .WithVolumes(v => v.WithSource("pgdata").WithTarget("/var/lib/postgresql/data"))
        .WithNetworks("backend")
        .WithHealthcheck(h => h
            .WithTest("CMD-SHELL", "pg_isready -U admin")
            .WithInterval(TimeSpan.FromSeconds(10))
            .WithTimeout(TimeSpan.FromSeconds(5))
            .WithRetries(5)))
    .WithService("redis", redis => redis
        .WithImage("redis:7-alpine")
        .WithNetworks("backend"))
    .WithVolume("pgdata")
    .WithNetwork("frontend")
    .WithNetwork("backend")
    .Build();

// Bug 1: .WithDependsOn("database", ...) → Build() throws:
//   InvalidDependencyException:
//   "Service 'web' depends on 'database', but no service named
//    'database' is defined. Did you mean 'db'?"

Every property has a method. Misspelled properties don't compile. Service references are validated at build time. The depends_on condition is an enum, not a string -- you can't write condition: service_heathy. The port mapping is structured -- you can't accidentally swap target and published because they're named parameters, not a colon-separated string where left and right depend on which side of the colon you're on.

The ports case deserves special attention. YAML's short syntax is a well-known trap:

# Short syntax -- which side is host, which is container?
ports:
  - "8080:80"      # Published:Target -- OK
  - "80:8080"      # Oops, reversed. Container listens on 8080, host on 80.
  - "3000"         # Container port only? Or host port only? (It's container.)
  - "5432:5432"    # Same port, but only if nothing else uses 5432 on the host.

# Long syntax -- unambiguous but verbose
ports:
  - target: 80
    published: "8080"
    protocol: tcp

The typed builder uses named parameters exclusively:

.WithPorts(p => p
    .WithTarget(80)         // Container port
    .WithPublished("8080")  // Host port/range
    .WithProtocol(PortProtocol.Tcp))

There is no way to accidentally swap target and published. The names make the intent explicit. The compiler enforces the types. IntelliSense tells you what each parameter means.


The Real Cost

Every one of these bugs has a common root cause: we're using strings where we should be using types.

Strings don't have IntelliSense. Strings don't have version metadata. Strings don't validate references. Strings don't prevent typos. Strings flow through process boundaries and lose all structure. Strings are parsed with regex and lose all meaning.

I tracked the debugging time for Docker-related failures across one quarter in a three-developer team:

Flag typos:           4 incidents,  ~6 hours total debugging
Version drift:        2 incidents,  ~8 hours total debugging
Stdout parsing bugs:  7 incidents, ~14 hours total debugging
YAML typos/errors:    9 incidents, ~12 hours total debugging
                     ──────────────────────────────────────
Total:               22 incidents, ~40 hours/quarter

That's one full work week per quarter spent debugging problems
that a type system would have prevented entirely.

But the cost isn't just debugging hours. It's the cascade:

1 YAML typo (enviroment instead of environment)
  → db starts without credentials
    → web connects with wrong creds
      → web crashes on first request
        → healthcheck fails
          → orchestrator restarts web
            → web crashes again (db still wrong)
              → orchestrator marks web as unhealthy
                → load balancer removes web
                  → 503 for all users
                    → PagerDuty fires at 2:47 AM
                      → on-call engineer spends 90 minutes
                        → root cause: one misspelled YAML key

The blast radius of a single typo in untyped infrastructure is unbounded. The error message at the point of impact ("503 Service Unavailable") has no connection to the root cause ("enviroment" instead of "environment" in a YAML file). The on-call engineer has to work backwards through logs, container states, health checks, and configuration files to find it. Every layer of indirection adds time.

With the typed builder, the blast radius is zero: ComposeServiceBuilder doesn't have a WithEnviroment() method. The code doesn't compile. The developer fixes it in 3 seconds. Nobody gets paged.

And those are just the bugs we found. The silent failures -- the --platform flag that was ignored, the environment variable that was silently discarded, the stdout line that was silently dropped -- those are the ones still hiding in production, waiting for the next Docker upgrade to surface them.

The pattern is always the same:

  1. Developer writes string-based Docker invocation
  2. It works on their machine (Docker Desktop 4.x, latest everything)
  3. It passes code review (the reviewer doesn't memorize Docker flag names either)
  4. It passes CI (if CI has the same Docker version -- and it often doesn't)
  5. It ships to production
  6. It fails in production (different Docker version, different output format, different Compose version)
  7. The failure mode is opaque (wrong container started, missing env var, mangled stdout, silent misconfiguration)
  8. The on-call engineer spends 30-90 minutes tracing backwards through logs
  9. The root cause is trivial (a typo, a missing version check, a format assumption)
  10. The fix is a one-line change
  11. The developer writes "add integration test for Docker flag X" in the post-mortem
  12. That test never gets written because there's no good way to test flag names without a Docker daemon
  13. It happens again three weeks later with a different flag

Step 12 is the key insight. It's not that teams are lazy. It's that the testing infrastructure for "did I spell this flag correctly" doesn't exist in the string-based world. You can't unit test a string. You can't mock Process.Start in a way that validates Docker flag names. The only test is running the command against a real Docker daemon -- and that's an integration test that most CI pipelines don't have, because it requires Docker-in-Docker or a dedicated test environment.

With a typed API, step 12 becomes unnecessary. The compiler is the test. Every method name is a valid flag. Every enum value is a valid option. Every version annotation is a valid constraint. The "integration test for flag spelling" is built into the type system itself.

This is the same shift that happened when ORMs replaced hand-written SQL strings. Nobody writes "SLECT * FROM users WEHRE id = " + userId anymore -- not because we got better at spelling SQL keywords, but because we stopped writing SQL as strings. The compiler checks db.Users.Where(u => u.Id == userId) at compile time. The same shift needs to happen for infrastructure commands.


Where Bugs Get Caught: A Comparison

This diagram shows when each category of bug is discovered in the typed pipeline versus the raw pipeline:

Diagram
The same four bug categories, caught at runtime in the raw pipeline and at compile time (or as explicit typed exceptions) in the typed pipeline — every bug moves left in the development lifecycle.

The shift is consistent: every bug category moves left in the development lifecycle. Compile time beats runtime. Explicit exceptions beat silent failures. Typed events beat regex. Named parameters beat positional strings.


The Vision

The compiler should catch infrastructure mistakes the same way it catches type errors.

Not "the linter should catch them." Not "the code reviewer should catch them." Not "the integration test should catch them." The compiler. The thing that runs every time you save a file. The thing that gives you red squiggles before you even finish typing.

This is what the rest of this series will show:

  1. Scraping: Point the BinaryWrapper pipeline at Docker and Docker Compose. Collect 40+ Docker versions and 57 Compose versions. Parse every --help output into structured JSON command trees. (Part III and Part IV)

  2. Generating: Feed those JSON files to a Roslyn incremental source generator. Emit 200+ typed command classes, fluent builders, and a nested client API -- all with [SinceVersion]/[UntilVersion] annotations. (Part VI)

  3. Executing: Replace Process.Start with a typed executor that builds argument strings from typed commands, parses stdout into typed events, and collects results into typed objects. (Part IX)

  4. Composing: Take the Docker Compose specification's JSON Schema (32 versions), merge them into one unified C# type system, and generate builders that produce valid docker-compose.yml from code -- with IntelliSense, version awareness, and reference validation. (Part X through Part XII)

  5. Contributing: Build composable service definitions that teams can share, test, and reuse -- each one a self-contained unit that contributes to a typed ComposeFile. (Part XIII)

The end result:

Diagram
The target pipeline — every node is typed, from the fluent builder through the command executor and output parser down to a strongly-typed result object, with no regex or unsupervised strings.

Every node in this pipeline is typed. The input is a typed builder. The command is a typed object. The executor knows the Docker version and validates flags before spawning the process. The parser produces typed events. The collector produces a typed result. No strings flow unsupervised between components. No regex. No hope.


What This Series Is Not

This is not an academic exercise. Every line of code in this series is from production packages in the FrenchExDev.Net monorepo. The numbers are real: 97 scraped CLI versions, 32 JSON Schema versions, ~500 generated files. The packages are used daily to deploy actual services.

This is not a wrapper around Docker.DotNet. Docker.DotNet wraps the Docker Engine API -- the HTTP/Unix socket REST API. This wraps the Docker CLI -- the docker and docker compose commands that developers actually type. They solve different problems, and the distinction matters:

// Docker.DotNet -- talks to the Docker Engine API over HTTP/Unix socket
// Good for: container management, image inspection, event streaming
var client = new DockerClientConfiguration().CreateClient();
await client.Containers.CreateContainerAsync(new CreateContainerParameters
{
    Image = "nginx:latest",
    Name = "web",
    HostConfig = new HostConfig
    {
        PortBindings = new Dictionary<string, IList<PortBinding>>
        {
            { "80/tcp", new List<PortBinding> { new() { HostPort = "8080" } } }
        }
    }
});

// But Docker.DotNet CANNOT do:
// - docker build with BuildKit (BuildKit uses a different gRPC protocol)
// - docker compose up/down/build (Compose is a CLI-only tool)
// - docker login (credential management is CLI-side)
// - docker context (context switching is CLI-side)
// - docker buildx (BuildKit extended features)
// - docker scout (image analysis)
// - docker init (project scaffolding)
// - Any Docker CLI plugin command

// Typed Docker CLI -- talks to the docker binary via Process.Start
// Covers EVERYTHING the CLI can do, because it IS the CLI
var result = await executor.ExecuteAsync(
    new DockerImageBuildCommandBuilder()
        .WithTag("myapp:latest")
        .WithBuildArg("VERSION", "1.0.0")
        .WithTarget("runtime")
        .WithFile("Dockerfile")
        .WithPath(".")
        .Build());

Docker.DotNet is the right choice when you need to interact with the Docker Engine programmatically -- watching container events, managing networks, inspecting images. But most developer-side infrastructure work happens through the CLI: building images, running compose stacks, managing multi-stage builds, running BuildKit with cache mounts and SSH forwarding. That's where the CLI wrapper lives.

This is not specific to Docker. The same BinaryWrapper pattern works for any CLI binary with --help output. I've applied it to Packer, Vagrant, Podman, and others. Docker is just the most complex and most impactful example. If you've read the BinaryWrapper post, this series is the deep dive into the Docker application of that pattern.

And this is not a "let's generate some code" exercise. This is about a philosophy: infrastructure should be as type-safe as domain models. If your User class has a string Email property and you accidentally write user.Emial, the compiler stops you. Why should docker run --detatch be any different? The tools exist. The patterns exist. We just haven't applied them to this domain yet. This series shows how.


Next

Part II: The Four Layers of Typed Docker breaks down the architecture: four NuGet packages, three source generators, one attribute each. How the Docker CLI wrapper, Compose CLI wrapper, Compose Bundle, and service contributors stack together -- and why each layer exists as a separate concern.


This is Part I of a sixteen-part series. The series index has the full table of contents and reading guides for different audiences.

⬇ Download