Part 37: Bringing DevLab Up — Single-VM and Multi-VM in One Sequence📋

"Idempotence is what vagrant up should have always been."

Why📋

We have spent thirty-six parts designing HomeLab. This part is the payoff: the actual sequence of commands a user runs to bring DevLab up from nothing, end to end. Both topologies. Same commands, different config.

The thesis of this part is: the single-VM and multi-VM sequences are the same eight commands. The only difference is one field in config-homelab.yaml. The pipeline does the work; the user types the verbs.

The eight commands📋

# 0. Initial setup (once per machine)
$ homelab init --name devlab --acme-name frenchexdev --acme-tld lab
$ cd devlab
$ vim config-homelab.yaml   # set topology, plugins, secrets references

# 1. Build the box
$ homelab packer init
$ homelab packer build

# 2. Register the box (locally and to the registry)
$ homelab box add --local

# 3. Generate the Vagrantfile + config-vos.yaml + boot the VMs
$ homelab vos init
$ homelab vos up

# 4. DNS + TLS
$ homelab dns add gitlab.frenchexdev.lab 192.168.56.10   # or 56.11 for multi
$ homelab tls init --provider native
$ homelab tls install
$ homelab tls trust   # adds the CA to the OS trust store

# 5. Compose stacks
$ homelab compose init
$ homelab compose deploy

# 6. GitLab post-install
$ homelab gitlab configure
$ homelab gitlab runner register

# 7. Verify
$ homelab gitlab status
$ homelab verify

That's it. Eight verbs, two of them in groups (packer init && packer build, vos init && vos up, compose init && compose deploy, gitlab configure && runner register). The user does not edit any generated file.

Single-VM vs multi-VM: the diff📋

The two sequences are byte-for-byte identical except for the config field:

# config-homelab.yaml — single-VM
topology: single

# config-homelab.yaml — multi-VM
topology: multi

The single-VM run produces:

• One Vagrantfile entry
• One compose file (compose.main.yaml) with all 14 services
• One Traefik config
• One DNS entry pointing at 192.168.56.10

The multi-VM run produces:

• Four Vagrantfile entries (gateway, platform, data, obs)
• Four compose files (compose.gateway.yaml, compose.platform.yaml, compose.data.yaml, compose.obs.yaml), each with the relevant subset of services
• One Traefik config (the same one — Traefik runs on gateway and routes to backends on the other VMs by IP)
• Multiple DNS entries — gitlab.frenchexdev.lab → 192.168.56.10 (gateway), data.frenchexdev.lab → 192.168.56.12, etc.

homelab compose deploy walks the compose files in dependency order and runs docker compose up -d on each VM via the engine wrapper. The dependency order comes from the Ops.Dsl Plan stage's DAG.

What happens at each step📋

homelab packer init📋

• Loads config-homelab.yaml
• Runs every IPackerBundleContributor in order
• Writes packer/variables.pkr.hcl, packer/sources.pkr.hcl, packer/build.pkr.hcl, packer/http/answers, packer/scripts/install-docker.sh, …
• Publishes PackerBundleGenerated event

homelab packer build📋

• Calls IPackerClient.BuildAsync("./packer")
• Packer runs the build (15–30 minutes on first run, faster with cache)
• Output: packer/output-vagrant/devlab-dockerhost-virtualbox.box
• Publishes PackerBuildStarted and PackerBuildCompleted events

homelab box add --local📋

• Calls vagrant box add devlab/alpine-3.21-dockerhost packer/output-vagrant/devlab-dockerhost-virtualbox.box
• Publishes VagrantBoxAdded

homelab vos init📋

• Loads config-homelab.yaml
• Calls IVosConfigGenerator.Generate(config)
• Writes Vagrantfile (the fixed one), config-vos.yaml (the data), schemas/vos-config.schema.json

homelab vos up📋

• Calls IVosOrchestrator.UpAllAsync(), which calls IVosBackend.UpAsync() per machine
• Vagrant boots the VM(s) using the box from step 2
• Publishes VosUpStarted per VM and VosUpCompleted per VM

homelab dns add / homelab tls init / homelab tls install / homelab tls trust📋

• Each is a small CLI verb that delegates to the corresponding handler
• DNS entries are created via IDnsProvider
• Cert is generated via ITlsCertificateProvider, written to data/certs/
• The Vagrant synced folder mounts data/certs/ into /etc/ssl/devlab on the VM
• tls trust enrols the CA into the host OS trust store

homelab compose init📋

• Loads the resolved config + the topology resolver's machine list
• Runs every IComposeFileContributor for each VM
• Writes one compose.<vm-role>.yaml per VM

homelab compose deploy📋

• Walks the per-VM compose files
• For each, sets DOCKER_HOST=tcp://<vm-ip>:2375 (or 2376 with mTLS)
• Calls IDockerComposeClient.UpAsync(file, projectName)
• Publishes ComposeStackDeployed per stack
• Waits for healthchecks (which means waiting up to 10 minutes for GitLab to start)

homelab gitlab configure📋

• Calls the GitLab REST API (via the typed IGitLabApi) to:
  • Wait for /api/v4/version
  • Reset the root password (using the seed from the secret store)
  • Create the frenchexdev namespace
  • Create the homelab repo
  • Create the runner registration token
• Publishes GitLabConfigured

homelab gitlab runner register📋

• Reads the registration token from the secret store
• Calls the GitLab API to register a runner
• Updates the runner config in the platform VM via IDockerClient.ExecAsync
• Publishes GitLabRunnerRegistered

homelab verify📋

• Runs the Verify stage of the pipeline
• Walks every OpsObservability.HealthCheck declared by the contributors
• Reports pass/fail per check
• Exit 0 if all pass, exit 1 if any fail

Idempotence📋

Every verb is idempotent. Running homelab packer build twice does not produce two boxes; it skips if the box is already up to date. Running homelab vos up twice does not boot two VMs; it sees they are already up and prints "no-op". Running homelab compose deploy twice does not duplicate containers; docker-compose v2 reconciles the desired state.

This matters because the typical user runs the eight commands two or three times in their lifetime — once on a fresh machine, then again after a config change, then again after a HomeLab upgrade. Each run is additive: only the changes propagate, the rest is left alone.

The pipeline enforces idempotence at the stage level. Each stage's RunAsync is expected to be idempotent for the same input context. The architecture test asserts:

[Fact]
public async Task running_the_pipeline_twice_with_the_same_input_produces_the_same_output()
{
    var sp = TestServices.WithEverythingFaked();
    var pipeline = sp.GetRequiredService<IHomeLabPipeline>();
    var request = new VosUpRequest(MachineName: null, ConfigPath: TestConfigPath);

    var first = await pipeline.RunAsync(request, default);
    var second = await pipeline.RunAsync(request, default);

    first.IsSuccess.Should().BeTrue();
    second.IsSuccess.Should().BeTrue();
    second.Value.Artifacts.Should().BeEquivalentTo(first.Value.Artifacts);
}

The test📋

public sealed class DevLabBringUpE2ETests
{
    [Fact]
    [Trait("category", "e2e")]
    public async Task single_vm_topology_brings_up_devlab_end_to_end()
    {
        using var lab = await TestLab.NewAsync(name: "e2e-single", topology: "single");
        await lab.WriteConfig(StandardSingleConfig());

        await lab.Cli("packer", "init").AssertExitZero();
        await lab.Cli("packer", "build").AssertExitZero();      // ~20 min on cold cache
        await lab.Cli("box", "add", "--local").AssertExitZero();
        await lab.Cli("vos", "init").AssertExitZero();
        await lab.Cli("vos", "up").AssertExitZero();             // ~3 min
        await lab.Cli("dns", "add", "gitlab.frenchexdev.lab", "192.168.56.10").AssertExitZero();
        await lab.Cli("tls", "init", "--provider", "native").AssertExitZero();
        await lab.Cli("tls", "install").AssertExitZero();
        await lab.Cli("compose", "init").AssertExitZero();
        await lab.Cli("compose", "deploy").AssertExitZero();    // ~10 min for GitLab
        await lab.Cli("gitlab", "configure").AssertExitZero();
        await lab.Cli("gitlab", "runner", "register").AssertExitZero();

        var verify = await lab.Cli("verify");
        verify.ExitCode.Should().Be(0);
        verify.StdOut.Should().Contain("✓ gitlab healthy");
        verify.StdOut.Should().Contain("✓ postgres healthy");
        verify.StdOut.Should().Contain("✓ minio healthy");
        verify.StdOut.Should().Contain("✓ traefik healthy");
        verify.StdOut.Should().Contain("✓ baget healthy");
    }

    [Fact]
    [Trait("category", "e2e")]
    [Trait("category", "slow")]
    public async Task multi_vm_topology_brings_up_devlab_end_to_end()
    {
        using var lab = await TestLab.NewAsync(name: "e2e-multi", topology: "multi");
        // ... same eight commands, just with topology: multi
        // ... ~45 minutes total because four VMs boot in sequence
    }
}

The single-VM test runs nightly. The multi-VM test runs weekly. Both are gated behind the slow trait.

What this gives you that bash doesn't📋

A bash script that brings up a homelab is the README on every personal-infrastructure repo on GitHub. It is a list of commands the user is expected to copy-paste into a terminal, in order, hoping each one succeeds. There is no idempotence. There is no observability. There is no event log. If step 7 fails, the user has to know which earlier step is partially complete.

Eight typed CLI verbs backed by a six-stage pipeline give you, for the same surface area:

• Eight commands for both topologies, same shape
• Idempotence at every stage
• Event publication at every meaningful step
• Per-stage failure clarity (the event log tells you exactly which stage failed)
• The same commands in CI as on a developer's laptop
• A verify verb that closes the loop with health checks

The bargain is the entire payoff of HomeLab: the user goes from zero to a working DevLab in eight commands, regardless of whether they want one VM or four.

Cross-links📋

• Part 06: The Dogfood Target
• Part 30: Topology Composition
• Part 36: The Bootstrap Paradox
• Part 39: DevLab GitLab and Runners
• Part 14: The Test Pyramid