Skip to main content
Welcome. This site supports keyboard navigation and screen readers. Press ? at any time for keyboard shortcuts. Press [ to focus the sidebar, ] to focus the content. High-contrast themes are available via the toolbar.
serard@dev00:~/cv

Part 05: Git-Composable Config — Shared Base, Personal Overrides

"The alternative — everyone maintains their own fork of the config — leads to drift, merge conflicts, and 'it works on my machine'." — HomeLab/doc/PHILOSOPHY.md

Why

A homelab is rarely a one-person endeavour. Even when it is, it is rarely a one-machine endeavour. You want the same DevLab on your laptop, on your desktop, on the colleague you onboard next month, and on the ephemeral CI instance that validates every pull request. But each of those machines has different RAM, different IP ranges, different VirtualBox versions, different SSH keys, different Bitwarden vaults.

The naive answer is "everyone forks the config". That answer ends in tears within a quarter. Configurations drift. Pull requests touch one developer's config-homelab.yaml with personal IPs in it. CI fails because somebody committed a value only their machine can satisfy. The shared bits and the personal bits are tangled together in a single file with no way to separate them.

The pattern HomeLab uses is the same pattern Vos already uses for its config-vos.yaml (and for which there is precedent in Helm values.yaml overlays, Kustomize bases, Ansible group_vars, and a dozen other places): a committed base, plus a gitignored local override, plus git submodules for shared building blocks. The merge happens at load time. The merged result feeds the pipeline. The user never sees the merge happen — they just edit two files (or one, if they have nothing personal to override).

The shape

Three layers, in order of precedence (lowest first):

Layer 1 — Shared submodule (optional)

Some teams want to share more than a single config file. They want to share machine-type definitions, provisioning scripts, compose contributors, certificate templates. These live in separate git repos and are pulled in via git submodule:

homelab init --submodule frenchexdev/vos-alpine-dockerhost

That command does three things:

  1. git submodule add https://github.com/frenchexdev/vos-alpine-dockerhost shared/vos-alpine-dockerhost
  2. Adds the submodule path to a top-level homelab.shared.yaml listing all the shared sources
  3. Wires the shared types into the lib via IHomeLabPlugin discovery (see Part 10)

The submodule itself is a normal git repo with a homelab.module.yaml manifest at its root:

# shared/vos-alpine-dockerhost/homelab.module.yaml
name: vos-alpine-dockerhost
version: 1.0.0
description: "Standard Alpine 3.21 Docker host machine type"
provides:
  machineTypes:
    - alpine-3.21-dockerhost
  packerContributors:
    - FrenchExDev.Net.Packer.Alpine.DockerHost.DockerContributor

Layer 2 — Committed base

config-homelab.yaml is the file every team member shares. It contains everything that should be the same for everybody:

# config-homelab.yaml — committed
name: devlab
topology: multi
acme:
  name: frenchexdev
  tld: lab
packer:
  distro: alpine
  version: "3.21"
  kind: dockerhost
vos:
  box: frenchexdev/alpine-3.21-dockerhost
  cpus: 4                       # ← team default
  memory: 4096                  # ← team default
  subnet: "192.168.56"          # ← team default
  provider: virtualbox
compose:
  traefik: true
  gitlab: true
  gitlab_runner: true
  domain: frenchexdev.lab
tls:
  provider: native
  domain: frenchexdev.lab
dns:
  provider: pihole
  pihole_url: http://pihole.frenchexdev.lab
gitlab:
  external_url: https://gitlab.frenchexdev.lab
  admin_email: ops@frenchexdev.local
plugins:
  - FrenchExDev.Net.HomeLab.Plugin.Cloudflare
  - FrenchExDev.Net.HomeLab.Plugin.Bitwarden

This file is the shared truth. It is in version control. It is reviewed in pull requests. It is what homelab validate runs against in CI.

Layer 3 — Local overrides

local/config-homelab-local.yaml is the file each developer maintains for their own machine. It is .gitignored. It contains everything that can't be the same for everybody:

# local/config-homelab-local.yaml — gitignored
vos:
  cpus: 8                       # ← my machine has 16 cores, give the lab 8
  memory: 8192                  # ← I have 64 GB of RAM
  subnet: "192.168.57"          # ← my LAN already uses 192.168.56
  provider: parallels           # ← I'm on macOS with Parallels
dns:
  pihole_url: http://192.168.1.5
  pihole_token_file: ~/.config/homelab/pihole.token
gitlab:
  admin_password_file: ~/.bw/devlab-admin

The merge precedence is: submodule defaults → committed base → local overrides. Every field the local file specifies wins. Every field it doesn't specify falls through to the base. Every field the base doesn't specify falls through to the submodule defaults. Every field nothing specifies takes the C# default declared on the [Builder] record.

The wiring

The merge happens inside IHomeLabConfigLoader, called by Stage 1 of the pipeline (Resolve):

[Injectable(ServiceLifetime.Singleton)]
public sealed class HomeLabConfigLoader : IHomeLabConfigLoader
{
    private readonly IFileSystem _fs;
    private readonly IYamlSerializer _yaml;
    private readonly IConfigMerger _merger;

    public HomeLabConfigLoader(IFileSystem fs, IYamlSerializer yaml, IConfigMerger merger)
    {
        _fs = fs;
        _yaml = yaml;
        _merger = merger;
    }

    public async Task<Result<HomeLabConfig>> LoadAsync(FileInfo configPath, CancellationToken ct)
    {
        var dir = configPath.Directory!;

        // 1. Submodule defaults (if any)
        var submoduleDefaults = await LoadSubmoduleDefaultsAsync(dir, ct);

        // 2. Committed base
        if (!_fs.File.Exists(configPath.FullName))
            return Result.Failure<HomeLabConfig>($"config not found: {configPath.FullName}");
        var baseYaml = await _fs.File.ReadAllTextAsync(configPath.FullName, ct);
        var baseConfig = _yaml.Deserialize<HomeLabConfig>(baseYaml);

        // 3. Local overrides (optional)
        var localPath = Path.Combine(dir.FullName, "local", "config-homelab-local.yaml");
        HomeLabConfig? local = null;
        if (_fs.File.Exists(localPath))
        {
            var localYaml = await _fs.File.ReadAllTextAsync(localPath, ct);
            local = _yaml.Deserialize<HomeLabConfig>(localYaml);
        }

        // 4. Merge in precedence order
        var merged = _merger.Merge(submoduleDefaults, baseConfig, local);

        return Result.Success(merged);
    }
}

The IConfigMerger is a deep-merge utility (~150 lines) that walks two config trees and produces a third. Lists are replaced by default (so local.plugins replaces base.plugins rather than concatenating), but specific fields can opt into list-append semantics via a [MergeAppend] attribute on the property. Records are merged field-by-field, with non-default fields from the override winning.

The merge is pure. It does not touch the file system, does not call any binaries, does not do anything that requires mocking. It is unit-tested with input pairs and expected outputs:

[Fact]
public void local_override_replaces_vos_subnet_only()
{
    var baseConfig = new HomeLabConfig
    {
        Name = "devlab",
        Topology = "multi",
        Vos = new VosConfig { Subnet = "192.168.56", Cpus = 4, Memory = 4096 }
    };
    var local = new HomeLabConfig
    {
        Name = "devlab",
        Topology = "multi",
        Vos = new VosConfig { Subnet = "192.168.57" } // only subnet specified
    };

    var merged = new ConfigMerger().Merge(baseConfig, local);

    merged.Vos.Subnet.Should().Be("192.168.57");      // overridden
    merged.Vos.Cpus.Should().Be(4);                    // inherited from base
    merged.Vos.Memory.Should().Be(4096);               // inherited from base
}

Validation runs on the merged result

Part 04 showed schema validation. The schema runs on each individual file (so a typo in local/config-homelab-local.yaml is caught immediately) and on the merged result (so cross-field invariants that only emerge after the merge are also caught). For example, topology: ha is in the base, and vos.machines: [only-one] is in the local override. Each file individually is valid; the merged result violates the "ha topology requires ≥9 machines" invariant. The cross-field validator catches it.

This is also why homelab validate runs at every meaningful trigger:

  • Before every CLI command (cheap, in-process, < 50 ms)
  • In git pre-commit hooks (recommended, optional)
  • In CI (mandatory, blocks merges)
  • In the homelab init template (so the user gets a valid file from day zero)

The submodule loop

The shared submodule layer is the part most teams skip. It is also the part with the highest leverage. Here is the use case:

You have three repos in your team's GitHub org:

  1. frenchexdev/devlab-config — the team's HomeLab repo (this one)
  2. frenchexdev/vos-alpine-dockerhost — the standard machine type
  3. frenchexdev/compose-traefik — the standard Traefik service contributor
  4. frenchexdev/compose-gitlab — the standard GitLab service contributor

devlab-config declares submodules for the other three:

# devlab-config/homelab.shared.yaml
shared:
  - path: shared/vos-alpine-dockerhost
    repo: https://github.com/frenchexdev/vos-alpine-dockerhost
    branch: main
  - path: shared/compose-traefik
    repo: https://github.com/frenchexdev/compose-traefik
    branch: main
  - path: shared/compose-gitlab
    repo: https://github.com/frenchexdev/compose-gitlab
    branch: main

Now everyone on the team consumes the same machine-type definition, the same Traefik routing rules, the same GitLab compose contributor. When the SRE team updates the GitLab major version, they update compose-gitlab once and bump the submodule pointer in devlab-config. Every team member who runs git submodule update picks up the change. Nobody copies and pastes a new compose snippet. Nobody emails a YAML diff. The change is reviewed once, in the upstream repo, and propagated by version control.

Compare this to a team where everyone copies the GitLab compose snippet into their personal config. The first time the GitLab major version changes, somebody updates their copy. The second person copies the first person's update — slightly differently, with a typo. The third person updates from the docs page, which is six months out of date. By month six, three different team members have three different "working" GitLab compose snippets, none of which match. By month twelve, the team gives up and writes a wiki page that they "should standardise on". The wiki page never gets written.

Submodules are not glamorous. They are sometimes annoying (git submodule update --recursive is a phrase everyone has muttered with venom). They solve this problem better than every alternative for the simple reason that the submodule is a pinned commit. You always know exactly which version of compose-gitlab your devlab-config depends on, because it is in the git tree.

The test

public sealed class HomeLabConfigLoaderTests
{
    [Fact]
    public async Task base_only_loads_unchanged()
    {
        var fs = new MockFileSystem();
        fs.AddFile("/lab/config-homelab.yaml", new MockFileData("name: solo\ntopology: single"));
        var loader = new HomeLabConfigLoader(fs, new YamlSerializer(), new ConfigMerger());

        var result = await loader.LoadAsync(new FileInfo("/lab/config-homelab.yaml"), CancellationToken.None);

        result.IsSuccess.Should().BeTrue();
        result.Value.Name.Should().Be("solo");
        result.Value.Topology.Should().Be("single");
    }

    [Fact]
    public async Task local_overrides_specific_fields_only()
    {
        var fs = new MockFileSystem();
        fs.AddFile("/lab/config-homelab.yaml", new MockFileData("""
            name: dl
            topology: multi
            vos:
              cpus: 4
              memory: 4096
              subnet: 192.168.56
            """));
        fs.AddFile("/lab/local/config-homelab-local.yaml", new MockFileData("""
            vos:
              subnet: 192.168.57
            """));
        var loader = new HomeLabConfigLoader(fs, new YamlSerializer(), new ConfigMerger());

        var result = await loader.LoadAsync(new FileInfo("/lab/config-homelab.yaml"), CancellationToken.None);

        result.Value.Vos.Subnet.Should().Be("192.168.57");
        result.Value.Vos.Cpus.Should().Be(4);
        result.Value.Vos.Memory.Should().Be(4096);
    }

    [Fact]
    public async Task missing_base_returns_failure()
    {
        var fs = new MockFileSystem();
        var loader = new HomeLabConfigLoader(fs, new YamlSerializer(), new ConfigMerger());

        var result = await loader.LoadAsync(new FileInfo("/lab/config-homelab.yaml"), CancellationToken.None);

        result.IsFailure.Should().BeTrue();
        result.Errors.Should().Contain(e => e.Contains("config not found"));
    }
}

The mock file system makes the merge tests trivial. We do not need a real disk to test that two YAML strings produce a particular merged config. That speed (sub-millisecond per test) is why we wrote the file system as an interface in the first place — see Part 11 for the full toolbelt argument.

What this gives you that bash doesn't

Bash configurations end up in one of three places, none of them git-composable:

  1. Hard-coded in the script — works, until you onboard a second person.
  2. Sourced from ~/.homelabrc — works, until you change machines.
  3. Read from environment variables in CI — works, until you have more than three.

Git-composable configuration with a base + local + submodules gives you, for the same surface area:

  • One source of truth for the team-wide defaults (the base)
  • One private file for personal differences (the local)
  • N pinned shared building blocks that the whole team consumes (the submodules)
  • Deep-merge precedence that is testable, deterministic, and explained in one diagram
  • Schema validation at every layer and at the merged result
  • A clear ownership model: shared files are reviewed in the upstream repo, the base is reviewed in the team repo, the local is whatever the developer wants

The bargain is paid back the first time you onboard a new colleague. They run git clone, git submodule update, homelab init --local, fill in three fields in local/config-homelab-local.yaml, run homelab validate, run homelab vos up, and have a working DevLab in twenty minutes. No pair-debugging. No "did you install mkcert". No "what version of VirtualBox". The config is the contract; the merge is the algorithm; the result is reproducible.

Cross-links

⬇ Download