Skip to main content
Welcome. This site supports keyboard navigation and screen readers. Press ? at any time for keyboard shortcuts. Press [ to focus the sidebar, ] to focus the content. High-contrast themes are available via the toolbar.
serard@dev00:~/cv

Part 29: The Vagrant Box Registry

"Vagrant Cloud is fine until your .box file is 2 GB and contains secrets you cannot upload."

Why

Part 26 and Part 27 produce .box files. Those files have to live somewhere. The default is "on your laptop" — you vagrant box add ./output/package.box --name local/alpine-dockerhost and the box ends up in ~/.vagrant.d/boxes/. That works for one developer on one machine.

The moment you have a second developer or a second machine, you need a registry. Vagrant Cloud is the public option, but it has limitations: it imposes a maximum box size, it requires an account, it cannot host private boxes for free, and — most importantly — uploading a .box file that contains a build-time SSH password to a public registry is a security disaster.

The thesis of this part is: Vagrant.Registry is a small HTTP service that hosts .box files locally, serves the Vagrant catalog JSON, and runs as one of the DevLab compose stacks. HomeLab publishes its own boxes to the registry HomeLab provisioned. This is dogfood loop #3.

The shape

A Vagrant box registry has three responsibilities:

  1. Serve the catalog JSON at a stable URL. Vagrant fetches https://registry.frenchexdev.lab/frenchexdev/alpine-3.21-dockerhost and expects a JSON document describing all available versions.
  2. Serve the box files at the URLs the catalog points to.
  3. Accept new uploads from authenticated clients (so homelab box publish can push).
// FrenchExDev.Net.Vagrant.Registry — a small ASP.NET Core service

[ApiController]
[Route("{user}/{box}")]
public sealed class VagrantCatalogController : ControllerBase
{
    private readonly IBoxRegistry _registry;

    public VagrantCatalogController(IBoxRegistry registry) => _registry = registry;

    [HttpGet]
    public async Task<IActionResult> GetCatalog(string user, string box, CancellationToken ct)
    {
        var versions = await _registry.GetVersionsAsync(user, box, ct);
        if (versions.IsFailure || !versions.Value.Any()) return NotFound();

        var catalog = new
        {
            description = $"{user}/{box} (FrenchExDev local registry)",
            short_description = $"{user}/{box}",
            name = $"{user}/{box}",
            versions = versions.Value.Select(v => new
            {
                version = v.Version,
                providers = v.Providers.Select(p => new
                {
                    name = p.Name,    // "virtualbox" / "hyperv" / "parallels"
                    url = $"{Request.Scheme}://{Request.Host}/{user}/{box}/{v.Version}/{p.Name}.box",
                    checksum = p.Sha256,
                    checksum_type = "sha256"
                })
            })
        };

        return Ok(catalog);
    }

    [HttpGet("{version}/{provider}.box")]
    public async Task<IActionResult> GetBox(string user, string box, string version, string provider, CancellationToken ct)
    {
        var stream = await _registry.OpenBoxStreamAsync(user, box, version, provider, ct);
        if (stream.IsFailure) return NotFound();
        return File(stream.Value, "application/octet-stream", $"{box}-{version}-{provider}.box");
    }

    [HttpPost("{version}/{provider}")]
    [Authorize]
    public async Task<IActionResult> UploadBox(string user, string box, string version, string provider, IFormFile file, CancellationToken ct)
    {
        var result = await _registry.PublishAsync(user, box, version, provider, file.OpenReadStream(), ct);
        return result.IsSuccess ? Ok() : BadRequest(result.Errors);
    }
}

The registry stores boxes in MinIO (which is also part of DevLab). The catalog metadata lives in a small SQLite or in MinIO as JSON files. The whole service is a single binary, ~1500 lines, that runs in a container.

Authentication

The upload endpoint is gated by [Authorize]. The simplest auth is a long-lived API token issued at registry init:

[Injectable(ServiceLifetime.Singleton)]
public sealed class TokenAuthenticationHandler : AuthenticationHandler<AuthenticationSchemeOptions>
{
    private readonly ISecretStore _secrets;

    protected override async Task<AuthenticateResult> HandleAuthenticateAsync()
    {
        if (!Request.Headers.TryGetValue("X-Registry-Token", out var headerValue))
            return AuthenticateResult.NoResult();

        var expected = await _secrets.ReadAsync("VAGRANT_REGISTRY_TOKEN", default);
        if (expected.IsFailure || expected.Value != headerValue.ToString())
            return AuthenticateResult.Fail("invalid token");

        var ticket = new AuthenticationTicket(
            new ClaimsPrincipal(new ClaimsIdentity(new[] { new Claim(ClaimTypes.Name, "publisher") }, "token")),
            "token");
        return AuthenticateResult.Success(ticket);
    }
}

For HomeLab v1 this is enough. Future plugins (HomeLab.Plugin.OAuth) can add SSO.

The publish flow

homelab box publish is a CLI verb that uses the IGitClient and IHttpClient from the toolbelt:

[Injectable(ServiceLifetime.Scoped)]
public sealed class BoxPublishRequestHandler : IRequestHandler<BoxPublishRequest, Result<BoxPublishResponse>>
{
    private readonly IVagrantRegistryApi _registry;
    private readonly IFileSystem _fs;
    private readonly ISecretStore _secrets;
    private readonly IHomeLabEventBus _events;

    public async Task<Result<BoxPublishResponse>> HandleAsync(BoxPublishRequest req, CancellationToken ct)
    {
        if (!_fs.File.Exists(req.BoxFile))
            return Result.Failure<BoxPublishResponse>($"box file not found: {req.BoxFile}");

        var token = await _secrets.ReadAsync("VAGRANT_REGISTRY_TOKEN", ct);
        if (token.IsFailure) return token.Map<BoxPublishResponse>();

        var (user, box) = ParseBoxName(req.Name); // "frenchexdev/alpine-3.21-dockerhost"

        await using var stream = _fs.File.OpenRead(req.BoxFile);
        var result = await _registry.UploadBoxAsync(
            user, box, req.Version, req.Provider, stream, token.Value, ct);

        if (result.IsFailure) return result.Map<BoxPublishResponse>();

        await _events.PublishAsync(new VagrantBoxPublished(req.Name, req.Version, _registry.BaseUri.ToString(), DateTimeOffset.UtcNow), ct);
        return Result.Success(new BoxPublishResponse(req.Name, req.Version));
    }
}

Five lines of meaningful logic, the rest is wiring and event publication. The same IVagrantRegistryApi can target a remote registry (the production DevLab) or a local one (a throwaway test instance). The CLI verb is homelab box publish --name frenchexdev/alpine-3.21-dockerhost --box-file ./packer/output-vagrant/devlab-dockerhost-virtualbox.box --version 1.0.5.

The wiring inside DevLab

Vagrant.Registry is a compose service contributed by VagrantRegistryComposeContributor:

[Injectable(ServiceLifetime.Singleton)]
public sealed class VagrantRegistryComposeContributor : IComposeFileContributor
{
    public void Contribute(ComposeFile compose)
    {
        compose.Services["vagrant-registry"] = new ComposeService
        {
            Image = "frenchexdev/vagrant-registry:1.0",
            Restart = "always",
            Environment = new()
            {
                ["MINIO_ENDPOINT"] = "http://minio:9000",
                ["MINIO_BUCKET"]   = "vagrant-boxes",
                ["MINIO_ACCESS_KEY_FILE"] = "/run/secrets/minio_access_key",
                ["MINIO_SECRET_KEY_FILE"] = "/run/secrets/minio_secret_key",
                ["REGISTRY_TOKEN_FILE"]   = "/run/secrets/vagrant_registry_token"
            },
            Networks = new[] { "platform" },
            Labels = new TraefikLabels()
                .Enable()
                .Router("vagrant-registry", r => r
                    .Rule("Host(`registry.frenchexdev.lab`)")
                    .EntryPoints("websecure")
                    .Tls(certResolver: "default"))
                .Build(),
            DependsOn = new() { ["minio"] = new() { Condition = "service_healthy" } }
        };
    }
}

The registry runs on the platform VM (in multi-VM topology) or alongside everything else (in single-VM topology), reads from MinIO for storage, and is exposed via Traefik at https://registry.frenchexdev.lab.

The test

public sealed class BoxRegistryTests
{
    [Fact]
    public async Task uploaded_box_appears_in_catalog()
    {
        using var fixture = await TestRegistry.NewAsync();
        var token = await fixture.GetTokenAsync();

        await using var box = TestData.SmallBoxStream();
        var upload = await fixture.Api.UploadBoxAsync(
            user: "test", box: "alpine", version: "1.0.0", provider: "virtualbox", box, token, default);

        upload.IsSuccess.Should().BeTrue();

        var catalog = await fixture.Api.GetCatalogAsync("test", "alpine", default);
        catalog.IsSuccess.Should().BeTrue();
        catalog.Value.Versions.Should().ContainSingle(v => v.Version == "1.0.0");
        catalog.Value.Versions[0].Providers.Should().ContainSingle(p => p.Name == "virtualbox");
    }

    [Fact]
    public async Task unauthenticated_upload_returns_401()
    {
        using var fixture = await TestRegistry.NewAsync();
        await using var box = TestData.SmallBoxStream();

        var upload = await fixture.Api.UploadBoxAsync("test", "alpine", "1", "vbox", box, token: "wrong", default);

        upload.IsFailure.Should().BeTrue();
        upload.Errors.Should().Contain(e => e.Contains("401") || e.Contains("invalid token"));
    }

    [Fact]
    public async Task vagrant_can_consume_the_catalog_format()
    {
        using var fixture = await TestRegistry.NewAsync();
        var token = await fixture.GetTokenAsync();
        await using var box = TestData.SmallBoxStream();
        await fixture.Api.UploadBoxAsync("test", "alpine", "1.0.0", "virtualbox", box, token, default);

        // Use the real `vagrant box add` command against our local registry
        var result = await fixture.RunVagrant("box", "add",
            "--name", "test/alpine",
            "--box-version", "1.0.0",
            $"{fixture.BaseUri}/test/alpine");

        result.ExitCode.Should().Be(0);
    }
}

The third test is a small integration test that uses the real Vagrant binary against the local registry. It is the most important test in the file, because it asserts that our catalog JSON shape is one Vagrant actually accepts.

What this gives you that bash doesn't

A bash script that "publishes" a Vagrant box is cp to a fileserver and a manual edit of a JSON file by hand. Or it is vagrant cloud publish to the public service, which is fine for public boxes and impossible for private ones.

A typed local Vagrant registry gives you, for the same surface area:

  • A local HTTP service that runs as a compose stack inside the lab it serves
  • A typed catalog with version + provider + checksum metadata
  • An authenticated upload endpoint with a long-lived token from the secrets store
  • A typed IVagrantRegistryApi for the publish CLI verb
  • MinIO-backed storage so the boxes share infrastructure with the rest of DevLab
  • Tests including a real-Vagrant integration test that exercises the catalog format

The bargain pays back the first time you run homelab box publish and watch the box land in a registry HomeLab provisioned, then run homelab vos up on a colleague's machine and watch Vagrant pull it back from the same registry — without anyone editing JSON.

Cross-links

⬇ Download