Skip to main content
Welcome. This site supports keyboard navigation and screen readers. Press ? at any time for keyboard shortcuts. Press [ to focus the sidebar, ] to focus the content. High-contrast themes are available via the toolbar.
serard@dev00:~/cv

Part 20: Talking to Packer HCL2 — The Default

"Don't write HCL. Generate it from typed C# and let the contributors compose."

Why

Part 19 handled the legacy JSON format. This part handles the default: HCL2. HomeLab generates HCL2 by composing typed contributors, not by templating strings or by editing a .pkr.hcl by hand. The contributor pattern is the same one we have already seen for compose files and Traefik configs (Part 32), and it is the same pattern Packer.Bundle already uses for its proven Alpine + DockerHost pipeline.

The thesis of this part is: a Packer build is the result of N contributors mutating a shared PackerBundle, then a PackerBundleWriter rendering that bundle to multi-file HCL2 on disk. No string templates. No Format-style interpolation. No sed post-processing. The bundle is the source of truth; the writer is the I/O; everything in between is typed.

The shape

public sealed class PackerBundle
{
    public Dictionary<string, object?> Variables { get; set; } = new();
    public List<PackerSource> Sources { get; set; } = new();
    public List<PackerProvisioner> Provisioners { get; set; } = new();
    public List<PackerPostProcessor> PostProcessors { get; set; } = new();
    public List<PackerHttpFile> HttpFiles { get; set; } = new();
    public List<PackerScript> Scripts { get; set; } = new();
}

public sealed record PackerSource
{
    public required string Type { get; init; }       // "virtualbox-iso", "qemu", "parallels-iso", ...
    public required string Name { get; init; }       // "alpine-3.21"
    public Dictionary<string, object?> Properties { get; init; } = new();
}

public sealed record PackerProvisioner
{
    public required string Type { get; init; }       // "shell", "file", "ansible-local"
    public Dictionary<string, object?> Properties { get; init; } = new();
}

public interface IPackerBundleContributor
{
    void Contribute(PackerBundle bundle);
}

A contributor mutates the bundle. Multiple contributors compose: an AlpineBaseContributor adds the source and the autoinstall provisioning; a DockerHostContributor adds the docker-installation script; a GpuPassthroughContributor from a plugin adds the nvidia-container-toolkit script and a post-processor. The order is by [Order] attribute on the contributor class, defaulting to insertion order.

[Injectable(ServiceLifetime.Singleton)]
[Order(10)]
public sealed class AlpineBaseContributor : IPackerBundleContributor
{
    private readonly HomeLabConfig _config;
    public AlpineBaseContributor(IOptions<HomeLabConfig> config) => _config = config.Value;

    public void Contribute(PackerBundle bundle)
    {
        var version = _config.Packer.Version;             // "3.21"
        var diskSize = _config.Packer.DiskSize;           // 20480

        bundle.Variables["disk_size"] = diskSize;
        bundle.Variables["alpine_version"] = version;

        bundle.Sources.Add(new PackerSource
        {
            Type = "virtualbox-iso",
            Name = $"alpine-{version}",
            Properties = new()
            {
                ["iso_url"] = $"https://dl-cdn.alpinelinux.org/alpine/v{version}/releases/x86_64/alpine-virt-{version}.0-x86_64.iso",
                ["iso_checksum"] = "file:https://dl-cdn.alpinelinux.org/.../sha256sums",
                ["disk_size"] = "var.disk_size",
                ["memory"] = _config.Packer.Memory,
                ["cpus"] = _config.Packer.Cpus,
                ["ssh_username"] = "root",
                ["ssh_password"] = "alpine-build",
                ["ssh_timeout"] = "20m",
                ["http_directory"] = "http",
                ["boot_command"] = new[]
                {
                    "<wait5>root<enter><wait>",
                    "setup-alpine -e -f http://{{ .HTTPIP }}:{{ .HTTPPort }}/answers<enter>",
                    "<wait>alpine-build<enter><wait>alpine-build<enter><wait5>",
                    "y<enter>"
                },
                ["shutdown_command"] = "poweroff",
            }
        });

        bundle.HttpFiles.Add(new PackerHttpFile("answers", AlpineAnswerFileGenerator.Generate(_config)));
    }
}

[Injectable(ServiceLifetime.Singleton)]
[Order(20)]
public sealed class DockerHostContributor : IPackerBundleContributor
{
    public void Contribute(PackerBundle bundle)
    {
        bundle.Scripts.Add(new PackerScript("install-docker.sh", DockerInstallScript.AlpineApk));
        bundle.Scripts.Add(new PackerScript("enable-docker-tcp.sh", DockerTcpExposeScript.OnPort(2375)));

        bundle.Provisioners.Add(new PackerProvisioner
        {
            Type = "shell",
            Properties = new()
            {
                ["scripts"] = new[] { "scripts/install-docker.sh", "scripts/enable-docker-tcp.sh" },
                ["execute_command"] = "sh -c '{{ .Vars }} {{ .Path }}'",
            }
        });

        bundle.PostProcessors.Add(new PackerPostProcessor
        {
            Type = "vagrant",
            Properties = new()
            {
                ["output"] = "output-vagrant/{{.BuildName}}-docker-{{.Provider}}.box",
                ["provider_override"] = "virtualbox",
            }
        });
    }
}

AlpineBaseContributor runs first (Order 10), adds the source and the answer file. DockerHostContributor runs second (Order 20), adds the install scripts and the post-processor. A plugin contributor for Podman host overlay would have Order = 25 and replace the docker scripts with podman ones (or run alongside, depending on the engine setting).

The writer

[Injectable(ServiceLifetime.Singleton)]
public sealed class PackerBundleWriter
{
    private readonly IFileSystem _fs;

    public async Task<Result<PackerWriteOutput>> WriteAsync(PackerBundle bundle, DirectoryInfo outputDir, CancellationToken ct)
    {
        if (!_fs.Directory.Exists(outputDir.FullName))
            _fs.Directory.CreateDirectory(outputDir.FullName);

        // Write variables.pkr.hcl
        var varsHcl = HclEmitter.EmitVariables(bundle.Variables);
        await _fs.File.WriteAllTextAsync(Path.Combine(outputDir.FullName, "variables.pkr.hcl"), varsHcl, ct);

        // Write sources.pkr.hcl
        var sourcesHcl = HclEmitter.EmitSources(bundle.Sources);
        await _fs.File.WriteAllTextAsync(Path.Combine(outputDir.FullName, "sources.pkr.hcl"), sourcesHcl, ct);

        // Write build.pkr.hcl (the build block referencing all sources, provisioners, post-processors)
        var buildHcl = HclEmitter.EmitBuild(bundle);
        await _fs.File.WriteAllTextAsync(Path.Combine(outputDir.FullName, "build.pkr.hcl"), buildHcl, ct);

        // Write http/answers and scripts/* files
        var httpDir = Path.Combine(outputDir.FullName, "http");
        if (!_fs.Directory.Exists(httpDir)) _fs.Directory.CreateDirectory(httpDir);
        foreach (var http in bundle.HttpFiles)
            await _fs.File.WriteAllTextAsync(Path.Combine(httpDir, http.FileName), http.Content, ct);

        var scriptsDir = Path.Combine(outputDir.FullName, "scripts");
        if (!_fs.Directory.Exists(scriptsDir)) _fs.Directory.CreateDirectory(scriptsDir);
        foreach (var script in bundle.Scripts)
            await _fs.File.WriteAllTextAsync(Path.Combine(scriptsDir, script.FileName), script.Content, ct);

        return Result.Success(new PackerWriteOutput(
            VariablesFile: "variables.pkr.hcl",
            SourcesFile: "sources.pkr.hcl",
            BuildFile: "build.pkr.hcl",
            HttpFiles: bundle.HttpFiles.Count,
            Scripts: bundle.Scripts.Count));
    }
}

The writer produces multi-file HCL2 output: variables.pkr.hcl, sources.pkr.hcl, build.pkr.hcl, plus an http/ directory and a scripts/ directory. Packer reads all *.pkr.hcl files in the working directory at once, so the multi-file split is purely for human readability — and it makes the diff between two builds much clearer.

The HclEmitter is ~200 lines of code that knows how to format HCL2 values (strings, numbers, lists, maps, nested blocks). It handles escaping, multi-line strings, and the quoting rules. It is unit-tested with golden files: feed it a PackerBundle, compare the output to a committed .pkr.hcl reference, fail on diff.

The wiring

PackerBundleWriter is consumed by the Generate stage of the pipeline (see Part 07):

public async Task<Result<HomeLabContext>> RunAsync(HomeLabContext ctx, CancellationToken ct)
{
    var bundle = new PackerBundle();
    foreach (var contributor in _packerContributors.OrderBy(c => c.GetOrder()))
        contributor.Contribute(bundle);

    var writeResult = await _packerWriter.WriteAsync(bundle, new DirectoryInfo(ctx.Request.OutputDir + "/packer"), ct);
    if (writeResult.IsFailure) return writeResult.Map<HomeLabContext>();

    var artifacts = ctx.Artifacts! with { PackerOutput = writeResult.Value };
    return Result.Success(ctx with { Artifacts = artifacts });
}

The Apply stage then invokes IPackerClient.BuildAsync(ctx.Artifacts.PackerOutput.Directory) — the typed Packer wrapper from [BinaryWrapper("packer")] — and the resulting .box file lands in output-vagrant/.

The test

public sealed class PackerBundleWriterTests
{
    [Fact]
    public async Task writes_variables_sources_and_build_files()
    {
        var fs = new MockFileSystem();
        var bundle = new PackerBundle
        {
            Variables = new() { ["disk_size"] = 20480 },
            Sources = new()
            {
                new PackerSource
                {
                    Type = "virtualbox-iso",
                    Name = "alpine",
                    Properties = new() { ["iso_url"] = "https://...", ["disk_size"] = "var.disk_size" }
                }
            },
            Provisioners = new()
            {
                new PackerProvisioner { Type = "shell", Properties = new() { ["script"] = "install.sh" } }
            }
        };
        var writer = new PackerBundleWriter(fs);

        var result = await writer.WriteAsync(bundle, new DirectoryInfo("/out"), CancellationToken.None);

        result.IsSuccess.Should().BeTrue();
        fs.FileExists("/out/variables.pkr.hcl").Should().BeTrue();
        fs.FileExists("/out/sources.pkr.hcl").Should().BeTrue();
        fs.FileExists("/out/build.pkr.hcl").Should().BeTrue();

        var sources = fs.File.ReadAllText("/out/sources.pkr.hcl");
        sources.Should().Contain("source \"virtualbox-iso\" \"alpine\"");
        sources.Should().Contain("disk_size = var.disk_size");
    }

    [Fact]
    public void contributors_compose_in_order()
    {
        var bundle = new PackerBundle();
        var alpine = new AlpineBaseContributor(Options.Create(new HomeLabConfig
        {
            Name = "x", Topology = "single",
            Packer = new() { Distro = "alpine", Version = "3.21", DiskSize = 20480, Cpus = 2, Memory = 1024 }
        }));
        var docker = new DockerHostContributor();

        alpine.Contribute(bundle);
        docker.Contribute(bundle);

        bundle.Sources.Should().ContainSingle(s => s.Name == "alpine-3.21");
        bundle.Provisioners.Should().ContainSingle(p => p.Type == "shell");
        bundle.PostProcessors.Should().ContainSingle(pp => pp.Type == "vagrant");
        bundle.Scripts.Should().Contain(s => s.FileName == "install-docker.sh");
    }

    [Fact]
    public void hcl_emitter_round_trips_a_complex_bundle_against_golden_file()
    {
        var bundle = TestFixtures.AlpineDockerHostBundle();
        var sourcesHcl = HclEmitter.EmitSources(bundle.Sources);
        var golden = File.ReadAllText("fixtures/golden/alpine-docker-sources.pkr.hcl");
        sourcesHcl.Should().Be(golden);
    }
}

The third test is the golden-file test. Whenever HclEmitter formatting changes, the golden file changes; the diff is reviewed in PR. This is the cheapest way to lock down a code generator's output.

What this gives you that bash doesn't

A bash script that produces a .pkr.hcl is cat <<EOF > template.pkr.hcl ... EOF. Every variable is interpolated by the shell. Every escape is fragile. Every typo in HCL2 syntax produces a Packer parse error at build time, with a line number that points at the heredoc, not at the bash that generated it.

A typed contributor pipeline with a bundle writer gives you, for the same surface area:

  • Composable contributors that mutate a shared bundle in declared order
  • A typed model that catches missing fields, wrong types, and circular references at compile time
  • Multi-file output that humans can read, that diffs cleanly, and that Packer happily consumes
  • Golden-file tests that lock the output format
  • Plugin extensibility — a plugin contributor adds a new IPackerBundleContributor and slots into the pipeline automatically

The bargain pays back the first time a colleague says "can we add a sysctl tweak to the Alpine image" and the answer is "add a contributor with Order = 15" instead of "open template.pkr.hcl and find the right place to interpolate".

Cross-links

⬇ Download