Skip to main content
Welcome. This site supports keyboard navigation and screen readers. Press ? at any time for keyboard shortcuts. Press [ to focus the sidebar, ] to focus the content. High-contrast themes are available via the toolbar.
serard@dev00:~/cv

Part 19: Talking to Packer JSON — Legacy We Cannot Ignore

"Deprecation does not mean disappearance. The wild is full of templates from 2018."

Why

HashiCorp deprecated Packer JSON templates in 2021 in favour of HCL2. New projects should use HCL2; new HomeLab projects do use HCL2 (see Part 20). But the wild is full of JSON templates: tutorials, blog posts, GitHub repos, internal team wikis, Vagrant box recipes, and the bottom of every "personal infrastructure" repo I have ever seen. A homelab tool that refuses to read JSON cuts itself off from a decade of accumulated examples.

The thesis of this part is: HomeLab reads Packer JSON, parses it into the same typed model HCL2 produces, can round-trip it back to HCL2, and warns the user that the format is deprecated. The user can homelab packer migrate template.json and get back a template.pkr.hcl they can commit. The wrapper itself never runs JSON templates — it always runs HCL2. The JSON support is read-only.

The shape

public interface IPackerJsonReader
{
    Task<Result<PackerBundle>> ReadAsync(string jsonPath, CancellationToken ct);
}

public interface IPackerJsonToHclConverter
{
    Result<string> Convert(PackerBundle bundle);
}

[Injectable(ServiceLifetime.Singleton)]
public sealed class PackerJsonReader : IPackerJsonReader
{
    private readonly IFileSystem _fs;
    private readonly ILogger<PackerJsonReader> _log;

    public async Task<Result<PackerBundle>> ReadAsync(string jsonPath, CancellationToken ct)
    {
        if (!_fs.File.Exists(jsonPath))
            return Result.Failure<PackerBundle>($"file not found: {jsonPath}");

        var content = await _fs.File.ReadAllTextAsync(jsonPath, ct);
        JsonDocument doc;
        try
        {
            doc = JsonDocument.Parse(content);
        }
        catch (JsonException ex)
        {
            return Result.Failure<PackerBundle>($"invalid JSON in {jsonPath}: {ex.Message}");
        }

        _log.LogWarning("Reading Packer template in deprecated JSON format from {Path}. " +
                        "Run `homelab packer migrate {Path}` to convert to HCL2.", jsonPath, jsonPath);

        var root = doc.RootElement;
        var bundle = new PackerBundle();

        if (root.TryGetProperty("variables", out var vars))
            bundle.Variables = ParseVariables(vars);

        if (root.TryGetProperty("builders", out var builders))
            bundle.Sources = ParseBuilders(builders).ToList();

        if (root.TryGetProperty("provisioners", out var provs))
            bundle.Provisioners = ParseProvisioners(provs).ToList();

        if (root.TryGetProperty("post-processors", out var post))
            bundle.PostProcessors = ParsePostProcessors(post).ToList();

        return Result.Success(bundle);
    }

    private IEnumerable<PackerSource> ParseBuilders(JsonElement builders)
    {
        foreach (var b in builders.EnumerateArray())
        {
            var type = b.GetProperty("type").GetString()!;
            var name = b.TryGetProperty("name", out var n) ? n.GetString() : type;

            yield return new PackerSource
            {
                Type = MapBuilderTypeToHclSource(type),  // e.g. "virtualbox-iso" → "virtualbox-iso.alpine"
                Name = name!,
                Properties = ExtractScalarProperties(b),
            };
        }
    }

    // ... ParseProvisioners, ParsePostProcessors, ParseVariables similar
}

The reader walks the JSON, extracts each top-level section, and produces a PackerBundle — the same typed model Packer.Bundle exposes for HCL2 generation. Most of the work is mapping JSON-style "type" strings to HCL2-style "source.." identifiers; the rest is property extraction.

The conversion

[Injectable(ServiceLifetime.Singleton)]
public sealed class PackerJsonToHclConverter : IPackerJsonToHclConverter
{
    public Result<string> Convert(PackerBundle bundle)
    {
        var sb = new StringBuilder();

        // 1. Variables become `variable "name" { default = ... }` blocks
        foreach (var (name, value) in bundle.Variables)
        {
            sb.AppendLine($"variable \"{name}\" {{");
            sb.AppendLine($"  default = {FormatHclValue(value)}");
            sb.AppendLine("}");
            sb.AppendLine();
        }

        // 2. Sources become `source "<type>" "<name>" { ... }` blocks
        foreach (var src in bundle.Sources)
        {
            sb.AppendLine($"source \"{src.Type}\" \"{src.Name}\" {{");
            foreach (var (k, v) in src.Properties)
                sb.AppendLine($"  {k} = {FormatHclValue(v)}");
            sb.AppendLine("}");
            sb.AppendLine();
        }

        // 3. Builds wrap the sources and reference provisioners + post-processors
        sb.AppendLine("build {");
        sb.AppendLine("  sources = [");
        foreach (var src in bundle.Sources)
            sb.AppendLine($"    \"source.{src.Type}.{src.Name}\",");
        sb.AppendLine("  ]");
        sb.AppendLine();
        foreach (var p in bundle.Provisioners)
        {
            sb.AppendLine($"  provisioner \"{p.Type}\" {{");
            foreach (var (k, v) in p.Properties)
                sb.AppendLine($"    {k} = {FormatHclValue(v)}");
            sb.AppendLine("  }");
            sb.AppendLine();
        }
        foreach (var pp in bundle.PostProcessors)
        {
            sb.AppendLine($"  post-processor \"{pp.Type}\" {{");
            foreach (var (k, v) in pp.Properties)
                sb.AppendLine($"    {k} = {FormatHclValue(v)}");
            sb.AppendLine("  }");
            sb.AppendLine();
        }
        sb.AppendLine("}");

        return Result.Success(sb.ToString());
    }
}

The converter is mechanical: map JSON shapes to HCL2 shapes. The hard cases are interpolations ({{user disk_size}} becomes var.disk_size) and conditional builders, but those have well-known transformations. We can lean on HashiCorp's own packer hcl2_upgrade command for the gnarly bits and use HomeLab's converter for the read path only.

The wiring

The homelab packer migrate verb is one new IHomeLabVerbCommand:

[Injectable(ServiceLifetime.Singleton)]
public sealed class PackerMigrateCommand : IHomeLabVerbCommand
{
    private readonly IPackerJsonReader _reader;
    private readonly IPackerJsonToHclConverter _converter;
    private readonly IBundleWriter _writer;
    private readonly IHomeLabConsole _console;

    public Command Build()
    {
        var jsonPath = new Argument<FileInfo>("json-template");
        var outputDir = new Option<DirectoryInfo>("--output", () => new DirectoryInfo("./packer"));

        var cmd = new Command("migrate", "Convert a Packer JSON template to HCL2");
        cmd.AddArgument(jsonPath);
        cmd.AddOption(outputDir);

        cmd.SetHandler(async (FileInfo j, DirectoryInfo o) =>
        {
            var bundleResult = await _reader.ReadAsync(j.FullName, CancellationToken.None);
            if (bundleResult.IsFailure) { _console.Render(bundleResult); Environment.ExitCode = 1; return; }

            var hclResult = _converter.Convert(bundleResult.Value);
            if (hclResult.IsFailure) { _console.Render(hclResult); Environment.ExitCode = 1; return; }

            var hclName = Path.GetFileNameWithoutExtension(j.Name) + ".pkr.hcl";
            var hclPath = Path.Combine(o.FullName, hclName);
            await File.WriteAllTextAsync(hclPath, hclResult.Value);

            _console.WriteLine($"✓ migrated {j.Name} → {hclPath}");
            _console.WriteLine($"  remember to: rm {j.FullName}  # after verifying the HCL2 is correct");
        }, jsonPath, outputDir);

        return cmd;
    }
}

The verb delegates to the reader, then the converter, then writes the file via IBundleWriter. Standard thin-CLI pattern.

The test

public sealed class PackerJsonReaderTests
{
    [Fact]
    public async Task reads_a_minimal_alpine_iso_template()
    {
        var fs = new MockFileSystem();
        fs.AddFile("/lab/alpine.json", new MockFileData("""
            {
              "variables": { "disk_size": "20480" },
              "builders": [
                {
                  "type": "virtualbox-iso",
                  "name": "alpine-3.21",
                  "iso_url": "https://dl-cdn.alpinelinux.org/.../alpine-virt-3.21.0-x86_64.iso",
                  "iso_checksum": "sha256:abc",
                  "disk_size": "{{user `disk_size`}}",
                  "ssh_username": "root"
                }
              ],
              "provisioners": [
                { "type": "shell", "scripts": ["scripts/install-docker.sh"] }
              ]
            }
            """));

        var reader = new PackerJsonReader(fs, NullLogger<PackerJsonReader>.Instance);
        var result = await reader.ReadAsync("/lab/alpine.json", CancellationToken.None);

        result.IsSuccess.Should().BeTrue();
        result.Value.Variables.Should().ContainKey("disk_size");
        result.Value.Sources.Should().ContainSingle();
        result.Value.Sources[0].Type.Should().Be("virtualbox-iso");
        result.Value.Sources[0].Name.Should().Be("alpine-3.21");
        result.Value.Provisioners.Should().ContainSingle();
        result.Value.Provisioners[0].Type.Should().Be("shell");
    }

    [Fact]
    public void converter_emits_valid_hcl2_for_a_minimal_bundle()
    {
        var bundle = new PackerBundle
        {
            Variables = new() { ["disk_size"] = "20480" },
            Sources = new()
            {
                new PackerSource
                {
                    Type = "virtualbox-iso",
                    Name = "alpine",
                    Properties = new() { ["iso_url"] = "https://...", ["disk_size"] = "var.disk_size" }
                }
            },
            Provisioners = new()
            {
                new PackerProvisioner { Type = "shell", Properties = new() { ["scripts"] = new[] { "install.sh" } } }
            }
        };

        var hcl = new PackerJsonToHclConverter().Convert(bundle).Value;

        hcl.Should().Contain("variable \"disk_size\"");
        hcl.Should().Contain("source \"virtualbox-iso\" \"alpine\"");
        hcl.Should().Contain("build {");
        hcl.Should().Contain("provisioner \"shell\"");
    }

    [Fact]
    public async Task reader_warns_about_deprecated_format()
    {
        var fs = new MockFileSystem();
        fs.AddFile("/lab/x.json", new MockFileData("{}"));
        var log = new TestLogger<PackerJsonReader>();
        var reader = new PackerJsonReader(fs, log);

        await reader.ReadAsync("/lab/x.json", CancellationToken.None);

        log.Entries.Should().Contain(e => e.Level == LogLevel.Warning && e.Message.Contains("deprecated JSON"));
    }
}

What this gives you that bash doesn't

A bash script that handles Packer JSON either (a) calls packer build template.json blindly (which works until the user wants to migrate, then they have to do it by hand) or (b) shells out to packer hcl2_upgrade and hopes the output is valid (it usually is, but the bash script cannot verify).

A typed JSON reader with a converter gives you, for the same surface area:

  • A migration verb that produces a clean HCL2 file
  • A typed parse that catches JSON syntax errors with a clear path and line number
  • A loud deprecation warning the first time a JSON template is read
  • A round-trippable model so the same PackerBundle type works for both formats
  • Tests that lock the converter's output

The bargain pays back the first time you onboard a colleague who has a five-year-old Packer JSON template they want to use as a starting point — they migrate it in one command and never look back.

Cross-links

⬇ Download