Part 9: Tests, Property Checks, and Quality Gates📋

Two test projects, two test surfaces📋

The bundle splits its tests into two projects because the surfaces are completely different:

test/
├── FrenchExDev.Net.Traefik.Bundle.Tests/                          (net10.0)
│   ├── ModelTests.cs              ← model instantiation
│   ├── BuilderTests.cs            ← builder fluent API
│   ├── SerializerTests.cs         ← YAML/JSON round-trip on minimal fixtures
│   ├── SchemaLoadingTests.cs      ← embedded schema discovery
│   ├── RealisticRoundTripTests.cs ← samples/realistic-dynamic.yaml end-to-end
│   └── Fixtures/
│       ├── static-minimal.yaml
│       └── dynamic-minimal.yaml
└── FrenchExDev.Net.Traefik.Bundle.SourceGenerator.Tests/          (net10.0)
    ├── EmitterTests.cs            ← string assertions on generated source
    ├── AnalyzerTests.cs           ← TFK001 via hand-rolled CSharpCompilation
    └── IrEqualityTests.cs         ← cache contract on the IR types

The runtime tests (Bundle.Tests) need the generated code on disk — they reference the Bundle project, which means the generator has run and produced a real TraefikStaticConfig, TraefikHttpMiddleware, etc. They are ordinary xUnit + Shouldly tests over a real assembly.

The generator tests (Bundle.SourceGenerator.Tests) reference the source generator's own project via InternalsVisibleTo so they can poke at SchemaModel, PropertyModel, TraefikBuilderHelper directly. Roslyn's Microsoft.CodeAnalysis.CSharp.SourceGenerators.Testing package is not referenced — these tests are deliberately low-dependency, using hand-rolled CSharpCompilation instances and plain string assertions on emitter output. The reason: every dependency a generator's test project pulls in is one more thing that has to load successfully into Roslyn's analyzer host environment, and the bundle's quality bar prefers a few more lines of test code over package-version pinning headaches.

EmitterTests: pinning generated source as strings📋

EmitterTests.cs exercises TraefikBuilderHelper directly and asserts on the string output of BuilderEmitter.Emit(...). No Roslyn driver, no compilation step. The pin is at the level "this exact substring is in the generated code":

public sealed class EmitterTests
{
    [Fact]
    public void DiscriminatedBuilder_EmitsExactlyOneBranchEpilogue()
    {
        var branches = new List<DiscriminatedBranch>
        {
            new() { PropertyName = "addPrefix", RefName = "addPrefixMiddleware" },
            new() { PropertyName = "basicAuth", RefName = "basicAuthMiddleware" },
        };

        var model = TraefikBuilderHelper.CreateDiscriminatedBuilderModel(
            "FrenchExDev.Net.Traefik.Bundle", "TraefikHttpMiddleware", branches);
        var source = BuilderEmitter.Emit(model);

        source.ShouldContain("__setCount");
        source.ShouldContain("if (AddPrefix is not null) __setCount++;");
        source.ShouldContain("if (BasicAuth is not null) __setCount++;");
        source.ShouldContain("if (__setCount != 1)");
        source.ShouldContain("TraefikHttpMiddleware requires exactly one branch");
    }

    [Fact]
    public void DiscriminatedBuilder_DoesNotEmitEpilogueOnEmptyBranches()
    {
        var model = TraefikBuilderHelper.CreateDiscriminatedBuilderModel(
            "FrenchExDev.Net.Traefik.Bundle", "Empty", new List<DiscriminatedBranch>());
        var source = BuilderEmitter.Emit(model);

        // Epilogue is still emitted, but with zero counters — count != 1
        // means the build will always fail. That's the right thing for an
        // empty union: there's no valid construction.
        source.ShouldContain("__setCount");
        source.ShouldContain("if (__setCount != 1)");
    }

    [Fact]
    public void StandardBuilder_DoesNotEmitDiscriminatorEpilogue()
    {
        var props = new List<UnifiedProperty>
        {
            new() { Property = new PropertyModel { JsonName = "rule", CSharpName = "Rule", Type = PropertyType.String } },
        };

        var model = TraefikBuilderHelper.CreateBuilderModel(
            "FrenchExDev.Net.Traefik.Bundle", "TraefikHttpRouter", props);
        var source = BuilderEmitter.Emit(model);

        source.ShouldNotContain("__setCount");
        source.ShouldNotContain("requires exactly one branch");
    }
}

These look brittle but they aren't, because what's pinned is the invariant that determines whether the runtime check runs at all: discriminated builders must contain __setCount, standard builders must not. The test would survive a complete refactor of the per-branch counting logic as long as the marker substring stays. The only thing that would break it is removing the epilogue entirely, which is exactly what should break.

The third test is the negative: StandardBuilder_DoesNotEmitDiscriminatorEpilogue proves the epilogue is only emitted on the discriminated path. Without this test, an emitter bug that accidentally added __setCount to every builder would still pass the first two tests.

AnalyzerTests: hand-rolled CSharpCompilation📋

The TFK001 analyzer tests build a small CSharpCompilation containing two strings — the marker attribute and a fake discriminated-union type — plus the user-supplied code under test, then ask Roslyn for the diagnostics:

public sealed class AnalyzerTests
{
    private const string MarkerAttribute = """
        namespace FrenchExDev.Net.Traefik.Bundle.Attributes
        {
            [System.AttributeUsage(System.AttributeTargets.Class)]
            public sealed class TraefikDiscriminatedUnionAttribute : System.Attribute { }
        }
        """;

    private const string FakeUnion = """
        namespace FakeBundle
        {
            [global::FrenchExDev.Net.Traefik.Bundle.Attributes.TraefikDiscriminatedUnion]
            public sealed class FakeMiddleware
            {
                public string? AddPrefix { get; set; }
                public string? BasicAuth { get; set; }
                public string? StripPrefix { get; set; }
            }
        }
        """;

    [Fact]
    public async Task TFK001_FlagsTwoBranchesSet()
    {
        var diagnostics = await RunAnalyzerAsync("""
            class Test
            {
                FakeBundle.FakeMiddleware Make() => new FakeBundle.FakeMiddleware
                {
                    AddPrefix = "x",
                    BasicAuth = "y",
                };
            }
            """);

        var tfk = diagnostics.Where(d => d.Id == "TFK001").ToList();
        tfk.Count.ShouldBe(1);
        tfk[0].GetMessage().ShouldContain("FakeMiddleware");
        tfk[0].GetMessage().ShouldContain("2");
    }

    [Fact]
    public async Task TFK001_FlagsThreeBranchesSet() { … }

    [Fact]
    public async Task TFK001_DoesNotFlagSingleBranch() { … }

    [Fact]
    public async Task TFK001_DoesNotFlagExplicitNullAssignments() { … }
    // …
}

Three things this approach buys over Microsoft.CodeAnalysis.Testing:

1. No package-version drift. The Microsoft testing package needs to match the Roslyn version the generator targets (netstandard2.0 + an old Microsoft.CodeAnalysis.CSharp version), and the constant minor-version mismatches were the single biggest pain when this project was bootstrapped.
2. Test inputs are plain strings. The test reads top-down: marker attribute, fake type, user code, expected diagnostic. No VerifyCS.VerifyAnalyzerAsync ceremony, no markup syntax for diagnostic locations.
3. The fake type is fake. It uses string? for branches instead of nested middleware classes, because the rule under test only cares about the attribute and the assignment count — not the actual types of the branches. That keeps the test fixture small and lets the analyzer run without dragging in the full generated bundle.

The negative tests are the load-bearing ones: TFK001_DoesNotFlagSingleBranch and TFK001_DoesNotFlagExplicitNullAssignments. The first proves the rule doesn't fire on the correct shape. The second proves that defensive BasicAuth = null lines don't trigger false positives — the user might write that to make the intent explicit.

IrEqualityTests: pinning the cache contract📋

Part 4 showed why structural equality on the IR types is load-bearing for incremental performance. IrEqualityTests is the file that keeps it honest. The full file is short — three to five tests per IR type — but every test has the same pattern: build two structurally-identical instances and assert they're equal, then mutate one field and assert they're not.

The pattern is mechanical enough that adding a new field to PropertyModel should trigger a compile error in the test file (because the test builder lambda doesn't set the new field), reminding the maintainer to add a new equality test. That's not enforced by tooling; it's a code-review convention. The day someone forgets, the cache silently degrades and nothing fails.

SchemaLoadingTests: the runtime side of Part 2📋

SchemaLoadingTests proves that TraefikSerializer.LoadEmbeddedSchema(...) actually finds the schemas the <EmbeddedResource> glob in the .csproj was supposed to embed. It's a tiny test, but it catches the class of bugs where someone reorganizes the schemas/ folder and forgets to update the embedded-resource pattern.

This pairs symmetrically with the generator-side TFK004: TFK004 catches "no schemas at compile time", SchemaLoadingTests catches "no schemas at runtime". The same root cause (.csproj glob is wrong) is detected from both directions.

What's not tested (and the honest acknowledgement)📋

A few things the test suite does not cover today:

• Snapshot tests on .g.cs files. No Verify / VerifyTests, no *.verified.cs files. The trade-off is identical to the Microsoft.CodeAnalysis.Testing decision: a snapshot library would catch more emitter regressions, but also needs to match the Roslyn host environment, and the maintenance cost was judged higher than the missed-regression cost. EmitterTests covers the parts that matter (the __setCount epilogue), and the runtime Bundle.Tests cover the parts that matter at the consumer level (the generated types compile and round-trip).
• Property-based tests on the round-trip itself. A CsCheck generator that produces random TraefikDynamicConfig instances and proves Deserialize(Serialize(c)) ≡ c would catch a class of YamlDotNet edge cases that the realistic sample doesn't reach. This is on the next-steps list in Part 10.
• Tests for TFK002. TFK002 (dangling router → service reference) is a reserved diagnostic ID with no implementation. There's nothing to test until the rule exists.

Quality gate📋

The repo ships a quality-gate.yml and a coverage.runsettings at the root. These are the gating files the FrenchExDev "DevOps is dev-side" tooling uses to decide whether a build passes. The coverage settings exclude generated files (<auto-generated/> is honored), so the coverage number reflects the emitter and the runtime serializer, not the .g.cs they produce. The quality gate's failure modes:

• Any test failure ⇒ red.
• Any analyzer warning that isn't TFK001 promoted to error ⇒ red. (TFK001 is intentionally a warning, not an error — see Part 7 for the reasoning.)
• Coverage below the per-project threshold ⇒ red.

The gate is run locally before pushing to main, by convention. There is no cloud CI; the build → push → static-site-deployment cycle is entirely local-tooling-driven. That makes the per-project thresholds the only enforcement point, which is why the runtime + generator test split matters: regressions in either layer are isolated to the project that introduced them.

← Part 8: YAML & JSON Round-Tripping · Next: Part 10 — Versioning, Packaging, and Lessons Learned →