Schema Acquisition and JSON Schema Deep Dive📋

GitLab's Release Tag Convention📋

GitLab uses tags like v18.10.0-ee (Enterprise Edition). The Design project strips the v prefix and -ee suffix to get clean semantic versions:

v18.10.0-ee  →  18.10.0
v18.0.0-ee   →  18.0.0

Each tag points to a specific version of the CI schema at:

https://gitlab.com/gitlab-org/gitlab/-/raw/v{version}-ee/app/assets/javascripts/editor/schema/ci.json

The Design Pipeline📋

The Design project is a simple CLI that uses the Wrapper.Versioning framework (the same framework used by BinaryWrapper for downloading CLI help trees):

using FrenchExDev.Net.Wrapper.Versioning;

var outputDir = Path.GetFullPath(Path.Combine(
    AppContext.BaseDirectory, "..", "..", "..", "..",
    "FrenchExDev.Net.GitLab.Ci.Yaml", "schemas"));

var pipeline = new DesignPipeline<string>()
    .UseHttpDownload(v =>
        $"https://gitlab.com/gitlab-org/gitlab/-/raw/v{v}-ee/" +
        $"app/assets/javascripts/editor/schema/ci.json")
    .UseSave()
    .Build();

return await new DesignPipelineRunner<string>
{
    ItemCollector = new GitLabReleasesVersionCollector(
        "gitlab-org%2Fgitlab",
        tagToVersion: tag =>
        {
            // Tags are "v18.10.0-ee" → strip v prefix and -ee suffix
            var v = tag.StartsWith('v') ? tag[1..] : tag;
            var eeIdx = v.IndexOf("-ee", StringComparison.Ordinal);
            return eeIdx >= 0 ? v[..eeIdx] : v;
        }),
    Pipeline = pipeline,
    KeySelector = v => v,
    ItemFilter = VersionFilters.LatestPatchPerMinor,
    OutputDir = outputDir,
    OutputFilePattern = "gitlab-ci-v{key}.json",
    DefaultMinVersion = "18.0.0",
}.RunAsync(args);

Let's break this down:

1. GitLabReleasesVersionCollector — queries the GitLab API for release tags matching the gitlab-org/gitlab project. The tagToVersion lambda strips the v prefix and -ee suffix from each tag.

2. VersionFilters.LatestPatchPerMinor — from all available versions, selects only the latest patch for each minor version. This means we get 18.0.x, 18.1.x, ..., 18.10.x — one schema per minor release, not dozens of patch versions.

3. UseHttpDownload — constructs the download URL for each version and fetches the JSON schema.

4. UseSave — writes the downloaded schema to disk using the OutputFilePattern.

5. DefaultMinVersion = "18.0.0" — ignores GitLab releases before 18.0.

The output lands directly in the main library's schemas/ directory:

src/FrenchExDev.Net.GitLab.Ci.Yaml/schemas/
  gitlab-ci-v18.0.0.json
  gitlab-ci-v18.1.0.json
  gitlab-ci-v18.2.0.json
  gitlab-ci-v18.3.0.json
  gitlab-ci-v18.4.0.json
  gitlab-ci-v18.5.0.json
  gitlab-ci-v18.6.0.json
  gitlab-ci-v18.7.0.json
  gitlab-ci-v18.8.0.json
  gitlab-ci-v18.9.0.json
  gitlab-ci-v18.10.0.json

These JSON files are committed to source control. The Design project runs on-demand — only when you want to pick up new GitLab versions. Normal builds never hit the network.

Running the Design CLI📋

# Download all schemas (min version 18.0)
dotnet run --project src/FrenchExDev.Net.GitLab.Ci.Yaml.Design

# List available versions without downloading
dotnet run --project src/FrenchExDev.Net.GitLab.Ci.Yaml.Design -- --list

After running, rebuild the main library. The source generator automatically picks up the new JSON files and regenerates all models and builders.

JSON Schema Deep Dive📋

To understand what the source generator does, you need to understand what it's working with. GitLab's CI schema is a JSON Schema draft-07 document with some unusual characteristics.

Schema Format📋

The schema declares itself as http://json-schema.org/draft-07/schema# and uses definitions (not the newer $defs from draft 2020-12, though the parser supports both). It has about 54 definitions in the definitions block.

Root Structure📋

The root properties are the reserved keywords:

{
  "properties": {
    "spec": { ... },
    "image": { "$ref": "#/definitions/image" },
    "services": { "$ref": "#/definitions/services" },
    "before_script": { "$ref": "#/definitions/before_script" },
    "after_script": { "$ref": "#/definitions/after_script" },
    "variables": { "$ref": "#/definitions/globalVariables" },
    "cache": { "$ref": "#/definitions/cache" },
    "default": { ... },
    "stages": { ... },
    "include": { ... },
    "pages": { ... },
    "workflow": { ... }
  },
  "additionalProperties": {
    "$ref": "#/definitions/job"
  }
}

The additionalProperties key is crucial: it says "any key not listed in properties is a job definition." This is how GitLab knows that build:, test:, and deploy: are jobs, not configuration keywords. In the C# model, this becomes a Dictionary<string, GitLabCiJob> Jobs property.

Definition Categories📋

The 54 definitions fall into three categories:

1. Full Object Definitions (14) — These have properties and generate C# classes:

2. Type Aliases (~35) — These are $ref targets that map to primitives:

3. Special Definitions (~5) — Skipped during generation:

The allOf Pattern📋

GitLab's schema uses allOf for job inheritance. The job definition is defined as:

{
  "job": {
    "allOf": [
      { "$ref": "#/definitions/job_template" }
    ]
  }
}

This means job inherits all properties from job_template. The SchemaReader's second pass resolves these references by copying properties from the referenced definition:

private static void ResolveAllOf(string defName, JsonElement allOf,
    Dictionary<string, DefinitionModel> definitions)
{
    if (!definitions.TryGetValue(defName, out var target))
        return;

    // If the definition has no properties of its own, merge from allOf refs
    if (target.Properties.Count > 0)
        return;

    foreach (var item in allOf.EnumerateArray())
    {
        if (item.TryGetProperty("$ref", out var refProp))
        {
            var refName = ExtractRef(refProp.GetString()!);
            if (definitions.TryGetValue(refName, out var source))
            {
                foreach (var prop in source.Properties)
                {
                    if (!target.Properties.Exists(p => p.JsonName == prop.JsonName))
                        target.Properties.Add(prop);
                }
                target.Description ??= source.Description;
            }
        }
    }
}

After resolution, GitLabCiJob has the same 30+ properties as GitLabCiJobTemplate.

Union Types in the Schema📋

The schema uses oneOf and anyOf extensively for union types. The source generator must decide how to represent each union in C#:

The resolution priority in ParseOneOf():

1. string + object with properties → generates an inline config class (e.g., GitLabCiJobTemplateEnvironmentConfig) and maps to StringOrObject
2. string + integer → maps to int? (loses the string variant)
3. string + boolean → maps to bool? (loses the string variant)
4. string + array → maps to List<string>? via StringOrList
5. null + $ref → maps to the nullable ref type
6. Refs only → maps to the first ref type
7. null + another type → resolves the non-null type
8. Fallback → string?

This is a pragmatic trade-off: some type fidelity is lost (e.g., string | boolean becomes just bool?), but the API stays clean and usable.

Schema Adaptations Summary📋

The Source Generator Pipeline📋