Part 27: Namespace Strategy — dev / stage / prod
"One cluster per client. Three namespaces per cluster: dev, stage, prod. Quotas keep them honest. NetworkPolicies keep them isolated."
Why
Once you have a cluster per client, the next question is: how do you split it into environments? The two extremes:
- One cluster per environment per client: 9 clusters for 3 clients × 3 environments. Maximum isolation, maximum cost. Not feasible on a 128 GB workstation.
- One cluster per client, one namespace per environment: 3 clusters total. Three namespaces each (
dev,stage,prod). Resource quotas per namespace prevent dev from eating prod's headroom. NetworkPolicies prevent dev from talking to prod.
The thesis: HomeLab K8s uses option 2. Three namespaces per cluster, with resource quotas, with default-deny network policies, with separate ArgoCD Application objects per environment. This is a pattern Kubernetes was designed for; we should use it instead of fighting it.
The shape
Each namespace is declared via [Namespace]:
[Namespace("acme-dev", DefaultDeny = true)]
[Namespace("acme-stage", DefaultDeny = true)]
[Namespace("acme-prod", DefaultDeny = true)]
[Namespace("acme-platform")] // shared infra; not default-deny
[Namespace("acme-monitoring")] // observability; not default-deny
public sealed class AcmeNamespaces { }The [Namespace] attribute from Part 06 projects to a NamespaceManifest in the bundle. The DefaultDeny = true flag triggers the DefaultDenyNetworkPolicyContributor from Part 16 to add a deny-all policy.
Resource quotas
Each namespace gets a ResourceQuota that bounds its total CPU, memory, and PVC storage:
[Injectable(ServiceLifetime.Singleton)]
public sealed class NamespaceQuotaContributor : IK8sManifestContributor
{
public void Contribute(KubernetesBundle bundle)
{
var quotaTable = new Dictionary<string, NamespaceQuotaSpec>
{
// Generous in dev, tighter in stage, tightest in prod
["acme-dev"] = new(Cpu: "8", Memory: "16Gi", Pvc: "50Gi", PodCount: 100),
["acme-stage"] = new(Cpu: "4", Memory: "8Gi", Pvc: "30Gi", PodCount: 50),
["acme-prod"] = new(Cpu: "4", Memory: "8Gi", Pvc: "30Gi", PodCount: 50),
};
foreach (var (ns, q) in quotaTable)
{
bundle.CrdInstances.Add(new RawManifest
{
ApiVersion = "v1",
Kind = "ResourceQuota",
Metadata = new() { Name = "default-quota", Namespace = ns },
Spec = new Dictionary<string, object?>
{
["hard"] = new Dictionary<string, object?>
{
["requests.cpu"] = q.Cpu,
["requests.memory"] = q.Memory,
["requests.storage"] = q.Pvc,
["pods"] = q.PodCount.ToString(),
["persistentvolumeclaims"] = "20"
}
}
});
}
}
}The quotas are intentionally generous in dev (16 GB of memory) and tighter in stage and prod (8 GB each). This matches the intent: dev is where you experiment, stage is where you simulate, prod is what survives a restart and shows up in the dashboards.
NetworkPolicies between namespaces
DefaultDeny = true from Part 16 gives every flagged namespace a deny-all policy. To allow specific cross-namespace traffic (e.g. dev → stage for promotion testing), the user adds explicit allow rules:
[Injectable(ServiceLifetime.Singleton)]
public sealed class AcmeNamespaceTrafficContributor : IK8sManifestContributor
{
public void Contribute(KubernetesBundle bundle)
{
// Allow acme-dev pods to reach the GitLab in acme-platform (for `git push`)
bundle.NetworkPolicies.Add(new NetworkPolicyManifest
{
Name = "allow-egress-to-gitlab",
Namespace = "acme-dev",
PodSelector = new Dictionary<string, string>(), // all pods in acme-dev
PolicyTypes = new[] { "Egress" },
Egress = new[]
{
new NetworkPolicyEgressRule
{
To = new[]
{
new NetworkPolicyPeer
{
NamespaceSelector = new Dictionary<string, string> { ["kubernetes.io/metadata.name"] = "acme-platform" },
PodSelector = new Dictionary<string, string> { ["app"] = "gitlab" }
}
},
Ports = new[] { new NetworkPolicyPort { Port = 80, Protocol = "TCP" } }
}
}
});
// Allow acme-monitoring's Prometheus to scrape every namespace
foreach (var ns in new[] { "acme-dev", "acme-stage", "acme-prod" })
{
bundle.NetworkPolicies.Add(new NetworkPolicyManifest
{
Name = "allow-prometheus-scrape",
Namespace = ns,
PodSelector = new Dictionary<string, string>(),
PolicyTypes = new[] { "Ingress" },
Ingress = new[]
{
new NetworkPolicyIngressRule
{
From = new[]
{
new NetworkPolicyPeer
{
NamespaceSelector = new Dictionary<string, string> { ["kubernetes.io/metadata.name"] = "acme-monitoring" },
PodSelector = new Dictionary<string, string> { ["app"] = "prometheus" }
}
},
Ports = new[] { new NetworkPolicyPort { Port = 9090, Protocol = "TCP" } }
}
}
});
}
// Notably absent: NO rules allowing acme-dev → acme-prod or acme-stage → acme-prod.
// The default-deny policy is the firewall.
}
}The pattern: default-deny everything, then add minimal allow rules for the cross-namespace flows that should exist. The list of allows is short, audited, and visible in the contributor file. Anything that is not in the file is denied at the CNI layer.
What this gives you that one big namespace doesn't
A single-namespace cluster is what most kind tutorials show. It is also the source of every "I deleted the wrong thing" story. Three namespaces per cluster gives you, for the same surface area:
- Logical separation between dev, stage, and prod
- Resource quotas that prevent dev from starving prod
- NetworkPolicies that prevent dev from talking to prod
- A
kubectl --namespace acme-prodprefix that is hard to confuse withacme-devbecause the namespace is part of every command - Per-namespace ArgoCD Applications so each environment is reconciled independently
- A pattern Kubernetes was designed for instead of fighting the tool
The bargain pays back the first time you kubectl delete deployment payments in acme-dev and discover that acme-prod is unaffected because the namespace was different.