Hub-and-spoke networking
Hub-and-spoke networking connects every workload network to one hub that owns the shared services and, where the edition routes it there, the egress. Every edition of the BuiltForProd Baseline builds the hub in the core-network account, subscription or project, keeps production and non-production in separate isolation domains that cannot reach each other, and draws every address from one plan file.
Why it matters in production
Connecting every network to every other network does not scale and gives no single place to control or inspect traffic. A hub gives one place for the shared services every stage needs (DNS, remote access, CI runners), one place to enforce which stage may talk to which, and one boundary between production and everything else. Centralized addressing matters for the same reason: overlapping CIDRs break routing and peering, and re-addressing a live network means rebuilding it.
How each edition implements it
Every edition declares its isolation domains in the region's network.hcl, prod with the prod stage and nonprod with sandbox, dev and staging, and every edition fails the plan when a stage is listed in two domains. What carries the traffic and where the domains are separated differ:
| Aspect | AWS Enterprise Baseline | Azure Enterprise Baseline | GCP Enterprise Baseline |
|---|---|---|---|
| Hub | Hub VPC and Transit Gateway in core-network | Hub VNet with Azure Firewall in acme-core-network | Hub VPC in the Shared VPC host acme-core-network |
| Spokes | One VPC per workload account, attached to the gateway | One VNet per plat subscription and the runner VNet, each peered to the hub | One VPC per isolation domain in the host; plat projects attach as service projects |
| Transit between stages | Transit Gateway route tables | Azure Firewall, the next hop of every spoke | Within a domain VPC; the domains peer only with the hub |
| Egress | NAT Gateway per Availability Zone in the hub | Azure Firewall (default) or a NAT gateway per spoke | Cloud NAT per VPC per region |
| Isolation of prod and nonprod | Separate route tables with blackhole routes | Firewall policy rule collections | Separate VPCs, not peered with each other, and firewall policies |
| Remote access | Client VPN, off by default | Point-to-site VPN gateway, off by default | Identity-Aware Proxy, no VPN |
| Address plan | eks_vpc_map.yaml or non_eks_vpc_map.yaml | vnet_map.yaml or vnet_map_compact.yaml | network_map.yaml |
In every edition scripts/check-no-hardcoded-cidrs.py fails the build on any private CIDR literal outside the map, and the region's network.hcl holds the switches several units must agree on, each with its price in a comment.
AWS Enterprise Baseline
The hub VPC (acme-usw2-network-vpc) spans three Availability Zones with one NAT Gateway per AZ. The Transit Gateway (acme-usw2-tgw) is shared with the organization through AWS Resource Access Manager. Each workload account's spoke VPC comes from stacks/plat-account; its private route tables send 0.0.0.0/0 to the Transit Gateway, and it keeps an Internet Gateway only for internet-facing load balancers.
The Transit Gateway has default association and propagation disabled, so an attachment reaches nothing until the transit-gateway-routes unit places it in a domain's route table:
isolation_domains = {
prod = ["prod"]
nonprod = ["sandbox", "dev", "staging"]
}
Two domain tables never learn each other's routes, and each carries a blackhole route for every stage outside it, which beats 0.0.0.0/0 on longest-prefix match. The shared table holds the hub and runner VPCs and learns every attachment.
Every CIDR comes from eks_vpc_map.yaml (or the compact non_eks_vpc_map.yaml) through the ipam unit, and every VPC unit looks its addresses up with one accessor keyed by ipam_key:
cidr = dependency.ipam.outputs.vpc_cidrs_by_region[include.root.locals.aws_region][include.root.locals.ipam_key].primary_cidr
private_subnets = dependency.ipam.outputs.vpc_cidrs_by_region[include.root.locals.aws_region][include.root.locals.ipam_key].private
public_subnets = dependency.ipam.outputs.vpc_cidrs_by_region[include.root.locals.aws_region][include.root.locals.ipam_key].public
scripts/check-vpc-maps.py validates both maps. The switches are nat_gateway_per_az (on; about $32 per month per additional gateway), enable_network_firewall (off; about $865 per month plus $0.065 per GB) and enable_tgw_peering (off; about $36 per month per peering).
Azure Enterprise Baseline
The hub virtual network in acme-core-network holds the firewall, firewall management, gateway, DNS resolver and private endpoint subnets. One spoke VNet per plat subscription and the runner VNet in acme-core-auto are each peered to the hub. Private endpoints sit in each spoke's snet-endpoints, the central Private Link zones are linked to every VNet, and the DNS Private Resolver (enable_dns_private_resolver, on, about $365 per month) serves VPN clients and the firewall's DNS proxy.
In the default mode, hub_egress = "azure_firewall", the routes unit sends 0.0.0.0/0 and the organization range 10.0.0.0/8 from every workload subnet to the firewall's private IP, so the Azure Firewall is the single egress and transit point. Its policy evaluates the isolation-domains rule collection group: traffic inside a domain is allowed, spokes reach the hub and runner networks and back, and the deny-cross-domain collection denies anything else inside 10.0.0.0/8. The egress-web group then allows TCP 80 and 443 to the internet.
In hub_egress = "spoke_nat" mode each spoke gets a NAT gateway (about $33 per month each), there is no inspection, and spokes cannot reach each other. The mode is chosen before the first blueprint apply: an AKS cluster with Node Auto Provisioning cannot change its outbound type.
vnet_map.yaml (or vnet_map_compact.yaml) is the only place addresses are written, with purpose-named subnets such as snet-aks and snet-endpoints, because an Azure subnet spans every zone. The AKS pod and service ranges and the VPN pools sit outside the organization range under reserved. The switches are hub_egress, firewall_sku (Standard, about $912 per month plus $0.016 per GB; Basic about $288; Premium about $1,277), firewall_zones, enable_dns_private_resolver, enable_client_vpn (off, about $263 per month plus connection hours) and enable_traffic_analytics.
GCP Enterprise Baseline
acme-core-network is the only Shared VPC host, with three VPCs: the hub and one per isolation domain. The plat projects attach as service projects and use only their own subnets of their domain's VPC. The hub peers with each domain VPC and the domains never peer with each other; VPC Network Peering is not transitive, so prod and nonprod have no route between them. The organization policy compute.restrictVpcPeering allows peering only inside the network project.
The hub VPC holds what every domain shares: the shared subnet, a proxy-only subnet, the runner subnet of acme-core-auto and the optional bastions. Each domain VPC has one proxy-only subnet per region, owned by one stage (prod for prod, staging for nonprod) and used by every stage of the domain.
Each VPC egresses through its own Cloud Router and Cloud NAT in each region (enable_cloud_nat, about $32 per month each plus $0.045 per GB), and the organization policy compute.vmExternalIpAccess keeps external IPs off every VM. A network firewall policy per VPC, fwp-acme-<domain>, lets the stages of one domain reach each other, the hub reach every domain and every domain reach the hub, and allows egress on TCP 80 and 443 and UDP 53 and 123. Any other private source or destination is denied and logged, and the final rule at priority 65000 denies and logs every other egress. The hierarchical policy at the organization runs first and admits health checks and Identity-Aware Proxy.
network_map.yaml is the only place addresses are written: per stage a /16 with a nodes subnet that carries the GKE pod and service secondary ranges, a data subnet, a Private Service Connect subnet and a Private Service Access range. check-network-maps.py validates it, including exactly one proxy-only subnet per domain and region. Remote access is IAP TCP forwarding with OS Login, plus one optional bastion per domain (enable_bastion, about $7 per month each).
Worked example: a request from dev to the internet
On the Azure edition in the default firewall mode, a pod in the dev AKS cluster opens an HTTPS connection to a public API:
- The spoke route table on
snet-akssends0.0.0.0/0to the firewall's private IP in the hub. - The firewall policy evaluates the
isolation-domainsgroup (priority 300). The destination is outside10.0.0.0/8, so neither the intra-domain allow nordeny-cross-domainmatches. - The
egress-webgroup (priority 400) allows TCP 443, and the firewall translates the source to one of its public IPs, drawn from its public IP prefix. - The reply returns through the firewall and the peering to the pod.
The same pod opening a connection to an address in the prod VNet matches deny-cross-domain in step 2 and is dropped, and the firewall's diagnostic logs record it. Centralized egress is one of the layers described under defense in depth, on top of the isolation units of the landing zone.
Common mistakes
- Typing a CIDR anywhere but the map file. The guard rejects it, and a copied address is exactly how overlaps happen.
- Switching map files after the first apply. Every address differs between the maps, so every network, subnet and route is destroyed and recreated.
- Putting a stage in two domains, or none. A stage in two domains fails the plan in every edition, and so does a stage in none on AWS and GCP; on Azure a stage in no domain reaches no other stage.
shared(AWS, Azure) andhub(GCP) are reserved names. - Adding NAT to a spoke (AWS, and Azure in firewall mode). Spokes egress through the hub by design; a local NAT bypasses the hub egress path and its inspection.
- Changing
hub_egressafter a blueprint cluster exists (Azure). The AKS outbound type is fixed at creation, so switching means recreating the stage clusters. - Peering two domain VPCs (GCP). The
compute.restrictVpcPeeringconstraint limits peering to the network project, and a direct peering would remove the isolation between prod and nonprod.