Environments and promotion

An environment is a stage of the software lifecycle with its own AWS account, its own VPC and its own sizing; promotion is how one reviewed change moves through those stages in order. The BuiltForProd platform has four stages, sandbox, dev, staging and prod, each a member account of the plat organizational unit, and every stage is generated from the same definitions with a small block of per-stage values.

Why it matters in production

Environments that were built by hand drift apart, and then staging proves nothing about prod. Environments that share an account share a blast radius, and then a mistake in dev can touch customer data. Separate accounts give each stage a hard identity and billing boundary; identical definitions give each stage the same shape; explicit per-stage values make the differences reviewable; and a fixed promotion order with approvals at the right point makes every production change deliberate.

How the platform applies it

One account per stage

Stage	Slug	Account	Purpose	Isolation domain
sandbox	sbx	plat-sandbox (151515151515)	Experimentation; looser access, no production data, excluded from conformance packs	nonprod
dev	dev	plat-dev (121212121212)	Daily development and automated testing; every merge to main deploys here	nonprod
staging	stg	plat-staging (131313131313)	Production mirror for QA and release verification; high-availability layout like prod	nonprod
prod	prd	plat-prod (141414141414)	Customer workloads; changes only through reviewed pull requests and the prod GitHub Environment	prod

Every workload account gets the same seven landing-zone units from the stacks/plat-account template, and its own spoke VPC from the address plan; see landing zones. Resource names carry the stage: acme-usw2-prd-vpc, acme-usw2-dev-eks. SSM parameters follow /acme/usw2/<stage>/<service>/<key>.

The same shape, different values

The blueprint repositories describe each stage with one stack file that instantiates a template and sets only what differs. The web platform's dev and prod files are identical apart from their values:

Value	dev	prod
public_domain	dev.company.com	prod.company.com
eks_node_min_size / max	2 / 3	3 / 4
argocd_ha	false	true
argocd_auto_sync	true	false
log_retention_days	30	365
docdb_instance_count	1	3
docdb_deletion_protection	false	true
redis_num_cache_clusters / redis_multi_az	1 / false	2 / true

Staging mirrors prod's high-availability layout (multi-AZ data stores, ArgoCD in HA mode, a three-node node group) so that failover is exercised before it matters in prod; dev stays single-instance for cost. Each value is tagged # @optional: with its consequence and price, such as about $60 per month per extra DocumentDB instance. The mechanism is described under Terragrunt units and stacks.

Promotion

Infrastructure repositories. One merge to main plans and applies every stage. The blueprint workflows run a matrix of dev, staging and prod whose jobs start in parallel (fail-fast: false), each under the GitHub Environment of the same name; the prod job waits for that Environment's required reviewers. The landing zone's apply is one job gated by the prod Environment. Applies queue per branch and are never cancelled.

Stage	Trigger	Approval	Rollback
dev	Merge to main	None	git revert, merge, auto-apply
staging	Same merge	Pull request review	git revert, merge, auto-apply
prod	Same merge	Required reviewers of the prod Environment	git revert plus manual validation

Application code. Images are built once and promoted by immutable tag (main-<sha> per build, vX.Y.Z added at release). The web application reaches dev on merge (auto-merged deploy pull request, ArgoCD auto-sync), staging on a GitHub Release (auto-merged, auto-sync), and prod through promote-prod.yml with the release tag: prod Environment approval, a deploy pull request a person merges, then a manual ArgoCD sync. The ETL pipeline reaches dev on merge and staging on release, then waits for prod approval inside the release workflow before deploying the same tag to prod. Rollback is promoting or re-running the previous tag; nothing is rebuilt. See immutable artifacts and GitOps.

Access narrows toward prod

Engineers hold PowerUser access in sandbox, dev and staging and ReadOnly in prod; only Platform Leads and DevOps Leads can change prod. Secrets follow the same line: every engineer group can decrypt the sandbox and dev secrets, only the lead groups can decrypt staging and prod. In the Baseline, code ownership assigns environments/plat/prod/ to the platform leads and the security team; an approval from either team satisfies the rule.

Worked example: a larger prod database

Prod needs a third DocumentDB replica. An engineer changes docdb_instance_count = 3 in environments/prod/us-west-2/terragrunt.stack.hcl of the web platform and opens a pull request. The plan workflow shows one added instance in the prod matrix job and no change in dev or staging, because their stack files carry their own values. A reviewer approves the pull request; on merge, the dev and staging apply jobs are no-ops and the prod job waits for the prod Environment's reviewers before adding the instance. Had the engineer wanted the same in staging, the same value in the staging file would have applied in the staging job of the same merge, which starts in parallel with the prod job.

Common mistakes

• Testing in prod because dev "is not the same". Dev is generated from the same definitions; the differences are the values in its stack file, and they are the only differences.
• Setting a per-stage value in the template or the definition. A template default applies to every stage; per-stage choices belong in the stage file, tagged.
• Promoting by rebuilding. Staging and prod run the tag that was built for dev; a rebuild is a different artifact.
• Granting prod write access to speed up a release. Prod changes go through the prod Environment and a person's merge; that is the control, not an obstacle.
• Adding a stage without placing it in an isolation domain. A new stage must be in exactly one domain in network.hcl, or the plan fails.