GitOps
GitOps means the desired state of a system lives in Git, every change is a pull request, and an automated process makes the running system match the merged state. In the BuiltForProd platform this holds for infrastructure (a merge to main applies the plan the pull request showed) and for applications (a deploy is a pull request against the GitOps repository that ArgoCD reconciles into the cluster).
Why it matters in production
When Git is the only writer, every change has an author, a reviewer, a diff and a revert. Rollback stops being an emergency procedure and becomes git revert. Nobody needs cluster credentials or console access to ship, so credentials can be scoped to the pipeline and the reconciler. And because the reconciler keeps the cluster equal to the repository, a change made by hand is corrected rather than accumulated.
How the platform applies it
Infrastructure: pull request, plan, merge, apply
Every infrastructure repository runs plan.yml on pull requests and apply.yml on merge to main. The plan job runs the guards, formatting, tflint, Checkov and Trivy, then terragrunt run --all plan and posts the plan as a pull request comment. Code owners review the plan. On merge, the apply job applies every unit in dependency order behind a GitHub Environment gate. Applies queue per branch and are never cancelled, so two merges in quick succession cannot race on the same state. Both workflows trigger on everything a plan depends on: environments/, units/, stacks/, modules/, root.hcl, common.hcl, the VPC maps, the scripts and the workflow files themselves.
Applications: build once, deploy by pull request
The web application's CI never touches a cluster. It builds an image, pushes it to ECR under an immutable tag, and opens a pull request against acme-aws-blueprint-webapp-gitops that sets image.repository and image.tag in blueprint-app/envs/<stage>/values.yaml. ArgoCD, running in each stage's cluster, watches that file and syncs.
| Stage | Opened by | Merge | ArgoCD sync |
|---|---|---|---|
| dev | cd-integration.yml (merge to main) | auto-merge | automatic, with prune and self-heal |
| staging | cd-release.yml (GitHub Release) | auto-merge | automatic, with prune and self-heal |
| prod | promote-prod.yml (manual, by release tag) | a person reviews and merges | manual |
Nothing pushes to the GitOps repository directly. The three workflows share one composite action, .github/actions/gitops-pr, which mints a short-lived installation token for the acme-runner GitHub App (no deploy key, no personal access token), edits the values file with yq, pushes its own deploy/<stage>-<ref> branch and opens or reuses the pull request. Workflow permissions on the code repository stay contents: read.
The GitOps repository holds the Helm chart and per-stage values. Helm layers values.yaml, then values-<stage>.yaml, then envs/<stage>/values.yaml (the image). ArgoCD Application resources are created by the web platform's argocd unit, so the reconciler itself is infrastructure as code.
The ETL variant
The ETL pipeline has no cluster, so it records instead of reconciling: after every deployment cd-integration.yml and cd-release.yml write the deployed tag to the SSM parameter /acme/usw2/<stage>/etl-trigger/image_tag, which the lambda-trigger unit reads at plan time. The parameter is written only by the pipeline; a hand edit would make the next infrastructure apply deploy that tag outside the pipeline.
Worked example: a hot fix reaches production
An engineer merges a fix to main in acme-aws-blueprint-webapp-code. CI builds the image once and pushes main-a1b2c3d; the dev pull request auto-merges and ArgoCD rolls dev. A lead publishes release v1.4.3; cd-release.yml adds that tag to the same image and the staging pull request auto-merges. After verification, someone runs promote-prod.yml with v1.4.3; the prod GitHub Environment asks its required reviewers, the workflow checks that the tag exists in ECR (proof it was built once and ran in staging), and opens the prod pull request. An app lead and a DevOps lead merge it, then sync the application in ArgoCD. If the fix is wrong, they revert the merged deploy pull request or promote the previous tag. The artifact never changed between stages; see immutable artifacts.
Common mistakes
- Editing
envs/<stage>/values.yamlby hand. The file is written only by deploy pull requests; a hand edit is overwritten by the next one and skips the review point. - Pushing to the GitOps repository directly. The design gives the workflows pull-request permissions through a GitHub App; a direct push bypasses the record.
- Expecting prod to sync itself. Prod is
argocd_auto_sync = false; after the merge, a person syncs. Dev and staging self-heal; prod does not. - Deploying with
kubectl apply. ArgoCD reconciles dev and staging back to the repository, and prod shows as out of sync. - Forgetting "Allow auto-merge" on a fork. Dev and staging pull requests merge themselves only when the GitOps repository allows it.