Skip to main content

Architecture overview

The Secrets Blueprint keeps the secrets your people manage, such as third-party API keys and OAuth client secrets, in one Git repository, encrypted with SOPS under a separate Azure Key Vault key for each stage. When a change merges, a GitHub Actions workflow decrypts the files and writes each value as a secret in the application Key Vault of the matching stage, from where the External Secrets Operator delivers it to the pods.

How it works​

  1. A developer edits a stage's file with SOPS. SOPS encrypts every value with that stage's key in the Baseline's SOPS vault before the file is written, so Git only ever holds ciphertext. The developer's own az login session makes the call, and Key Vault decides whether it succeeds.
  2. The pull request runs Secrets Plan: one job per stage validates every file and dry-runs the sync against the stage's application vault, then comments the plan on the pull request.
  3. Review follows the folder. Developers approve dev, and sandbox once that opt-in stage is added; staging and prod need a platform lead or an infrastructure admin, and their jobs also wait for the GitHub Environment reviewers.
  4. On merge, Sync Secrets to Key Vault signs in to Azure as the federated identity id-acme-secrets-syncer, with no stored credential, decrypts the changed stages and writes one secret per key into kv-acme-eus2-<stage>-app.
  5. In the Web App Blueprint, the External Secrets Operator copies the listed secrets into a Kubernetes Secret, and the pods read them as environment variables.

The parts​

PartWhat it isWhere it comes from
Secrets repositoryOne folder per stage, one encrypted YAML file per application, two scripts and two workflowsThe Secrets Blueprint
SOPS keysFour RSA 4096 keys, sops-sandbox to sops-prod, in kv-acme-eus2-sec-sopsThe Azure Baseline
Syncer identityid-acme-secrets-syncer, trusted only for the stage Environments of this repositoryThe Azure Baseline
Application vaultskv-acme-eus2-<stage>-app, private, one per stage, where the sync writes <app>--<KEY>The Web App Blueprint
ConsumersThe azure-kv ClusterSecretStore of the External Secrets Operator, and the applicationThe Web App Blueprint

The Baseline creates the keys and the syncer identity in acme-core-security; the Web App Blueprint creates the application vaults and grants the syncer write access to them; this blueprint brings the repository, the scripts and the workflows. The Baseline's SOPS keys page documents its side.

Who can touch which secrets​

Access is enforced by Key Vault role-based access control (RBAC), not only by GitHub. Each group holds Key Vault Crypto User on the individual keys of the stages it may read:

Whosandbox and devstaging and prod
ACME_PlatformLeads, ACME_DevOpsLeadsEncrypt, decryptEncrypt, decrypt
The eight engineer and developer groupsEncrypt, decryptNone
id-acme-secrets-syncer (the workflows)Encrypt, decryptEncrypt, decrypt

A developer with write access to the repository still cannot read or change a production secret, because Key Vault refuses the decrypt call on the prod key. This is least privilege applied at the key.

Design choices​

  • Git is the record. Every change to a secret is a reviewed commit with an author, a date and an approver. Nobody edits a value in the portal.
  • One key per stage. Splitting by stage turns "who may see production secrets" into a role assignment on one key, checked on every decrypt.
  • The application vault is what workloads read. Pods never read the repository; each stage vault holds only its own stage's values.
  • No stored credential. Both workflows sign in through OpenID Connect federation and hold a token for the length of a job.
  • Private application vaults. The vaults have no public data plane, so the jobs run on the Baseline's self-hosted runners, which reach the vaults' private endpoints.

Where to go next​

What you receive lists the repository contents and the Baseline and Web App pieces the blueprint relies on. System architecture places every component in its subscription.