Versions
The Data and ETL Blueprint pins the same tools as the GCP Enterprise Baseline (OpenTofu 1.12.6, Terragrunt 1.1.6, the google and google-beta providers at ~> 8.5), runs its Spark batches on the Dataproc Serverless 2.3 runtime (Spark 3.5, Python 3.11) and its trigger on Python 3.12. Every Python package, pre-commit hook and GitHub Action is pinned to an exact version in the code, so a plan or a build means the same thing on every machine.
Where the pins live
root.hcl pins OpenTofu, Terragrunt and both Google providers for every unit:
# ─── Version Constraints ─────────────────────────────────────────────────────
terraform_version_constraint = ">= 1.12.6, < 2.0.0"
terragrunt_version_constraint = ">= 1.1.6"
generate "versions" {
path = "versions.tf"
if_exists = "skip"
contents = <<-EOF
terraform {
required_version = ">= 1.12.6"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 8.5"
}
google-beta = {
source = "hashicorp/google-beta"
version = "~> 8.5"
}
}
}
EOF
}
No ETL module ships its own version file, so every unit uses exactly those pins. A module may add one only to declare provider aliases, and must then repeat the same pins: the guard script scripts/check-module-versions.py enforces it in pre-commit and in the plan workflow.
Tools and providers
| Component | Version |
|---|---|
| OpenTofu | 1.12.6 (up to, not including, 2.0.0) |
| Terragrunt | 1.1.6 or later |
google provider | ~> 8.5 |
google-beta provider | ~> 8.5 |
| tflint | 0.64.0 |
| tflint Google ruleset | 0.40.0 |
The plan, apply and drift-detection workflows install exactly OpenTofu 1.12.6 and Terragrunt 1.1.6, with the Terragrunt and tflint binaries checked against their release checksums.
Platform versions
| Component | Version | Set in |
|---|---|---|
| Dataproc Serverless runtime | 2.3 (Spark 3.5, Python 3.11), the same in every stage | spark_runtime_version in each stage stack file |
| Trigger base image | python:3.12-slim, non-root user 10001 | src/trigger/Dockerfile |
| Spark BigQuery connector | The one bundled with the runtime | Not pinned separately; catalog mode uses writeMethod=direct |
The runtime is a stage value, spark_runtime_version = "2.3", which the trigger passes to every batch it submits. Changing it is a change to the stage file, and the Spark test versions in ci.yml move with it.
Application runtime
| Component | Version |
|---|---|
| Trigger Python | 3.12 (image and CI) |
| Flask | 3.1.3 |
| Gunicorn | 26.2.0 |
google-cloud-dataproc | 5.31.0 |
| Spark script Python | 3.11 (the runtime's interpreter) |
| PySpark and Java for the Spark tests in CI | PySpark 3.5.3, Java 17 |
requires-python | >= 3.11 (pyproject.toml); code stays valid on both |
| Ruff | 0.16.10, target py311, line length 120 |
| pytest | 9.1.1 |
The trigger's packages are pinned exactly in src/trigger/requirements.txt, because the image is built once per merge and promoted unchanged.
Pre-commit hooks
| Hook source | Version | Repository |
|---|---|---|
pre-commit/pre-commit-hooks | v6.0.0 | Both |
tofuutils/pre-commit-opentofu | v2.4.2 | Infrastructure |
antonbabenko/pre-commit-terraform | v1.109.1 | Infrastructure |
bridgecrewio/checkov | 3.3.19 | Infrastructure |
bridgecrewio/checkov | 3.3.22 | Code |
astral-sh/ruff-pre-commit | v0.16.10 | Code |
rhysd/actionlint | v1.7.12 | Code |
shellcheck-py/shellcheck-py | v0.11.0.1 | Code |
The infrastructure repository adds four local hooks that run its guard scripts.
GitHub Actions
| Action | Version |
|---|---|
actions/checkout | v7.0.1 |
actions/setup-python | v7.0.0 |
actions/setup-java | v6.0.1 |
actions/upload-artifact | v7.0.1 |
actions/github-script | v9.0.0 |
google-github-actions/auth | v3.0.0 |
google-github-actions/setup-gcloud | v3.0.1 |
docker/setup-buildx-action | v4.4.1 |
docker/build-push-action | v7.4.0 |
aquasecurity/trivy-action | v0.36.0 |
bridgecrewio/checkov-action | v12.3128.0 |
opentofu/setup-opentofu | v2.0.2 |
Each is referenced by its full commit SHA with the version in a comment, so a moved tag cannot change what runs.
How versions change
A version change is its own pull request. For the infrastructure it is planned against all three stages before it merges; for the code it goes through the normal build and release path. The policy is on the versioning policy page, the Baseline's own pins are on the Baseline versions page, and the version matrix lines up every repository's pins.