Skip to main content

Versions

The Data and ETL Blueprint pins the same tools as the Azure Enterprise Baseline (OpenTofu 1.12.6, Terragrunt 1.1.6, azurerm ~> 5.7), adds the Databricks provider ~> 1.135, runs the job on the latest Databricks long-term support (LTS) runtime and the trigger on Python 3.12. Every Python library, pre-commit hook and GitHub Action is pinned to an exact version in the code, so a plan or a build means the same thing on every machine.

Where the pins live​

root.hcl generates a versions.tf with the OpenTofu and azurerm pins into every unit:

root.hcl
generate "versions" {
path = "versions.tf"
if_exists = "skip"
contents = <<-EOF
terraform {
required_version = ">= 1.12.6"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.7"
}
}
}
EOF
}

The two Databricks modules ship their own versions.tf, which replaces the generated one because they need a second provider. They repeat the azurerm pin exactly, and scripts/check-module-versions.py fails in pre-commit and in the plan workflow when they drift apart:

modules/databricks-job/versions.tf
terraform {
required_version = ">= 1.12.6"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 5.7"
}
databricks = {
source = "databricks/databricks"
version = "~> 1.135"
}
}
}

Tools and providers​

ComponentVersion
OpenTofu1.12.6 (up to, not including, 2.0.0)
Terragrunt1.1.6 or later
hashicorp/azurerm~> 5.7
databricks/databricks~> 1.135 (databricks-catalog and databricks-job only)
tflint0.64.0, with the azurerm ruleset 0.32.0

The plan, apply and drift-detection workflows install exactly OpenTofu 1.12.6 and Terragrunt 1.1.6, and the plan workflow's lint job adds tflint 0.64.0. Each Terragrunt and tflint download is verified against its release checksum. The Databricks resources have no tflint ruleset; tofu validate checks them.

Platform versions​

ComponentVersion
Azure Databricks workspacePremium
Databricks runtimeThe latest LTS runtime, looked up at plan time (databricks_spark_version with long_term_support = true)
Serverless environmentVersion 3, used only when the spark_serverless switch is on
Container AppsWorkload profiles environment, Consumption profile
Trigger base imagepython:3.12-slim, with Debian security updates applied at build time

The runtime follows Databricks: a new LTS release shows up as a change to the job cluster's spark_version in the next plan. Review it like any other change before it merges.

Application runtime​

ComponentVersion
Python>= 3.12 (pyproject.toml), 3.12 in CI
azure-identity1.26.0
azure-keyvault-secrets4.11.3
azure-storage-blob12.31.0
azure-storage-queue12.18.0
databricks-sdk0.146.0
PySpark for local tests4.0.4, on Java 17
Ruff0.16.10, target py312, line length 120
pytest9.1.1

The trigger image installs exactly the versions in src/trigger/requirements.txt. The image is built once per merge and promoted unchanged, so every stage runs the same dependency set. The transform itself uses plain PySpark and runs on whatever Spark the Databricks runtime provides.

Pre-commit hooks​

Hook sourceVersionRepository
pre-commit/pre-commit-hooksv6.0.0Both
tofuutils/pre-commit-opentofuv2.4.2Infrastructure
antonbabenko/pre-commit-terraformv1.109.1Infrastructure
bridgecrewio/checkov3.3.19Infrastructure
bridgecrewio/checkov3.3.22Code
astral-sh/ruff-pre-commitv0.16.10Code

The infrastructure repository adds four local hooks that run its guard scripts: mock outputs, stack layout, module provider versions and required inputs.

GitHub Actions​

ActionVersion
actions/checkoutv7.0.1
actions/setup-pythonv7.0.0
actions/setup-javav6.0.1
azure/loginv3.1.0
docker/setup-buildx-actionv4.4.1
docker/build-push-actionv7.4.0
aquasecurity/trivy-actionv0.36.0
bridgecrewio/checkov-actionv12.3128.0
opentofu/setup-opentofuv2.0.2
actions/upload-artifactv7.0.1
actions/github-scriptv9.0.0

Each is referenced by its full commit SHA with the version in a comment, so a moved tag cannot change what runs.

How versions change​

A version change is its own pull request. For the infrastructure it is planned against all three stages before it merges; for the code it goes through the normal build and release path. The Baseline's pins are in its version matrix, the policy is on the versioning policy page, and the versions across all BuiltForProd repositories are in the release notes.