What you receive

You receive a complete data engineering environment, a working ETL pipeline and a governed data lake, deployed into your plat accounts on top of the AWS Baseline, together with the two Git repositories that define it, customized for your organization, and the documentation to run and change it. After handover your engineers own all of it.

How delivery works

The blueprint is sold with the Baseline in the full deployment tier, or on its own for customers who already run the Baseline. In both cases the BuiltForProd team deploys it. See blueprints for how the three blueprints fit together and purchasing and licensing for the terms.

The repositories

Repository	You receive
acme-aws-blueprint-etl-infra	Six OpenTofu modules, six Terragrunt units, one stack template, a stack file per stage, four guard scripts, pre-commit hooks, and plan and apply workflows
acme-aws-blueprint-etl-code	The Glue PySpark transform, the Lambda trigger and its Dockerfile, unit tests for both, pre-commit hooks and three GitHub Actions workflows

acme in the names is your namespace. Your account ids, your GitHub organization and the registry in your artifacts account are set in the code before deployment.

The running pipeline

In each of dev, staging and prod:

• Three S3 buckets, one per zone (raw, processed, curated), encrypted, versioned, private and on a lifecycle rule.
• Three Glue Data Catalog databases, one per zone.
• The Glue PySpark job and a crawler.
• The Lambda trigger, running from a container image in the stage's private subnets, with its own security group and tracing on.
• Lake Formation set up with administrators, the three buckets registered, three LF-Tags and the permission grants for the job, the function and data engineers.
• A bucket policy on each zone that routes data access through Lake Formation.

Upload a JSON file under input/ in the raw bucket and the pipeline runs end to end.

The delivery pipeline

• Pull requests to the code repository run lint, unit tests, a container build and an image vulnerability scan.
• A merge to main builds the Lambda image once, uploads the Glue script and deploys both to dev.
• A GitHub Release adds the version tag to the same image and deploys it and the script to staging and, after a reviewer approves, to prod.
• Pull requests to the infrastructure repository plan all three stages; a merge applies them.

The documentation

This documentation set: the architecture, the data lake, Glue, the Lambda trigger, Lake Formation governance, the ETL code, the pipelines, querying, security, performance tuning, operations, how-to guides for day-2 work such as granting data access and releasing, and a reference of every unit, module, stack value and environment variable.

What stays with the Baseline

The accounts, networks, identity, security services and the container registry come from the AWS Baseline. The blueprint reads what it needs from the Baseline and does not change it. The architecture overview shows how the two connect.