Lake Formation

AWS Lake Formation is the permission layer for a data lake. Instead of granting S3 bucket access to every role, you register the buckets with Lake Formation and grant access to databases, tables and columns. The Data and ETL Blueprint uses it as the only path to the data.

What it does

Lake Formation sits between principals and the Glue Data Catalog. A principal asks for a table; Lake Formation checks its grants and, if allowed, hands back temporary S3 credentials for exactly that location. Grants can name a resource (this database, these tables, all columns except two) or match LF-Tags attached to resources. Data lake administrators manage the grants.

How BuiltForProd uses it

The lake-formation unit in acme-aws-blueprint-etl-infra is applied last in each stage, after the buckets, catalog, Lambda and Glue job exist. Its module does five things:

1. Names the administrators: the acme-terraform-deployer role and the PlatformAdminAccess permission set role. It also removes the IAM_ALLOWED_PRINCIPALS default on new databases and tables, matching what the catalog module does on existing ones.
2. Creates a service role acme-usw2-dev-lf-service-role, assumable only by Lake Formation, with read and write on the three buckets and the KMS actions S3 needs.
3. Registers the raw, processed and curated buckets as data locations under that role.
4. Defines LF-Tags zone (raw, processed, curated), sensitivity (public, internal, confidential, restricted) and domain (analytics, operations), and tags each database with its zone.
5. Grants permissions by name: the Glue job role gets DESCRIBE and SELECT on raw tables, CREATE_TABLE and ALL on processed tables, and data-location access on both buckets; the Lambda role gets DESCRIBE on the raw and processed databases. Optional inputs add SELECT on processed and curated data for data-engineer permission sets, and column-level grants that exclude named columns for a principal.

The bucket side is enforced by the data-lake unit. When enforce_lakeformation_access is true (the default in every stage file), the s3 module attaches a policy that denies all S3 actions unless the caller is an AWS service or one of the exempt roles: the Lake Formation service-linked role, the deployer, the LF service role, the Glue role (for crawling) and the PlatformAdminAccess role. Everything else must come through Lake Formation.

Terms you will see

Term	Meaning
Data lake administrator	A principal that can grant and revoke Lake Formation permissions.
Data location	A registered S3 path whose access Lake Formation controls.
LF-Tag	A key and value attached to catalog resources for tag-based grants.
Named-resource grant	A permission on a specific database, table or column set.
IAM_ALLOWED_PRINCIPALS	The legacy default that lets IAM policies bypass Lake Formation; removed here.

Where to read more

• Data and ETL Blueprint overview
• Glue for the catalog and job it governs
• S3 for the buckets it registers
• IAM Identity Center for the permission sets named in grants