Cloud Logging

Cloud Logging stores the logs of every Google Cloud project in log buckets, routed there by sinks. The GCP Enterprise Baseline turns on Data Access audit logs for every service and routes the audit trail of all 14 projects into one central log bucket and one archive bucket in acme-core-audit, where nobody can delete it.

What it does

Google Cloud writes audit logs for every API call: Admin Activity logs (configuration changes) are always on, Data Access logs (reads and data writes) are off unless configured. Every project routes its logs through the Log Router into log buckets; the _Default bucket keeps a project's own logs for its retention period. A sink sends logs that match a filter to another destination, and an aggregated sink at the organization does so for every project beneath it. Log Analytics lets a bucket be queried with SQL, and a linked dataset exposes it to BigQuery. A retention lock makes a bucket's retention irreversible.

How BuiltForProd uses it

Data Access logs everywhere. The organization-scoped org-logging unit enables ADMIN_READ, DATA_READ and DATA_WRITE for all services at the organization (enable_data_access_logs = true in security.hcl), with an exemption list for high-volume principals. They are billed as Logging ingestion, $0.50 per GiB after the free 50 GiB per project per month.

Two aggregated sinks. Both sinks use the same filter: audit logs of every kind, VPC flow and firewall logs, Cloud DNS queries, Security Command Center findings, Cloud NAT and router logs, and the Cloud Identity sign-in logs.

GCP/acme-gcp-platform-baseline/modules/org-logging/main.tf (lines 30-42)

  # Audit logs of every kind, network telemetry, DNS queries, SCC findings,
  # Cloud NAT and router logs, and the Google Workspace / Cloud Identity
  # sign-in audit logs (shared with Google Cloud from the Admin console).
  audit_filter = <<-EOT
    logName:("cloudaudit.googleapis.com/activity" OR "cloudaudit.googleapis.com/data_access" OR "cloudaudit.googleapis.com/system_event" OR "cloudaudit.googleapis.com/policy" OR "cloudaudit.googleapis.com/access_transparency")
    OR logName:"compute.googleapis.com/vpc_flows"
    OR logName:"compute.googleapis.com/firewall"
    OR resource.type="dns_query"
    OR logName:"securitycenter.googleapis.com"
    OR resource.type="gce_router" OR resource.type="nat_gateway"
    OR resource.type="audited_resource"
    OR logName:"login.googleapis.com"
  EOT

Sink	Destination in acme-core-audit	Holds
acme-org-audit	Log bucket acme-audit, home region	365 days, Log Analytics, linked BigQuery dataset acme_audit_analytics, CMEK; container logs excluded
acme-org-archive	Bucket acme-usw1-audit-logs	365-day retention policy, Nearline after 90 days and Coldline after 365, CMEK, versioning

Each sink's writer identity holds write access to its own destination only: Bucket Writer conditioned to the one log bucket, and Object Creator on the archive bucket. Both destinations encrypt with the audit-logs key of Cloud KMS. Application container logs stay in their stage project's _Default bucket.

Protection. The audit-protection IAM deny policy refuses sink changes everywhere and log or bucket deletion in acme-core-audit to everyone but its exception principals; sinks and buckets carry prevent_destroy. Two switches in security.hcl, off by default, lock the retention for good: lock_log_bucket_retention and lock_archive_bucket_retention.

Retention per project. The project baseline sets each project's _Default bucket, with Log Analytics on:

Projects	_Default retention
Every core project	365 days
acme-plat-sandbox, acme-plat-dev	30 days
acme-plat-staging	90 days
acme-plat-prod	365 days

Who reads what. The auditor groups read the central bucket's views in acme-core-audit and no workload logs; the stage engineers read their own projects. The 13 log-based detections of Cloud Monitoring run over the central bucket.

Terms you will see

Term	Meaning
Data Access log	The audit log of reads and data writes, on for every service.
Aggregated sink	An organization sink that routes the logs of every project beneath it.
Log bucket	A Cloud Logging store with its own retention: acme-audit, _Default.
Linked dataset	The BigQuery view of a log bucket, for SQL over logs.
Writer identity	The service account a sink writes to its destination as.
Retention lock	An irreversible lock on a bucket's retention.

Where to read more

• GCP Enterprise Baseline overview for the audit evidence the platform keeps.
• IAM deny policies for how the trail is protected.
• Observable for the observability pillar of the BuiltForProd Standard.