Skip to main content

Observable

An Observable system lets operators see what it is doing: every API call, every network flow, every metric, log line and trace, in one place, with alerts on the events that matter most. In every edition of the Baseline the audit evidence goes to core-audit, security telemetry is gathered in core-security, and neither can be switched off from a workload account, subscription or project.

Where telemetry flows​

How each edition delivers it​

Every edition keeps three kinds of data apart: the audit trail of the whole organization, security findings, and application telemetry per stage. The audit trail and the alert route are central; application telemetry stays in the stage that produced it.

SignalAWS Enterprise BaselineAzure Enterprise BaselineGCP Enterprise Baseline
Control-plane auditOrganization CloudTrailActivity Log of every subscription, Microsoft Entra ID logsAdmin Activity and Data Access audit logs, two organization sinks
Network flowsVPC Flow Logs from every VPCVNet flow logs from every VNetVPC flow logs on every workload subnet
Central viewCloudWatch OAM sink in securityAudit, security and application Log Analytics workspacesMetrics scope in acme-core-security, central log bucket
Security alertsSeven CIS metric filters, SNS topic10 activity-log and 6 KQL alerts, one action group13 log-based detections, one email channel
Application telemetryFluent Bit to CloudWatch, X-RayContainer Insights, Application InsightsCloud Logging per stage, Cloud Trace

AWS Enterprise Baseline​

One organization CloudTrail, multi-region, records management events for every account, including accounts created later, plus S3 data events for the audit bucket itself. Log file validation is on, so tampering is detectable, and the trail is KMS-encrypted and delivered to the audit bucket and to a CloudWatch log group kept 365 days. Every VPC sends Flow Logs for all traffic at 60-second aggregation to the audit bucket. AWS Config records every supported resource type and delivers to the same bucket, with the organization aggregator in audit. The CloudTrail explainer covers the trail.

CloudWatch Observability Access Manager makes the security account the monitoring account: the management, audit and network accounts and every workload account link to its sink and share metrics, log groups and X-Ray traces, so operators query every account from one console. Seven CIS metric filters on the CloudTrail log group watch for root use, unauthorized calls, IAM, CloudTrail, bucket policy and network changes; their alarms live in security and fire to the acme-security-alerts SNS topic, which also receives GuardDuty findings of severity 7 and above.

SignalMechanismRetention
Container logsFluent Bit DaemonSet to /acme/<stage>/<app>/eks/<cluster>/application, health-check noise filtereddev 30 days, staging 90, prod 365
EKS control planeAll five log types365 days
LambdaCloudWatch logs and X-Ray active tracing365 days
Public DNSRoute 53 query logs for the apex, prod and staging zones365 days

Azure Enterprise Baseline​

The subscription baseline creates the diagnostic setting acme-activity-log-to-audit in each of the 15 subscriptions, with all eight Activity Log categories, and one tenant-level setting, acme-entra-to-audit, sends the Microsoft Entra ID audit and sign-in logs (enable_entra_log_export, on; it needs Entra ID P1 or P2). Both reach the audit workspace law-acme-eus2-audit-audit and the write-once audit storage in acme-core-audit. Modules that create a resource with audit value set its diagnostic setting themselves, and the acme-actlog-dine policy assignment reports any subscription without the Activity Log setting. Every VNet writes version 2 VNet flow logs to the audit storage of its region; Traffic Analytics is a switch, off.

Log Analytics workspaces come in three kinds: audit in acme-core-audit and security in acme-core-security, both 365 days, and one application workspace per plat stage, law-acme-eus2-<stage>-app, with 30, 30, 90 and 365 days for sandbox, dev, staging and prod. The auditor groups and the DevOps leads hold Log Analytics Reader in the subscriptions, so cross-workspace queries need no forwarding. Every workspace has local authentication off.

Three kinds of alert notify the one action group ag-acme-security-alerts:

AlertWhereFires on
10 activity-log alertsEvery subscriptionCIS Microsoft Azure Foundations Benchmark section 6.2: policy assignments, network security groups, security solutions, SQL firewall rules and public IPs created or deleted
6 KQL alertsAudit workspaceTenant-root elevate access, role assignments at management-group or subscription scope, 10 failed sign-ins for one user in 5 minutes, any break-glass sign-in, a Conditional Access change, a Key Vault purge or access change
Defender for Cloud alert ruleSecurity workspaceAny Defender alert of severity High or Critical, every 5 minutes

In the Web App Blueprint, Container Insights sends cluster and container logs to the stage workspace, a workspace-based Application Insights component collects the API's telemetry, and staging and prod run an availability test on /health. Front Door sends its WAF and health-probe logs in every stage and its access log in staging and prod.

GCP Enterprise Baseline​

Admin Activity audit logs are always on, and one organization audit config turns on Data Access audit logs (ADMIN_READ, DATA_READ, DATA_WRITE) for every service (enable_data_access_logs). Two aggregated organization sinks copy every audit log type, VPC flow logs, firewall logs, Cloud NAT and router logs, DNS queries and Security Command Center findings into acme-core-audit: acme-org-audit to the central log bucket acme-audit (365 days, Log Analytics with a linked BigQuery dataset) and acme-org-archive to the archive bucket acme-usw1-audit-logs. Container logs are excluded from the central copy and stay in their stage project. Flow logs are on for every workload subnet at one-minute aggregation with every flow sampled (flow_logs_sampling = 1.0), and DNS query logging is on.

acme-core-security is the Cloud Monitoring metrics scope for every project. The auditor groups hold roles/monitoring.viewer in every project and read the audit trail through the central bucket's view in acme-core-audit; an IAM deny policy on the plat folder keeps workload logs closed to them. Each project's _Default log bucket keeps 365 days in the core projects and 30, 30, 90 and 365 days in sandbox, dev, staging and prod.

13 log-based metrics over the central bucket, eight of them from section 2 of the CIS Google Cloud Foundations Benchmark, each have an alert policy in acme-core-security that fires on any match in five minutes and emails security_alert_email: project ownership, audit configuration, custom role, firewall, route, network, bucket IAM and Cloud SQL changes, plus organization policy changes, service account key creation, break-glass sign-ins, HIGH and CRITICAL Security Command Center findings and sink changes. Security Command Center also publishes active HIGH and CRITICAL findings to the Pub/Sub topic acme-security-alerts.

In the Web App Blueprint, the API writes JSON lines with a severity field to stdout, which Cloud Logging stores in the stage project; the GKE Gateway logs every request; every plat project enables Cloud Trace and the application identity holds roles/cloudtrace.agent; and Managed Service for Prometheus collection is on.

Drift as a signal​

Observability includes the infrastructure itself. In every edition drift-detection.yml runs at 01:00 UTC, plans every folder on Monday and the critical folders from Tuesday to Friday (an @optional schedule on Azure), and opens or updates a GitHub issue labeled drift when reality differs from the code; a plan that cannot run opens an issue too. The critical folders are the network, identity and three active workload accounts on AWS; core-network, core-identity, core-security and the three active plat subscriptions on Azure; core-network, core-identity and the three active plat projects on GCP. The drift page explains what to do with such an issue.

How you verify it​

AWS Enterprise Baseline​

  • CloudTrail console in the management account: one organization trail, multi-region, log file validation enabled, delivering to the audit bucket.
  • CloudWatch console in the security account, Settings, Monitoring account configuration: the sink and its linked source accounts; Alarms: seven alarms in the acme/CISBenchmark namespace.

Azure Enterprise Baseline​

  • Monitor, Activity log, Export Activity Logs in any subscription: the setting acme-activity-log-to-audit to the audit workspace and storage.
  • Monitor, Alerts, Alert rules in acme-core-audit and acme-core-security: the six KQL rules and the Defender rule, each with ag-acme-security-alerts as action group.

GCP Enterprise Baseline​

  • gcloud logging sinks list --organization=161616161616: acme-org-audit and acme-org-archive, both including child resources.
  • Monitoring, Alerting in acme-core-security: 13 alert policies and the email notification channel.

In every edition, the GitHub issues list of the Baseline repository filtered by the drift label shows the infrastructure signal.