4.9 Infrastructure as Code

We describe managed cloud resources and their IAM grants in Terraform, review them like any other code, and apply the reviewed plan. Existing unmanaged projects and documented external prerequisites remain outside that ownership boundary. Human group membership is one such boundary: directory administrators maintain it in Google Workspace or Cloud Identity, not in Terraform.

Infrastructure and application delivery have separate owners. moonitoring-infra owns permanent resources and IAM: projects, networking, Cloud SQL instances/databases/identities, storage, messaging, secrets metadata, registry, build connections/triggers and delivery pipelines/targets. Application repositories own code, images, deployment manifests and schema migrations. Cloud Deploy owns the rollout of application Cloud Run Services and Jobs from those manifests. Terraform must not also manage those same Service/Job specifications or reset their images, environment or traffic. Any ownership transfer is explicit and reviewed; it does not delete an existing service. Infrastructure supplies resource IDs and deployment parameters; application migrations use their scoped SQL identity and do not administer cloud IAM or cross-system database roles. This is the integration contract, not a claim that the delivery pipelines or workloads are already active.

Terraform works in two steps. A plan compares the code with the cloud and lists what would change; an apply carries out a plan. Between the two, a person reads the plan.

Status reconciled on 7 November 2026

The 5–6 November records cover bootstrap/stage provisioning, group IAM grants, application CI and the first Access delivery. Access has an application rollout approval gate; this is separate from the still inactive automatic Terraform apply/drift triggers. infra-admins has shared/stage grants and Jacopo and Lorenzo were added to stage-release-admins on 6 November. Production is not activated. This is a reconciliation of dated code and operational evidence, not a new live cloud audit: see the stage baseline register for sources and remaining checks.

4.9.1 The Repository

All infrastructure lives in the moonitoring-infra repository. Terraform adopts the organization's existing folders (10-shared, 30-stage, 40-production, 90-sandbox) and never creates, renames or deletes a folder. There is no development environment in the cloud (decided 2026-11-11): the 20-development folder, its Terraform root and its identities, left from the earlier design, are to be removed. Projects that already existed before Terraform, in 40-production and 90-sandbox, are left alone.

Folder	Describes	Applied
bootstrap/	The seed and CI projects, the state buckets, the Terraform and build identities, the pipelines and the budget	By an explicitly authorized foundation operator; group membership does not grant organization administration
org/	Baseline organization policies on selected shared/nonproduction folders; currently shared and stage only	By an explicitly authorized policy operator; infrastructure-group membership alone does not grant this permission
envs/30-stage/, envs/40-production/	The permanent resources, IAM and delivery parameters of each environment; application Service/Job rollouts follow the ownership contract above	Currently reviewed manual stage plans; future pipelines require the approval gates below
envs/10-shared/, envs/90-sandbox/	Reserved	Not yet
modules/	Reusable resource and access definitions, including application-ci and workload-access; the common application delivery module is being introduced by P1	Never on their own; the legacy service module does not also own workloads released by Cloud Deploy

A folder with its own state is a root. Each root keeps its state, Terraform's memory of what it built, in moonitoring-infra-seed: one bucket for bootstrap and org, and one for each environment, so an environment's identities can reach only its own state. Versioning is on and older versions are deleted after 30 days.

There is one shared bootstrap and one CI project, moonitoring-cicd, for these environments. Environments are Terraform roots and activation settings, not long-lived Git branches; there is no separate stage branch or stage bootstrap.

Application onboarding follows Application Delivery. First registering a component or changing its resource permissions requires a reviewed Terraform configuration and plan. A release that changes only application code reuses that configuration. Build/release profiles are supported explicitly; a new execution mode can require extending the common modules. A repository does not receive an arbitrary set of IAM roles by requesting them in its own build file.

4.9.2 From Pull Request to Cloud

This is the target workflow after the required repository protections and triggers are activated. Plans, identities and state are limited to explicitly active environments. Production paths are inactive, and production approval activation is blocked while the shared-project approval boundary described below is unresolved.

sequenceDiagram actor eng as Engineer participant gh as GitHub participant cb as Cloud Build actor rev as Reviewer participant gcp as Google Cloud eng->>gh: Pull request to main gh->>cb: infra-pull-request cb->>gcp: Plan active environments (read-only) cb->>gh: Plans as a comment rev->>gh: Review and approve (CODEOWNERS) eng->>gh: Merge gh->>cb: infra-apply-nonprod pending approval rev->>cb: Approve the build that includes stage cb->>gcp: Plan and apply selected active nonprod environments Note over gh,gcp: Future prod path requires a resolved approval boundary eng->>gh: Release tag vX.Y.Z gh->>cb: infra-plan-prod (future) cb->>gcp: Save the prod plan with the tag rev->>cb: Read the saved plan, approve infra-apply-prod cb->>gcp: Apply exactly that plan to prod

1. Pull request. The pipeline checks formatting, validates, runs the tests and lints every root, then plans active environments with their read-only identities. The plans are posted on the pull request.
2. PR review. The documented policy calls for another code owner to read the plans and approve the pull request, with the relevant checks passed and no merge conflicts. Jacopo and Lorenzo review and approve each other's PRs. PR review and cloud release approval are separate decisions.
3. Stage gate. When stage is active, the future infra-apply-nonprod trigger requires Cloud Build approval before the build starts. stage-release-admins@moonitoring.com is the intended approver group. Approval covers the build, before it generates and applies its plan, not a review of that exact saved plan. After approval, it plans and applies the selected active nonproduction environments, development before stage when both are selected; a failure stops the sequence. Stage-only selection remains available. The gate is separate from PR review.
4. Prod, after its separate activation. Prod is released by a tag, vX.Y.Z, on a commit that stage has already applied successfully. The tag saves a production plan, with a readable copy, next to the tag. A second person reads it and approves the build, and the pipeline applies exactly that saved plan. If the state changed in the meantime, Terraform refuses the plan and a new tag is needed. The existing production approver group is release-admins@moonitoring.com; no production approval grant is live. Do not use the uncreated naming proposal prod-release-admins as a principal.
5. Drift check. The future weekday 06:00 UTC check plans active environments again. Any difference between code and cloud fails the check and is fixed through a pull request.

stage_approver_groups defaults to an empty set of group: principals. Enabling nonproduction triggers with stage active requires this set to be nonempty and makes the nonproduction apply trigger require approval. The stage approver IAM grant is created only when those stage triggers are enabled. Production's existing prod_approver_members input remains reserved for a future activation.

The automatic check that a production commit has successfully passed the stage apply and required checks is a production prerequisite still to implement and verify; a stage approval alone is insufficient. Application promotion separately requires a successful stage rollout and required tests for the candidate SHA and image digests. The workflows here do not certify either production check as active.

The shared CI project does not provide separate approval scopes by environment. The Cloud Build Approver role permits approval of builds in its project. Consequently, configuration rejects enable_prod_triggers = true while stage_approver_groups is nonempty. Production approval activation stays blocked until a scoped solution is reviewed; no extra CI project or configuration flag claims to solve that boundary here.

4.9.3 Identities

The tables describe supported deployment profiles using the account IDs declared by Terraform. The existing stage identities are sa-tf-plan-stage, sa-tf-apply-stage and sa-tf-foundation, with sa-cloudbuild-plan and sa-cloudbuild-apply in the shared CI project. These IDs are retained. The production paths are not activated by naming their profiles here.

Pipelines hold no keys. A build runs as its own service account and borrows a Terraform identity for the time of the run. Each environment has two Terraform identities, one that reads and one that writes, each limited to its own folder and its own state:

Identity	May	Used by
sa-tf-plan-stage, sa-tf-plan-prod	Read their environment, change nothing	Pull-request plans, the drift check, the prod release plan
sa-tf-apply-stage	Change its environment	The future nonproduction apply, after build approval when stage is active; authorized manual applies today
sa-tf-apply-prod	Create projects in 40-production and change what it created; never the projects made by hand	The prod apply, after approval
sa-tf-foundation	Administer the folders Terraform owns entirely	People only, never a pipeline

Build identity	Borrows
sa-cloudbuild-plan	Read-only identities of active environments only
sa-cloudbuild-apply	sa-tf-apply-stage, only when stage is active
sa-cloudbuild-release-plan	sa-tf-plan-prod, and writes the saved prod plans; future production activation
sa-cloudbuild-apply-prod	sa-tf-apply-prod, and reads the saved prod plans; future production activation

A pull request can change the pipeline files, so whatever a pull request runs as must never reach a writer: sa-cloudbuild-plan borrows only read-only identities and cannot see the saved prod plans.

4.9.4 Rules

• Changes to Terraform-managed resources and IAM grants are pull requests. A plan detects differences for the resources it manages; it does not prove that all cloud access is declared in code. The additive *_iam_member grants preserve unrelated assignments, so manual extra grants can remain and must be audited.
• Terraform owns group-to-resource grants; the directory owns people-to-group membership. infrastructure_admin_groups defaults to []; its intended principal is group:infra-admins@moonitoring.com. It grants foundation impersonation plus plan/apply impersonation for every explicitly active environment, including production only after activation. It preserves the shared and managed-project boundaries, not access to unrelated legacy projects. Existing foundation_admin_members and nonprod_operator_members grants remain; the latter still grants only active nonproduction access and requires review before adding environments. team_observer_groups supplies separate observer grants. stage-infra-admins receives none. Membership changes follow the access process; no Terraform run is required for each new colleague.
• Use predefined roles unless the boundary requires a narrower role. The proposed stage SQL observer is an explicit custom-role exception: metadata reads without the predefined Viewer's backup export permission. This does not create a SQL login or Studio access.
• Migrate access in separate steps: add group grants while retaining the existing named grant, verify directory membership and the group-derived path, then review removal of the superseded named grant and test again. Keep a verified recovery path through an already authorized administrator. Organization-policy delegation and other pre-existing personal grants require their own review; they are not silently transferred by changing a Terraform allowlist.
• Directory groups do not provision product authorization. Application users, capabilities, tenant memberships and entitlements remain owned by the application access model.
• Keep secret values out of Terraform configuration, outputs and plans. Use references and a reviewed provisioning mechanism; use write-only/ephemeral features only where the provider actually supports them. A provider may otherwise retain a value in state, even when marked sensitive, so state access and that risk require explicit review. The plan identity can read state. See Secrets Management.
• Tool versions are pinned: Terraform, the providers and the linter, and container images by digest.
• A project is protected from deletion by default; deleting one needs a pull request that removes the protection first.
• A second person reviews and approves a release. Cloud Build does not by itself enforce that the approver differs from the author. The broad infrastructure administrator group can impersonate apply identities and operate directly outside the trigger. Approval gates govern the configured pipeline; permissions and review procedure do not create hard author/approver separation or require approval for every Google Cloud action.