Skip to content

5.12.1 Introduction

status tells customers and staff whether the platform works: a status page at status.moonitoring.com with every component a customer uses, its uptime over 90 days, the incidents and their updates, and the maintenance we plan. It belongs to the Operations domain of the platform (decided 2026-11-10). It must keep answering when the platform is down, so the public page lives outside Google Cloud and says so when its data is late.

Proposal for review

The contracts of this system are a proposal, not confirmed design: endpoints, events and capabilities were written from the product owner's brief of 2026-11-10 and the practice of public status pages (GitHub's, on Atlassian Statuspage), and wait for review. Its entities and capabilities are not yet in the requirements file the data model is generated from.

5.12.1.1 Host Names

Environment Status page Staff API
Prod status.moonitoring.com, public, on Cloudflare Pages; /api/v1/ goes to status-watchdog console.moonitoring.com/api/status/
Stage stage-status.moonitoring.com, the Cloud Run service status-server behind Identity-Aware Proxy for infra-admins and the stage testers' group; /api/v1 goes to status-backend stage-console.moonitoring.com/api/status/
Local localhost:9600, the page; localhost:8900, the backend the backend directly

The host of status is its page, not its backend (decided 2026-11-13), the one exception to the host rule: the page reads /api/v1/summary.json from its own host, and the staff API is reached only through the Console.

The page and the system live in two repositories (decided 2026-11-10), as News does: the backend parts in moonitoring-system-status, the page, a Nuxt app built with the design system, in moonitoring-app-status.

5.12.1.2 What the Page Shows

The page shows only whether the servers work (decided 2026-11-10): no subscriptions, no history pages, no feeds for other tools. Its model is the front page customers know from GitHub's status page:

  • A banner with the overall state: all systems operational, or the worst open incident, or a maintenance in progress.
  • One row per component, grouped, with its status word and a bar of the last 90 days: one cell per day, coloured by how long the component was down that day, with the day's incidents on hover. The uptime percentage of the 90 days sits under the bar.
  • Open incidents and maintenance in progress above the components, each with its updates, newest first; scheduled maintenance of the next seven days below them.
  • Every text in Italian and English, the page's two languages, and a light and a dark theme the reader can switch.

The public page lists only what customers use. The stage page, behind IAP, also lists the internal components (the Console, the Dev ORMs, the event pipeline), so staff see the whole environment.

Each card of the page is one of our systems, and the components inside it say in plain words what that system does for a customer (decided 2026-11-10): the titles match the systems we run, and a customer still understands what is not working.

Card (system) Components (first set)
Access Sign-in and accounts
Farm Data sync; Data processing, once parsers run
Releases Client updates
News, MooChat, Grant, … One card each when the system is released to customers

A component is added when the product it stands for is released to customers, not before.

5.12.1.3 Statuses and How They Are Computed

Component status Meaning Counts as downtime
operational Works normally no
degraded_performance Works, but slowly no
partial_outage Fails for some customers or some operations 30% of its minutes
major_outage Fails for everyone every minute
under_maintenance Planned work announced in advance no

Uptime over a period is 1 − (major outage minutes + 0.3 × partial outage minutes) / minutes in the period, the formula of Atlassian Statuspage, so our numbers read the way customers expect. Announced maintenance does not count. A day's cell is green with no downtime, then shades towards yellow, orange and red as the day's downtime passes 1, 20, 40 and 60 minutes; grey before the component existed.

Incidents move through investigating, identified, monitoring and resolved, with an impact of none, minor, major or critical that colours the banner. Maintenance moves through scheduled, in_progress, verifying and completed, or is cancelled before it starts.

5.12.1.4 How It Works

flowchart LR scheduler[Cloud Scheduler] -->|every minute| backend[status-backend] backend -->|probes| edge[Public hosts of every system] staff{{Staff}} -->|Console| backend backend --> db[(status database)] backend -->|feed files| r2[(Cloudflare R2)] watchdog[status-watchdog<br/>Cloudflare Worker] -->|probes from outside Google Cloud| edge watchdog --> r2 reader{{Customer}} -->|status.moonitoring.com| page[status page<br/>Cloudflare Pages] page -->|/api/v1/*.json| watchdog

Probing

  1. Cloud Scheduler calls status-backend every minute. It runs every probe at once, each with a five-second limit: a GET of the readiness endpoint of each public host, through the same load balancer customers use, with the identity token of the prober's service account where IAP or Cloud Run requires one. Later, journeys add a real operation per component, such as issuing a token or a signed upload URL for a synthetic farm.
  2. A component's status follows its probes with hysteresis: partial_outage when some of its probes fail three runs in a row, major_outage when all do, degraded_performance when their 95th-percentile latency stays above one second for five minutes, and operational again after five passing runs. One slow minute never pages anyone.
  3. A component going to an outage opens an incident by itself, investigating, with a standard message in both languages, and alerts the on-call engineer. Staff then write what is happening. When the probes pass again the component returns to operational, but only staff resolve an incident: a fix is confirmed by a person.
  4. Staff can override a component (for example to partial_outage while the probes still pass) and hand it back to the probes; a paused probe changes nothing.

Publishing

  1. Every change is written with an outbox row in the same transaction, then published as an event on status.events.v1.
  2. status-backend renders the feed (the files of the Feed domain) and uploads it to a Cloudflare R2 bucket after every change and at least once a minute, with the time it was made and the time after which it is stale (three minutes).
  3. The page is static, served by Cloudflare Pages: it reads /api/v1/summary.json from its own host and draws everything from it. A page cached by the browser keeps working.

When Google Cloud is down

status-watchdog, a Cloudflare Worker, is the part that never depends on Google Cloud. It serves the feed files from R2 to the page, and every minute it probes the same public hosts from Cloudflare's network. If the feed is older than its stale time, the watchdog puts a notice into summary.json: "We cannot confirm the current status; we are investigating", and marks as major_outage every component whose probes also fail from Cloudflare. The page then shows the last known state, the notice and the time of the last update. When status-backend publishes again, its files replace the notice.

Staff can still post an incident during a Google Cloud outage: the break-glass procedure of the runbook writes a signed notice to R2 through the watchdog with a key kept outside Google Cloud, and the next publication by status-backend includes it as an incident.

Stage is the exception: the stage page does not need to survive Google Cloud, so it runs as a Cloud Run service behind IAP, and status-backend serves its feed directly, at /api/v1 of stage-status.moonitoring.com. A check after each deploy confirms that the stage page answers (decided 2026-11-13): it passes IAP with an identity IAP admits and loads the page and summary.json. There is no continuous uptime check behind IAP.

5.12.1.5 Components

Component Kind Does
status-backend Cloud Run service, minimum zero instances The staff API, probing every minute, the computation of statuses and uptime, publishing the feed
status-migrate Cloud Run job The schema of the status database
status-watchdog Cloudflare Worker, in Rust compiled to WebAssembly Serves the feed, probes from outside Google Cloud, warns when the feed is stale, receives the break-glass notice
status page Nuxt app in moonitoring-app-status: Cloudflare Pages in prod, a Cloud Run service behind IAP in stage What customers see

One component writes the database, status-backend, so the system fits the delivery profile accepted today (a service and its migration job, Application Delivery). Its Cloud Scheduler and Pub/Sub entry points need the acceptance of those profiles before stage.

5.12.1.6 Data and Retention

Data Kept
Components, their groups and probes While they exist
Probe results, one per probe per minute and vantage point 30 days
Status changes of each component 400 days
Daily downtime of each component, from which the bars and percentages come 400 days
Incidents, their updates, maintenance 400 days

The status page holds no customer data and no personal data: no Organization, farm or person appears on it.

5.12.1.7 Objectives

What Objective, prod
Availability of the status page 99.95% a month
An outage on the page Within three minutes of its start
A staff change on the page Within one minute

The targets of the systems themselves, measured by these probes, belong to SLI & SLO.

5.12.1.8 Security

  • Staff manage the page through the Console, with the staff token of access, and every change is an audit event. Proposed capabilities: status.incidents.read, status.incidents.manage, status.components.manage.
  • The break-glass key is held by the on-call engineers outside Google Cloud and rotated after every use.
  • Probes read readiness only: the readiness endpoint of a system answers whether it can serve, never its configuration or data.

5.12.1.9 Cost

On demand, like every system (Running on Demand): about 44,000 calls a month from Cloud Scheduler to status-backend, short and mostly within Cloud Run's free tier; one small database on the environment's Cloud SQL instance; Cloudflare Pages, R2 and Workers within their free tiers at our traffic. Expected under €5 a month per environment, an estimate to confirm with the first month of stage.

5.12.1.10 Domains

  • 5.12.2.1 Incidents

    Components, incidents and their updates, maintenance and probes, managed by staff in the Console. /incidents/v1

  • 5.12.2.2 Feed

    The one file the page reads, summary.json, served while Google Cloud is down. /api/v1

  • Called by: Console (REST); Public readers (REST); access (gRPC CatalogService); farm (gRPC CatalogService); moochat (gRPC CatalogService).
  • Calls: nobody yet.
  • Publishes events consumed by: nobody yet.
  • Consumes events of: access (1 entry).
  • Reads no other system's database: everything above goes through a contract on this page or an event.

5.12.1.11 Events

Events are facts in the past tense, published through the outbox on the topic status.events.v1. Today only status itself consumes them, to publish the feed.

Publishes (contract: events/v1/asyncapi.yaml):

Event Published when Consumers Outside
incident.opened.v1 An incident was opened, by the probes or by staff. status no
incident.updated.v1 An incident got an update, or its impact or components changed. status no
incident.resolved.v1 An incident was resolved. status no
component_status.changed.v1 A component's status changed, by the probes or by staff. status no
maintenance.changed.v1 A maintenance was scheduled, moved, started, completed or cancelled. status no

Consumes:

Event From Why
staff_membership.changed.v1, staff_role.updated.v1 access Authorizes the staff who manage incidents.

References