4.2 System Overview
Moonitoring Platform is the whole ecosystem: every system we run, the interfaces people use, and the rules that bind them. It is distributed and modular: it is made of small systems, each responsible for one thing, which can change, scale and fail without taking the others down. A new product starts by adding systems, not by growing an existing one.
4.2.1 How the Platform Is Organized¶
We organize the platform the way large engineering organizations catalog theirs, on three levels:
- A domain is an area of the business, such as Farm Data or Intelligence. It groups the systems that serve it.
- A system has one responsibility, owns its data and exposes it only through versioned interfaces. Access Control decides who may do what; Farm keeps the canonical farm data. A system has one owner.
- A component is a piece of software inside a system: a service, a job, a worker. Components are deployed; systems are what the rest of the platform sees.
Interfaces are not systems. The Console, the Organization Dashboard, the farm sync client and MooGrant own no data and decide no business rule: they call the APIs of the systems they need.
The rules that follow:
- A system never reads another system's data. It calls the other system's API or reacts to its events (see Choosing an Interface).
- A system is split when one of its parts gets a responsibility, an owner or a load of its own, not before.
- Every system has a short lowercase name, used in its host name, its API paths, its capabilities, its code and its repository:
access,farm,ai-gateway(repositorymoonitoring-system-ai_gateway, see Repository Naming).
4.2.2 Domains and Systems¶
| Domain | System and its interfaces | Responsible for | Data it owns |
|---|---|---|---|
| Identity & Access | access (Access Control) |
Who people are, which Organizations they belong to and what they may do; the access of Moonitoring staff; the audit of privileged actions | Organizations & Access, except entitlements and feature usage, which farm owns |
| Farm Data | farm |
What each Organization has purchased and how much of it it uses; bringing farm data in (sources, channels, devices, feeds, raw files, parsing, the runs of the farm sync client) and keeping it: the canonical farm data, animals, observations, events and periods, their reconciliation, the write number, and submissions to official bodies | Provenance & Ingestion, sync runs, clients, schedules and digests, Animals, Observations, Events & Periods, Reconciliation, writes, erasure requests, official submissions, entitlements and feature usage |
| Intelligence | moochat (MooChat) |
Agents, their tools and their conversations with users and staff | Agent Platform |
| Intelligence | ai-gateway |
Every call to a language model: routing by data class and plan, provider keys, token use per Organization | None yet |
| Intelligence | ai-model-platform (AI Model Platform) |
Our own models: datasets, training, model runs and the model APIs that other systems call | Datasets and model runs |
| Funding | grant (MooGrant) |
Helping farms apply for funding: funding calls and farmer documents | None yet |
| Distribution | releases |
Builds, channels, releases and installations of our installed clients | Client Distribution |
| Operations | status |
Whether the platform works: probing every component a customer uses, incidents and maintenance, uptime, and the status page at status.moonitoring.com, which keeps answering when Google Cloud is down |
Components, probes, incidents, maintenance and subscribers |
| Content | news (MooNitoring News) |
The articles staff write in the Console, in every language of the site, and the static public site at news.moonitoring.com |
Content |
4.2.3 Interfaces¶
| Interface | Used by | Calls |
|---|---|---|
| Organization Dashboard | Organization users of every type: farms, cooperatives, insurers and professionals, each shown what its type needs | access, farm, moochat, grant |
| Account | Every person, before and after they belong to an Organization: sign-in, registration, profile, invitations, join links, and the members, groups and roles of their Organizations; a web app written in Nuxt | access |
| Console | Moonitoring staff, after a Google sign-in check (see Identity & Access) | The admin APIs of every system: one Console, with a section per system |
| Farm sync client | A computer at the farm, which always calls us: nothing on our side ever connects into a farm | farm to upload, releases to update itself |
| MooGrant | Farmers looking for funding calls, as guests or signed in; a public web app written in Nuxt | grant, access to sign in |
| News site | Public readers at news.moonitoring.com; a static site written in Nuxt, in moonitoring-app-news |
news, only when it is built |
| Status page | Customers and anyone checking whether we work, at status.moonitoring.com; a static site written in Nuxt, in moonitoring-app-status; behind IAP in stage |
status, through its feed files |
| Dev ORM | Developers, with temporary, approved access | The database of one system: each system has its own (see below) |
Every system has its own Dev ORM, a component built from the same commit as the system's migrations, so it always matches the schema it opens and never reaches another system's tables. It writes only through the system's own data layer (decided 2026-11-05): a correction to versioned data is a WriteCommit of kind manual_correction, with the developer, a reason and, in prod, the approved grant, that appends a revision and emits the events the same change would emit through the API; a table that is not versioned is changed in one transaction with an audit event holding the row before and after. It never writes the outbox, audit rows, credentials or keys, and personal data is removed by an erasure request, never by a Dev ORM delete. It is a Cloud Run service of its system, reached at /devorm/<system>/ on the Console's host behind Identity-Aware Proxy (decided 2026-11-05); in prod a developer opens it only with the temporary, approved access of Identity & Access, and the Console links to the Dev ORM of each system. The part they share, the screens, the sign-in check and the statement log, is a library in moonitoring-common-rs.
One user system serves every product (decided 2026-11-05). People, Organizations, memberships, groups, invitations and join links exist once, in access, and every app signs in through it. Inviting someone or managing members is built once: in the access API and in the Account app, which every product links to, so a new dashboard adds no second way to invite people and no table of users of its own. The other systems keep a cached copy of memberships and entitlements, built from the events of access and farm: a copy that only access writes, which lets each system check rights itself without calling access on every request, never a second source of the same data. How a copy stays right (decided 2026-11-05): each row keeps the time of the last change applied, and an event or a snapshot older than it is ignored; a snapshot removes only rows of the kinds it streams, and only when no later event changed them. A copy is filled and repaired by the system's sweeper, never by a consumer at a cold start: on the sweeper's first run after a deploy finds it empty, then once a day, and for one Organization when an event names changed fields without their values. A cold start stays fast, and a consumer that meets an event for a row it lacks asks the sweeper's repair endpoint for that Organization.
Products are sections of one Organization Dashboard, shown to an Organization when it holds their entitlements, so a person signs in once and switches Organization in one place. A product gets an app of its own only when its users are different people, as MooGrant serves farmers who arrive from a search.
4.2.4 Who Uses Them¶
- Organization users are the people of our customers, whose Organization is always one of four types (decided 2026-11-11): a farm, a cooperative, an insurer, or a professional such as a veterinary practice, an advisor, a consultant or a laboratory. Only a farm holds animal data; the others reach it through the delegations farms give them. They sign in through
access, choose the Organization they act as (Choosing the Organization) and work in the Organization Dashboard. - Moonitoring staff use the Console, which shows a section for each system and calls that system's admin API.
- Data scientists work on datasets in
ai-model-platform, never on production databases. - Farmers looking for funding use MooGrant, as guests or with an account; a signed-in farmer gets a Farm Organization of their own.
- Public readers read
news. - Vendor clouds are the clouds of milking-robot and herd-software makers that
farmpulls data from; official bodies receive submissions such as milk recording fromfarm.
4.2.5 How the Systems Talk¶
- A system verifies a request itself:
accesssigns a short-lived token, and the system checks the user's role and the Organization's entitlement against its own copy of that data. accessandfarmpublish events when an Organization, a membership or an entitlement changes; the other systems keep their copy up to date from those events.- Systems call each other only through their APIs and events, chosen as described in Choosing an Interface. Data for models travels as datasets, not as database access.
4.2.6 Where the Systems Run Today¶
The systems that need PostgreSQL use Cloud SQL in Milan. The table maps their intended runtime hosting to the projects of the resource hierarchy; it is not an inventory of deployed workloads. The PostgreSQL databases physically live in the environment's database project, separately from the runtime projects. Stage retains project ID moonitoring-db-stage and instance sql-stage; new environments follow moonitoring-<env>-db:
| Runtime project or hosting | Systems | Database or storage |
|---|---|---|
moonitoring-<env>-platform; existing stage ID moonitoring-platform-stage |
access, releases, ai-gateway, the backend of news, and the Console |
access, releases, ai_gateway, news, and the media bucket of news |
moonitoring-<env>-farm; existing stage ID moonitoring-farm-stage |
farm, moochat, and the Organization Dashboard |
farm, moochat |
moonitoring-<env>-grant |
grant |
grant |
moonitoring-prod-ml-lab, moonitoring-prod-ml |
ai-model-platform |
Datasets in Cloud Storage |
| Cloudflare, outside Google Cloud | The static news site at news.moonitoring.com, the status page at status.moonitoring.com with status-watchdog, and the landing pages |
The feed files of status in R2 |
This documentation is an internal tool, not a system: it runs on Cloud Run in the existing project moon-internal-prod-tools, behind Identity-Aware Proxy, and Cloud Build publishes it on every merge to main.
Each PostgreSQL-backed system has its own database, named after it, on the Cloud SQL instance of its environment: access, releases, ai_gateway and news for the systems hosted in the platform project, farm and moochat for those hosted in the farm project. The required boundary is enforced through PostgreSQL privileges: each component identity must reach only its own system's database with the permissions its role needs. The target deployment runs the system's migrations as a separate Cloud Run job before deploying its workloads.
Systems that share a project must not read each other's tables: each system owns its own tables and reaches the others through their interfaces, so that its runtime can move to a project of its own without changing its callers.
Systems do not get a project each (decided 2026-11-04): they share the projects that the Terraform in moonitoring-infra creates, and a system moves to a project of its own only when its load, its access or its owner asks for it.
Stage phase 1¶
This section records the 5 November provisioning phase, not the complete current state. Access SQL checks, delivery and telemetry work followed on 6 November. Use the 7 November baseline register to distinguish that evidence from remaining work, and Application Delivery for the common onboarding contract.
Provisioned and verified on 5 November 2026. This phase added four empty databases, access, releases, ai_gateway and moochat, alongside the existing farm database in the stage database project moonitoring-db-stage. This ID and instance sql-stage are retained when the project's display name is aligned. The legacy platform database remains. Grant is deferred. The apply created 14 resources with no changes or deletions; live checks and a fresh complete plan confirmed the result with no further changes. See infrastructure PR #9 and the system databases runbook.
The phase also created ten new Google service accounts: one for each system's initial API component and a separate migration identity, listed in Identity & Access. It created no new SQL IAM users, roles/cloudsql.client or roles/cloudsql.instanceUser grants, PostgreSQL privileges or application deployments. The four existing legacy runtime identities and their access were not disabled by this phase.
At the end of phase 1, creating empty databases did not establish SQL isolation. PostgreSQL normally grants CONNECT and TEMPORARY on a new database to PUBLIC, which includes existing roles; creating a database or a Google service account does not remove those defaults. The later Access SQL verification records positive and cross-database negative checks for that system. Every new system still needs its baseline privileges and tests; the existence of databases is not permission to use them. See PostgreSQL privileges.
Additional worker identities and each system's Dev ORM remain future work. The Dev ORM's interactive access is a separate authorization step; creating the API or migration identity grants no implicit access to it.
References¶
- Backstage system model: domains, systems, components and APIs.
- Domain-Oriented Microservice Architecture at Uber: grouping services into domains with one entry point each.
- Team Topologies: one owner per system, interactions through interfaces.