Skip to content

5.8.1 Introduction

ai-model-platform, the AI Model Platform, manages Moonitoring-owned models and the assets needed to use them: datasets, reproducible data/model runs, model versions and deployments, online inference, and batch inference. It exposes controlled model operations to product systems and writes domain results back through Farm contracts. It belongs to the Intelligence domain of the platform. Every interface it offers follows Choosing an Interface.

Proposal for review

The contracts of this system are a proposal, not confirmed design: endpoints, events and calls were written from the refined requirements, the data model and the flows, and wait for review. What needs a decision is collected in Open Decisions.

5.8.1.1 Host Names

Environment Host
Prod ai-model-platform.moonitoring.com
Stage stage-ai-model-platform.moonitoring.com, behind IAP

The names follow the host name rule; on a developer's computer each service answers on a local port (Running Locally). Each domain below is a path on these hosts, followed by its version: https://ai-model-platform.moonitoring.com/<domain>/v1.

5.8.1.2 Domains

Each domain of ai-model-platform is one API, with its own contract and its own version.

  • 5.8.2.1 Datasets

    Datasets for model work: a named selection of canonical data, built at a write number into immutable versions of Parquet files in the datasets bucket. /datasets/v1

  • 5.8.2.2 Model Runs

    Every use of the data by a model, so it can be rebuilt exactly: training, scoring, pricing and analyses, with the write number read, the feature code version, the rule set and the knowledge clock. /runs/v1

  • 5.8.2.3 Models

    The model registry: models, their versions produced by runs, which version is deployed, and the online or batch inference operations that write domain results back to farm. /models/v1

5.8.1.3 How It Is Used

Dataset building

  1. A data scientist defines a named selection in the Datasets domain through the Console or a notebook workflow, never by reading the database.
  2. AI Model Platform asks Farm for an export at a selected write number.
  3. The resulting dataset version is immutable, addressed by its manifest, and readable by ml-lab notebooks with pseudonymized identifiers.
  4. A variant is cut from the version's files on the server when a narrower selection is needed.

Model lifecycle

  1. A training run records the dataset and feature inputs used to produce a model version.
  2. Staff register the model version, deploy it, or retire it through the Models domain.
  3. Farm receives deployment changes and lists the available model versions for Organization-facing use.

Online and batch inference

  1. Farm asks AI Model Platform for online inference through ModelsService, or starts batch inference over a defined collection of inputs.
  2. AI Model Platform executes the deployed model and records the model run and its input/output metadata.
  3. Results are written back to Farm as computed records when the model operation produces domain data for other product systems.

Reproducibility

  1. Every run records the write number, feature-code version, rule set, and knowledge clock it read.
  2. A later reviewer can identify the exact immutable dataset version and inputs used by the run.
flowchart LR scientist{{Data scientist}} -->|REST and notebooks| modelPlatform[AI Model Platform API] farm{{Farm}} -->|gRPC exports and inference| modelPlatform console{{Console}} -->|REST administration| modelPlatform modelPlatform -->|computed records| farm modelPlatform -->|membership snapshot| access[Access]
  • Called by: Console (REST); access (gRPC CatalogService); farm (gRPC CatalogService, gRPC ModelsService); moochat (gRPC CatalogService).
  • Calls: access (gRPC OrganizationsService); farm (gRPC ComputedRecordsService, gRPC EntitlementsService, gRPC ExportsService, gRPC WatermarkService).
  • Publishes events consumed by: farm (2 event types).
  • Consumes events of: access (1 entry), farm (4 entries).
  • Reads no other system's database: everything above goes through a contract on this page or an event.

5.8.1.4 Synchronous Calls Between Systems

ModelsService (list, predict) in Models. It calls WatermarkService, ExportsService and ComputedRecordsService of farm.

5.8.1.5 Events

Events are facts in the past tense, published through the outbox and delivered at least once on the topic ai-model-platform.events.v1; the envelope and the guarantees are in common/v1/events.yaml. Events marked outside reach the Pub/Sub subscriptions an Organization registers in Webhooks, pushed to its HTTPS endpoint or pulled by its own service account.

Publishes (contract: events/v1/asyncapi.yaml):

Event Published when Consumers Outside
dataset_version.ready.v1 A dataset version was built and is readable by its manifest. none yet no
dataset_version.failed.v1 A build failed. none yet no
model_version.deployed.v1 A model version was deployed and answers predictions. farm no
model_version.retired.v1 A model version was retired and refuses calls. farm no
scoring_batch.completed.v1 A batch scoring finished; its predictions are in farm as computed records. none yet no

Consumes:

Event From Why
dataset_export.completed.v1 farm Marks the version ready or failed.
write_commit.committed.v1 farm Knows when the watermark advanced, for scheduled builds.
feed_coverage.updated.v1 farm Keeps labels before the settled date of each feed.
staff_membership.changed.v1, staff_role.updated.v1 access Authorizes data scientists.
erasure_request.executed.v1 farm Rewrites kept datasets and versions.