Skip to content

6.9.2.4 MlDataset

6.9.2.4.1 Purpose

A named, reusable selection of canonical data for model work, created and managed by data scientists in the Console. It says what to extract (tables, time ranges, animal types, breeds, origins, nations, Organizations), with which export code and at which anonymization level; it holds no data itself. Every build produces an immutable MlDatasetVersion, so a definition can be rebuilt as the data grows while every past build stays exactly as it was.

6.9.2.4.2 Questions it answers

  • Which datasets exist, who owns them and what each one selects?
  • At which anonymization level may this dataset be read, and from where?
  • How long are old versions kept once a newer one is ready?
  • Is the dataset still being built, or archived?

6.9.2.4.3 Fields

Field Type Cardinality Description
id string (uuid) 1..1 Key.
slug string 1..1 Unique.
e.g. mastitis-holstein-2024-2025.
name string 1..1
description string 1..1
owner MoonitoringUser 1..1
selection MlDatasetSelection 1..1 What the build extracts; copied into every version, so editing it never changes an existing version.
feature_code_version string 1..1 Version (commit or hash) of the export code: tables, columns, time grid, pseudonymization. Same meaning as 62.4.
anonymization enum 1..1 Values: identified, pseudonymized, anonymized. Default: pseudonymized.
identified: farm and animal ids as they are; only for scoring and pricing in production, never readable from ml-lab.
pseudonymized: ids replaced through a keyed hash (key in Secret Manager, one per environment); still personal data.
anonymized: ids replaced by random values with no key, locations and dates coarsened; what may leave the company.
status enum 1..1 Values: draft, active, archived. Default: draft.
archived: no new build; existing versions stay readable until their keep_until.
retention_days integer 0..1 Sets keep_until of a version when a newer one becomes ready. Empty: versions are kept until deleted by hand.
system_version SystemVersion 1..1
created_at datetime 1..1
updated_at datetime 1..1

6.9.2.4.4 Relationships

References

Referenced by

Uses types

MlDatasetSelection, SystemVersion

6.9.2.4.5 Rules & constraints