Data Quality
The value of any observation dataset is determined entirely by the quality of what goes into it. Sentinel Watch is built around a single principle: the data we deliver must be accurate enough, consistent enough, and relevant enough to be used directly — without cleanup, correction, or guesswork on your end.
What Data Quality Means in Practice
In ML training data, quality is not a single metric. It is the combination of several properties that together determine whether a dataset will improve your model or degrade it. Sentinel Watch observation data is designed to meet all of them.
Accuracy
Every observation is classified by a vetted, briefed observer who understands your taxonomy and has applied it to exactly the situation they are documenting. The classification is not inferred, generated, or approximated — it is a direct, contextual human judgment made at the moment of observation.
Consistency
All observers working on your program are briefed on the same classification schema and boundary cases before deployment. Inter-rater agreement is monitored throughout the program. Observers whose classifications drift are retrained or replaced. The result is a dataset where the same event type is labeled the same way, regardless of which observer documented it or where.
Relevance
Observers are deployed to the specific environments, locations, and time windows where your target events actually occur. The dataset reflects what your model needs to learn — not what happened to be easy to collect. Every observation is captured against your taxonomy, not a generic schema.
Completeness
Every submitted observation is checked for completeness before it enters the review queue. Required fields, metadata, timestamps, and geolocation are verified at capture. Incomplete records are flagged for observer follow-up rather than passed through with missing values.
Integrity
Timestamps and geolocation are recorded at the moment of capture, not at submission. Metadata cannot be retroactively modified. The observation record reflects what the observer saw, when and where they saw it — not what was entered later.
Diversity
Programs are designed to capture the full range of scenarios your taxonomy covers — not just the common ones. Edge cases, boundary conditions, and low-frequency event types are tracked separately to ensure your dataset covers the distribution your model needs, not just the distribution that is easiest to observe.
The Quality Control Pipeline
Quality is enforced at every stage of the observation pipeline, not just at delivery. Problems are caught and corrected as early as possible — ideally before they enter the dataset at all.
- Observer briefing — Before deployment, every observer is briefed on your classification schema, event definitions, and boundary cases. The briefing includes worked examples and a competency check. Observers who do not meet the standard are not deployed.
- Structured capture — The observer reporting tool enforces your taxonomy at the point of entry. Free-text fields are minimised. Required fields cannot be skipped. This prevents the most common sources of annotation noise before they occur.
- Automated validation — On submission, each observation is checked for schema compliance, completeness, and metadata integrity. Records that fail validation are returned to the observer for correction rather than queued for review.
- Human review — A second human layer reviews all submissions for taxonomy compliance and classification accuracy. Ambiguous or borderline classifications are adjudicated against your defined schema. Records that cannot be confidently classified are rejected rather than passed through.
- Agreement monitoring — Inter-rater agreement is calculated continuously across observers working on the same event types. Systematic drift triggers rebriefing. Persistent non-compliance triggers replacement.
- Delivery — Only records that have passed all validation and review stages are included in your delivered dataset. Rejection rates and quality metrics are reported alongside the data.
Quality Reporting
Every dataset delivery includes a quality report covering submission volume, validation pass rate, review rejection rate, inter-rater agreement scores, and any taxonomy boundary cases that were adjudicated during the program. This gives your ML team the metadata they need to assess dataset quality independently, not just take our word for it.
Want to See a Sample Quality Report?
We can walk you through a sample observation dataset, the quality metrics that accompany it, and how our quality controls map to your specific model training requirements.