Annotation guidelines are the real deliverable
Two annotators disagreeing about the same example is not a training problem. It is a definitions problem, and it is far cheaper to solve before collection than after.
3 min read
Insights
Thinking about AI data.
What we have learned about specification, annotation and quality — written for the people who have to make these decisions.
8 articles
Why first-person video is a different kind of input
Egocentric footage preserves attention, approach and sequence — spatial context that third-person capture quietly discards.
3 min read
Occlusion is where video datasets quietly break
Most annotator disagreement in video happens when an object is partly hidden. Deciding that once, in writing, is what keeps a track coherent.
3 min read
Transcription conventions decide your model's ceiling
How you write numbers, hesitation and overlap is not a detail. It is a decision the entire corpus inherits, and changing it later means starting again.
3 min read
When annotators disagree, look at the schema first
Persistent disagreement on the same kind of example is almost never carelessness. It is a label set that cannot be applied consistently.
2 min read
Synthetic data is a complement, not a replacement
Generated data is genuinely useful for coverage and for privacy-constrained domains. It is not a substitute for the real-world anchor.
2 min read
How to scope a first dataset
The most common mistake is committing to volume before the specification has been tested. A small end-to-end pass costs less and tells you more.
3 min read
Inter-annotator agreement, and what it actually tells you
A single agreement score is a weak summary of a rich signal. What matters is where the disagreement sits and whether it is structured.
3 min read
Have a data challenge?
Let’s build the dataset behind it.
Tell us what you’re building, what data you need and the scale of your project.