Human Data & Annotation
Guideline design, inter-annotator agreement, preference collection, rater sourcing and label noise.
7concepts
74flashcards
51minutes of reading
- 01 Active Learning and Annotation Budgets How choosing which examples to label, by uncertainty or by coverage, can cut annotation cost, why cold starts and sampling bias undermine it, and why the gains that looked large for models trained from scratch often shrink or vanish with pretrained ones.
- 02 Guidelines Are the Model Specification Why the annotation guideline determines what the model learns more than the architecture does, what a usable guideline contains, and the iteration loop that produces one.