BJJ audits

Blind checks for a project that follows two grapplers through a whole match from competition livestreams. Each check asks a second rater to look again at a random subset of the first rater's labels, without seeing them.

How every audit runs

1. The question, the labels and the pass bar are registered before any frame is seen.
2. A first rater labels every frame; the label files are frozen by hash.
3. A second rater checks a seeded random subset here. These pages never carry the first rater's answers.
4. If the two raters agree less than the bar, the result is not reported as one.
5. Every result is published with its controls and its refusals.