An 84% agreement score sounds like a finished system. It is not. It is the point at which the system becomes usable by a reviewer — and the reviewer's experience decides whether it survives contact with a real morning.
Three things a reviewer actually needs
- Evidence, not confidence. Show the source lines that drove the classification. A reviewer can check evidence in seconds; a confidence score tells them nothing they can act on.
- A fast correction path. Overriding a decision must be cheaper than living with it. If correction is slow, reviewers stop correcting and start distrusting.
- Visible controls. Which rules ran, which model ran, what was skipped. When the system is boring and inspectable, people trust it more, not less.
The takeaway
Optimise the reviewer's minute, not the model's score. The score follows.