2026-09-02
Who checks the checker?
This is how a lot of AI gets tested today. One model gives an answer.
This is how a lot of AI gets tested today. One model gives an answer. Another model scores it. If the score goes up, we ship. A person looks at a sample sometimes.
I do it too. People are slow and expensive and they disagree with each other.
But the judge learnt from the same internet as the thing it is judging. Where one has a blind spot, the other probably has it too. When both are wrong the same way, the score goes up.
At the startup we now keep one check that only a human does, every week. It is slow, and it has caught things the models agreed on.