Hamel Husain
@hamelhusain
Can you use Jev for Evals? Yes! Remember that a LLM Judge is also classifier*.
Make sure to test your classifiers against human labels and don't overfit.
Hope this helps!
* https://hamel.dev/blog/posts/evals-faq/why-do-you-recommend-binary-passfail-evaluations-instead-of-1-5-ratings-likert-scales.html
