Q: Can I use the same model for both the main task and evaluation?
A: Usually, yes. Test whether the judge agrees with human labels on a held-out test set.
https://hamel.dev/blog/posts/evals-faq/can-i-use-the-same-model-for-both-the-main-task-and-evaluation.html
