Hamel Husain
@hamelhusain
I see some are drawing the wrong inference from this study: they are focusing on this meta benchmark as yet another benchmark for model picking, i.e. open vs closed models, model families etc.
That’s too narrow of an interpretation! You need to do your own private evals on your specific tasks to know what’s “better”
This should give you healthy skepticism of all public benchmarks. I believe that was the intention of this illustration to begin with.
