Sara Hooker
@sarahookr
This is actually more a side-effect of the death of pretraining.
As transformers saturate, most pretraining results are tweaking data recipes, not massive stepwise changes in performance.
Turns focus to innovation in postraining and test time compute. This is good for competition, since pretraining concentrated resources in a few labs.
