Sara Hooker

I literally had the debate at a party recently. Pretraining model size scaling is dead because transformers are saturated. Only gains now are data. The group found it to be a very controversial statement 😂