Yann LeCun
@ylecun
@andrzejdragan @francoisfleuret It's absolutely NOT auto-regressive token prediction in the sense that multiple hypothesized sequence of tokens/actions can be evaluated without being produced on the output.
That's what I mean by SEARCH.
Though there are way more efficient ways to perform search than generation-selection of discrete token sequences.
Some LLMs do search when they produce multiple outputs (auto-regressively) in silence and then evaluate them and select the best one.
The overall process is not auto-regressive token prediction.
