XResearch20:39Yann LeCun@ylecun@alexandr_wang 😅😅 Proprietary AI model companies are the IBMs of yesteryear 😅😅 (and the new IBM is actually a massive supporter of open source and nothing like the old IBM).Collected by WispRead in full↗
XResearch12:00Yann LeCun@ylecun@jakobmollerfi @JordanMich45361 @BachFrancis Yeah, like with multilayer nets, backprop, ConvNets, deep learning, and self-supervised learning. If you agreed with me you were clearly on the wrong side of history. 🤪Collected by WispRead in full↗
XResearch02:58Yann LeCun@ylecun@demishassabis Ah, BattleChess on the Amiga 🥲....Collected by WispRead in full↗
XResearch11:57Yann LeCun@ylecun@michaelkheifetz @random_walker The real question is why they have such drastically different opinions from us. They changed their opinions. We didn't.Collected by WispRead in full↗
XResearch02:54Yann LeCun@ylecun@random_walker These are all excellent points that I wish more people would know and understand. Particularly the people who should know better but are scaring everyone with predictions of cyber security apocalypse (in addition to job apocalypse, and plain apocalypse)Collected by WispRead in full↗
XResearch03:41Yann LeCun@ylecun@demishassabis My friend and Bell Labs colleague John S. Denker was the inspiration for the Mitch character in the movie. He got into Caltech at age 16, played a central role in a number of pranks (like winning a McDonald's sweepstake by printing thousands of entries using a high-speedCollected by WispRead in full↗
XResearch15:28Yann LeCun@ylecun@andrewgwils A generalisation theory that applies to self-supervised learning scenarios: 1. Unlabeled samples cost X per sample 2. Labeled samples for a given task cost Y per sample. - assume X is zero, what can you say about the regret curve for the task, i.e. the area under the curve ofCollected by WispRead in full↗
XResearch15:21Yann LeCun@ylecun@tdietterich @andrewgwils We would have to insist that we prefer models that can be expressed with a small number of short formula, using common mathematical primitives. That's an inductive bias and/or regularizers that symbolic regression employ. But then, why do we think this is good?Collected by WispRead in full↗
XResearch03:01Yann LeCun@ylecun@Helios_Hua Nice work! There's been some work along the same line by @garridoq_ @laurentnajman and me that you may not be aware of: https://t.co/Evxpx4MoIECollected by WispRead in full↗
XResearch22:56Yann LeCun@ylecun@parisofprairie @Kasparov63 I'm a big advocate of open source AI. You can now answer your own question.Collected by WispRead in full↗
XResearch22:49Yann LeCun@ylecun@Observer2077 @francoisfleuret A two-layer neural net is also Turing complete. That's irrelevant.Collected by WispRead in full↗
XResearch22:41Yann LeCun@ylecun@JoeBrilland @francoisfleuret Yes. That's no-longer auto-regressive. No-longer a "pure" LLM.Collected by WispRead in full↗
XResearch22:35Yann LeCun@ylecun@yieldthought @francoisfleuret Only a limited and ridiculously inefficient form of search.Collected by WispRead in full↗
XResearch22:53Yann LeCun@ylecun@Sam0xSol @Kasparov63 The behavior of a thermostat is largely predictable though.Collected by WispRead in full↗
XResearch22:47Yann LeCun@ylecun@yoshispeninsula @francoisfleuret False. Inference still does not involve any search.Collected by WispRead in full↗
XResearch22:40Yann LeCun@ylecun@LuKMalEcrit @francoisfleuret Right. They can only verify the correctness of their output by actually running it. It only works in the digital world, and only where actions are not likely to cause catastrophes. Otherwise, you would need a world model for the system to evaluate the consequences of its actions BEFORE executing them.Collected by WispRead in full↗
XResearch22:35Yann LeCun@ylecun@andrzejdragan @francoisfleuret It's absolutely NOT auto-regressive token prediction in the sense that multiple hypothesized sequence of tokens/actions can be evaluated without being produced on the output. That's what I mean by SEARCH. Though there are way more efficient ways to perform search than generation-selection of discrete token sequences. Some LLMs do search when they produce multiple outputs (auto-regressively) in silence and then evaluate them and select the best one. The overall process is not auto-regressive token prediction.Collected by WispRead in full↗
XResearch22:51Yann LeCun@ylecun@allaei @steeve Then I understand why you're following me 😅Collected by WispRead in full↗
XResearch22:42Yann LeCun@ylecun@cosminnegruseri @francoisfleuret It's a primitive form of search that is limited to discrete token sequences.Collected by WispRead in full↗
XResearch22:38Yann LeCun@ylecun@willessd @francoisfleuret Nothing. My critique is entirely directed at auto-regressive prediction of discrete tokens. Also at limiting inference to a search in the space discrete token sequences. Also at using generative models with discrete tokens to model real-world signals. Also at not having explicit world models that enable real planning/reasoning.Collected by WispRead in full↗