Yann LeCun

Several comments: 1. image/video SSL methods differ by whether they use reconstruction or not. Joint embedding architectures Amy or may not have a predictor (JEA vs JEPA). They also differ by their anti-collapse method and data corruption/augmentation methods. 2. DINO *is* a kind of JEPA (technically a JEA). The difference with other JEPA models is the anti-collapse criterion (based on contrative clustering+ stopgradient/EMA), and the data augmentation scheme (image transformations, which is task specific), 3. V-JEPA 2.1, like DINO, is from Meta-FAIR (corporate lab), involved significant engineering teams, large/proprietary datasets, considerable computing resources, lots of fine-tuning, and number-pushing on many benchmarks. 4. Many JEPA papers are applications and/or fine-tuned versions of DINO, V-JEPA or I-JEPA. 5. LeJEPA, LeWM, and most other papers with "JEPA" in the title are *research* papers focused on the methodology, often done by students in academic labs, involving no engineering support, very little computing resources, and small/public training sets. They are generally tested on toy problems, simulated environments, or small/public benchmarks. 6. You simply cannot compare 5 with 2/3/4. 5 gives you ideas and new methods, not models. 2/3 give you well-engineered models, with lots of empirically validated tricks. Only rarely will 2/3 give you a new, well-principled method with room to spare. 5 give you empirical results on new applications. 7. What is published in 5 are methods that will eventually become like 2 and 3, whenever someone with resources picks it up and scales it. DO NOT CONFUSE UPSTREAM FOUNDATIONAL RESEARCH ON METHODS AND DOWNSTREAM SCALING -> ENGINEERING -> NUMBER PUSHING -> MODELS -> PRODUCTS. The former makes the rest possible, but it takes time.