要点
- Kaedim/projectsim (HF; page date 2026-10-30) starts from a Robot.com deployment talk and inventories unpaid work between generalist policies and shippable systems: outcome checks, memory, skill constraints, safety.
- VoLo: full system 18/42 vs standalone policy 6/42; a simulated failure shows the monitor declaring an orange in the bowl while it remains on the table.
- Transfer signals: Figure Helix 2.5 in unseen homes—Index pretraining lifts full-task completion 9%→56%; GEN-1.5 averages 59% with one 3–12s demo and 83% with five minutes + ten fine-tune steps.
- Experience scale: NVIDIA reports GR00T N1 generated 780k simulated trajectories in 11 hours via DexMimicGen (~6,500 trajectory-hours); SIMPLER argues calibrated sim rollouts tracked real rankings better than action-prediction error—if difficulty is preserved.
At Long Horizon in San Francisco, a Robot.com talk made the gap blunt: even with a generalist starting point, fine-tuning, testing, integration, and systems engineering remain heavily manual. Kaedim’s Luca Cilio on Hugging Face (page date 30 Oct 2026) unpacks that gap: outcome monitors can be wrong, skills must serve the next step, and simulation must preserve real difficulty.
Thesis: plug-and-play will not arrive merely because policies generalize; what is missing is transferable experience plus an honest evaluation/adaptation loop.
[1]What happened
The piece braids recent systems and generalist results: VoLo adds planning/monitoring and grasp/place tools around a policy; POT-VLA keeps persistent 3D object records; BATON records constraints on skill transitions; Tool-Aligned VLA post-trains executors for how planners call them. Another thread is transfer: Helix 2.5, π0.7 laundry across embodiments, GEN-1.5 physical prompts, Skild S1 egocentric video. Further out, Recursive Harness Distillation and Gemini 4 Argon demos push some diagnosis/trial-and-error onto stronger agents. Experience supply lands on synthetic expansion and evaluations like SIMPLER.
[1]How hard is the evidence
Hard numbers mostly come from cited work, not a new author benchmark: VoLo hardware 18/42 vs 6/42; Helix 2.5 full-task completion 9%→56%; GEN-1.5 ten-task means 59% (one demo) and 83% (five minutes + ten steps); GR00T N1’s reported 780,000 simulated trajectories in 11 hours. SIMPLER’s claim is that on tested Google Robot tasks, simulated rollouts tracked real policy rankings better than action-prediction error on demos—while uncalibrated controllers can invent false failures.
This is secondary synthesis: the value is framing and juxtaposition, not a new leaderboard.
[1]Steelman and gaps
Steelmans: separating “stronger policy” from “system can finish the job” is an honest field checklist; insisting simulation preserve difficulty targets fake progress directly.
Gaps: community article, not a lab primary; some demos (e.g., Argon) are flagged as curated with heavy tooling; GEN-1.5/Helix figures need original papers for variance and task duration; “plug-and-play” still lacks a shared operational definition.
[1]Who pays, and what checklist changes
For deployers: acceptance should include monitor error rates, cross-step constraints, and sim-to-real calibration logs—not only generalist scores. For labs: synthetic throughput without a difficulty-preservation test is a liability. For buyers: ask whether the deliverable is a checkpoint or a system with an evaluation loop.
[1]What to watch in six months
Falsifiable next steps: with monitors trained independently of the policy, can end-to-end success stably clear VoLo’s “still failing half” zone; do minute-scale GEN-1.5-style adaptations replicate in third-party kitchens/warehouses; if sim difficulty calibration is removed, does SIMPLER-style rank correlation collapse. The claim is not that generalists are useless—it is that they lower the starting cost while plug-and-play still hinges on whether the experience loop stays honest.
[1]