Overview
The event opens by challenging the recurring claim that robotics will be “solved next year”: impressive demos have multiplied, yet general-purpose household robots remain unavailable and real deployments still collide with physics, sensing, data, and hardware degradation. Marcel presents Multiscale Embodied Memory, which gives robot policies dense short-term visual context for dexterity and compressed textual memory for long-horizon progress tracking and adaptation. Milan introduces a self-supervised framework that proposes, scores, and refines embodiment-specific reasoning, arguing that useful reasoning must be concise, non-trivial, and predictive of action rather than exhaustively describing every scene. Tyler demonstrates a complementary route to dexterity: train a goal-conditioned hand-and-arm policy entirely in massively parallel simulation, then prompt the frozen policy with object trajectories extracted from human videos. Niko shifts from research to company-building, recommending that robotics startups own narrow customer problems end to end, begin with teleoperation and off-the-shelf hardware, and continuously couple deployment, evaluation, and data collection. General Instinct closes with infrastructure for accelerating world-action models by distilling diffusion, separating video and action transformers, and avoiding unnecessary frame decoding. Together, the talks suggest that progress will come less from one universal breakthrough than from disciplined combinations of memory, selective reasoning, scalable simulation, operational feedback, specialized data systems, and efficient inference.
Sections
Progression of the Event
The talks move from unresolved foundations to research systems, deployment strategy, and inference infrastructure.
- The opening speaker reviews repeated waves of optimism following AlphaGo, MuJoCo, ALOHA-style data collection, diffusion policies, and vision-language-action models, then argues that 2026 has so far been the year of robotics demos rather than solved general autonomy.
- Marcel presents Multiscale Embodied Memory for long-horizon manipulation, combining short-term visual context with a recurrent long-term textual scratchpad.
- Milan presents R&B Encore, a self-supervised cycle for proposing, validating, and retraining on embodiment-specific reasoning traces.
- Tyler presents Sim-to-Real, a single simulation-trained policy for zero-shot dexterous tool use, followed by Play-to-Perfect for contact-rich assembly.
- Niko presents an application-company playbook centered on paid customer problems, teleoperation-first deployment, early evaluation, and continuous data iteration.
- General Instinct presents infrastructure optimizations for world-action models, including latent representation transfer and distilled flow matching.
Cross-Cutting Insights
Synthesis across the research and company-building perspectives.
- The central challenge is no longer merely generating plausible actions; it is maintaining the right state across time, physical contact, changing hardware, and operational context.
- Compression appears repeatedly as an enabling principle: compress visual history into memory, reasoning into selective traces, tasks into goal trajectories, physical experience into reusable priors, and future dynamics into latent representations.
- Several speakers reject exhaustive generality in favor of targeted sufficiency: reason only when useful, simulate only what can be modeled, automate a business that already works through teleoperation, and decode only the representations required for action.
- Evaluation is a shared bottleneck across research and deployment because real-world trials are expensive, noisy, and difficult to make statistically significant. Useful progress therefore depends on task-specific metrics, simulation proxies, failure classification, and human judgment.
- The strongest near-term systems may be hybrids that combine foundation-model priors with specialized memory, simulation, perception, teleoperation, and operational feedback rather than relying on one end-to-end model.
Technical Details
Specific architectures, data flows, performance figures, and implementation choices described by the speakers.
- Multiscale Embodied Memory uses a ViT-derived encoder with temporal attention for dense short-term history, then discards historical tokens while retaining their information in the current-image representation.
- Its high-level policy recurrently predicts a textual memory scratchpad and feeds that text back as tokens at the next decision point; the demonstrated version was trained with supervised fine-tuning and annotated memory content.
- R&B Encore treats reasoning as an unobserved latent variable and scores candidate traces using concision, non-triviality, and action predictiveness before resampling a synthetic reasoning dataset.
- The reasoning framework was evaluated across manipulation, legged navigation, and autonomous driving, with model sizes ranging from one billion to 30 billion parameters.
- Sim-to-Real runs a recurrent LSTM policy at 60 Hz and jointly controls a 22-degree-of-freedom hand and a seven-degree-of-freedom arm.
- The policy input contains proprioception, current object pose, a bounding box for the graspable region, and the current goal pose; its output is a set of joint-position targets.
- Training uses procedurally generated cylinders and cuboids, random pose goals, GPU-parallel reinforcement learning, system identification, domain randomization, and random external forces that expose the policy to dropped-object recovery.
- Human videos are processed with FoundationPose and SAM to extract desired object-pose trajectories. They neither provide robot actions nor fine-tune the frozen control policy.
- World-action-model optimization separates the video and action diffusion transformers, uses cross-attention to transfer latent world representations, avoids decoding unnecessary future frames, and distills autoregressive flow matching from roughly 50 to 100 steps down to one or two.
- The optimized system reportedly produces a 16-action chunk in approximately 500 milliseconds on a Jetson Thor.
Predictions and Forward Directions
Explicit forecasts and research directions proposed during the event.
- General-purpose household robotics may continue to lag behind impressive demonstrations unless physical modeling, tactile sensing, action representation, and embodiment drift are addressed.
- Memory-enabled vision-language-action policies could become capable of long-horizon progress tracking and in-context adaptation, potentially using reinforcement learning to discover what information should be stored.
- Embodied reasoning will likely move away from fixed, always-on textual templates toward reasoning that is selected and budgeted according to the embodiment, task, and moment.
- Simulation-trained dexterous priors will likely be fine-tuned with real-world experience for objects and interactions that cannot yet be modeled accurately, including deformables, liquids, zippers, and articulated tools.
- Robotics application companies may become analogous to the previous generation of SaaS companies by transforming narrow categories of physical work while building defensibility through operations, deployment, and proprietary task data.
- World-action models will need to move onto edge hardware and operate in real time, creating demand for specialized inference infrastructure that preserves world representations without full video generation.