Embodied AI 101

VPP2: Does Better Video Prediction Improve Robot Manipulation?

Shaoqing Tan — Mon, 12 Oct 2026 05:29:31 -0700

Can better video prediction make a robot manipulation policy more general? This episode examines what VPP2's experiments establish, how its training pipeline works, and where the causal story remains incomplete. We cover three questions: why complete manipulation events may provide better supervision than fixed video fragments; what the real-world ALOHA and simulation results say about generalization; and what the public implementation actually enables a robotics team to reproduce. The method combines manipulation-focused video pretraining, fixed-horizon adaptation, distillation, and a transformer action expert. Evaluations span real ALOHA manipulation and benchmark-specific policies for LIBERO and RoboDojo simulation. The central limitation is that the video ablation does not become a matched downstream-policy ablation, so improved prediction and improved action are not yet cleanly connected causally. For implementation context, the paper's initial video stage reports 30,000 optimization steps, batch size 1,024, and 49-frame training at 416 by 240 resolution. We also distinguish published results from evolving release configurations, and explain why strong average performance can conceal weak open-instruction transfer. Is your robot failing because it predicts the wrong future—or because its action head never uses the right one?

PointWAM: Does 3D World Modeling Improve Dexterous Manipulation?

Shaoqing Tan — Mon, 12 Oct 2026 05:14:41 -0700

Can forecasting scene and hand motion in 3D make dexterous robot policies more reliable? This episode explains what PointWAM's geometric representation achieves, which experiments support its gains, and where the evidence remains narrow. We examine three findings: the 11.7-percentage-point advantage over GR00T N1.6 on the paper's ten-task DexJoCo protocol; the controlled improvement from adding scene-trajectory supervision; and the surprisingly large advantage of anatomical hand keypoints over sampled hand surfaces. The discussion follows the full pipeline from EgoDex and VITRA human demonstrations to point-trajectory forecasting and learned robot-action retargeting. Evaluation spans simulated Franka–Allegro manipulation, a difficult parallel-jaw-gripper benchmark, and real OpenArm experiments with Inspire hands and stereo vision. The central limitation is scope: real-robot testing covers two tasks with small trial counts, while precision-task completion remains low outside the main benchmark. We also unpack a confounded 2D-versus-3D ablation, checkpoint selection, preprocessing costs, and the distinction between predictive supervision and planning with imagined futures. Is PointWAM's most important contribution a better world model—or a better interface between human motion and robot control?

Mulligan: Can Failure-Focused Data Improve Robot Imitation Learning?

Shaoqing Tan — Sun, 11 Oct 2026 05:11:49 -0700

Can a robot learn more from the same collection budget by revisiting the situations it gets wrong? This episode explains what Mulligan demonstrates about focused robot data collection—and why better learning per episode is not automatically better learning per operator hour. We examine three questions: how failed-state retries and counterfactual demonstrations differ from ordinary DAgger corrections; what value-based action selection adds beyond the imitation policy; and how to interpret the blinded, paired real-robot results. The experiments cover marker insertion, nut placement, and cable routing on a Franka Panda, alongside simulated square-nut insertion. We also unpack a consequential simulation caveat: some collection rounds use feedback from the fixed evaluation grid to guide training-state selection. The practical discussion focuses on failure logging, coverage-only controls, independent acceptance tests, and measuring human time rather than treating every supervised episode as equally expensive. The release offers substantial reproducibility support, but a fresh human-in-the-loop campaign remains a new experiment. Before investing in another policy architecture, should you change which failures your operator teaches next?

SOLE-R1: Can Video-Language Rewards Train Manipulation Policies?

Shaoqing Tan — Sun, 11 Oct 2026 05:11:44 -0700

Can a robot learn manipulation from random actions using only rewards generated from video? This episode explains what SOLE-R1 demonstrates, how its reward model is trained, and where its generalization claims need qualification. We examine three questions: why realistic failure coverage matters beyond reversed demonstrations; what the reasoning and reinforcement-learning ablations actually establish; and whether benchmark success translates into a practical on-robot learning system. The discussion connects video-language reward modeling with off-policy reinforcement learning across simulated manipulation benchmarks and physical Franka experiments. We also compare SOLE-R1's operating regime with RoboReward, Robometer, and ReWiND, rather than treating their results as interchangeable. Particular attention goes to absolute progress rewards, policies that exploit perceptual mistakes, and the difference between an available inference checkpoint and a reproducible training pipeline. The central limitation is scope: a controlled physical experiment is not evidence of reliable, unattended skill acquisition. For practitioners, we outline a staged evaluation strategy using difficult negative examples, independent completion checks, and closed-loop attempts to expose reward errors. What should you demand from a visual evaluator before giving an optimizer permission to trust it?

Odyssey-3: Does a Physics-IQ World Model Transfer to Real Robots?

Shaoqing Tan — Sat, 10 Oct 2026 05:13:12 -0700

Can a leading physics benchmark score translate into robots that need less task-specific training data? This episode separates Odyssey-3's documented video-generation results from its early evidence for embodied control. We examine three questions: how prompt enhancement and best-of-eight selection contribute to the 66.1 Physics-IQ Verified score; what representation transfer actually means for a downstream policy; and which experiments would establish reliable recovery and sim-to-real generalization. The headline score belongs to a selected-sample Pro configuration, rather than establishing the performance of every interactive deployment. We explain the autoregressive diffusion approach, contrast physical prediction with interactive-world consistency, and examine reported robot-arm manipulation, Flexion humanoid control, and real-road driving. We also correct an important scope distinction: the disclosed drone experiments are simulated, not demonstrated real-drone transfer. The central limitation is the absence of a complete, independently reproducible robotics evaluation package. For practitioners, the episode develops a concrete testing strategy around matched representations, controlled disturbances, policy latency, and trustworthy generated experience. What evidence would make you trust a learned world model with your robot's next action?

Long-WAM: Does Longer Visual Context Improve Real-Time Robot Control?

Shaoqing Tan — Sat, 10 Oct 2026 05:12:06 -0700

Can longer video histories make robot manipulation more reliable without making control too slow? This episode examines Long-WAM to clarify when temporal context helps, what autoregressive pretraining contributes, and how to interpret the deployment evidence. We focus on three questions: whether the context-scaling study isolates the effect of pretraining, whether predicting future visual latents improves actions enough to justify its cost, and what the reported inference latency does—and does not—measure. We connect the work to Fast-WAM's investigation of test-time imagination, LingBot-VA 2.0's causal pretraining, and DOMINO's dynamic manipulation benchmark. The discussion spans simulation experiments and physical manipulation on Unitree G1 and YAM, while distinguishing local execution from higher-level task planning. The central limitation is that separately trained context variants do not establish adaptive memory in one deployed controller. We also examine public checkpoints, training recipes, incomplete release validation, numerical reporting inconsistencies, and the experiments a robotics team should run before adopting the method. If you are weighing visual memory against inference cost, what evidence would convince you that the additional frames are earning their place in the control loop?

WR1.0: Does Whole-Body Humanoid Control Prove Generalization?

Shaoqing Tan — Fri, 09 Oct 2026 05:19:06 -0700

Does WR1.0's clothes-folding demonstration establish a general-purpose whole-body humanoid foundation model? This episode separates the announced capability from the research evidence needed to judge autonomous manipulation, generalization, and deployment readiness. We examine three questions: what Westlake's related GAE motion-tracking research actually measures; why coordinated body movement is not equivalent to successful object manipulation; and which experiments would make a long-horizon generalization claim convincing. The discussion spans simulation evaluation and real-hardware humanoid control, including the Unitree G1 and Westlake O1 platforms documented in the related GAE project. We also use SONIC, HumanPlus, HiF-VLA, and public implementation releases to frame fair comparisons and reproducibility expectations. For practitioners, the emphasis is on controller interfaces, latency compensation, garment variation, recovery, and human-intervention accounting—not the visual polish of a demonstration. The central limitation is explicit: the supplied X video could not be independently inspected here, and a complete WR1.0 technical report and evaluation package were not verified. What would convince you that a humanoid can recover from a slipped grasp, rather than merely repeat a successful fold?

WorldGuide: Better Video Planning, Not Proven Robot Control

Shaoqing Tan — Fri, 09 Oct 2026 05:15:31 -0700

Can a goal-directed video world model genuinely help a robot plan, or does it mainly produce better procedural videos? This episode clarifies what WorldGuide demonstrates, how its planner–executor architecture works, and which additional experiments would be needed for physical deployment. We examine three central questions: why visual feedback produces stronger evidence than the narrow headline baseline win; why a valid text plan is not equivalent to successful execution; and what released code and checkpoints do—and do not—make reproducible. WorldGuide combines a Qwen-based planner with a HunyuanVideo executor for generated procedures such as folding, construction, and cooking. Its benchmarks evaluate visual task execution, not a physical robot platform. We place the work alongside Video Language Planning, ORCA, VideoWorld 2, and NovaPlan to distinguish generated-state feedback from feedback grounded in real observations. The discussion also covers known-task versus unseen-task generalization, noisy completion supervision, evaluation uncertainty, and configuration details that matter when reproducing results. For practitioners, the focus is a concrete adaptation path: retain the action–observation supervision structure, test progress recognition with existing robot skills, and measure whether imagined futures actually improve action selection. What happens when reality contradicts the generated plan?

R2SGen: Can Constraint Solving Improve Real-to-Sim Driving Scenarios?

Shaoqing Tan — Thu, 08 Oct 2026 05:19:11 -0700

Can constraint solving turn a recorded traffic scene into useful simulation data without inheriting an LLM's geometric mistakes? This episode clarifies what R2SGen's guarantees mean, where its search improvements matter, and what remains unproven about sim-to-real transfer. We examine three questions: why merging equivalent search histories can matter more than choosing a different search order; whether lower program cost captures useful scenario abstraction; and how uncertainty in upstream perception changes the meaning of a verified output. The discussion focuses on static autonomous-driving scenario programs built from structured real-world observations, rather than a physical robot deployment. We connect the work to Scenic, ScenicNL, ChatScene, earlier scenario-querying research, and TrafficGen to distinguish reconstruction, probabilistic authoring, and learned traffic generation. The central limitation is the gap between consistency with supplied geometry and evidence of useful downstream transfer. We also outline concrete reproduction checks and an evaluation plan covering simulator validity, sampling yield, targeted failure preservation, and held-out performance. If your team is building synthetic-data or regression-testing infrastructure, what would you need to measure before trusting a mathematically consistent scene generator?

Humanoid Scooter Control: Does Sim-to-Real Reinforcement Learning Work?

Shaoqing Tan — Thu, 08 Oct 2026 05:17:51 -0700

Can reinforcement learning trained in simulation let a humanoid control a real scooter? This episode examines what the reported transfer demonstrates, how the controller is structured, and why balance control should not be confused with autonomous transportation. We cover three technical questions: why PPO outputs a steering target rather than humanoid joint commands; how domain randomization and mirrored simulation experience support transfer; and what separates successful balance from sustained precision through difficult maneuvers. The research builds on a THORMANG3 humanoid operating a commercial Gogoro scooter, combining simulated training with physical evaluation. We also inspect a consequential reward-implementation detail and discuss which artifacts another laboratory would need for a credible reproduction. The central limitation is scope: human assistance remains part of the operating procedure, and the harder tracking tasks are substantially less reliable. This episode also makes its research boundary explicit: the publisher's abstract and released code were accessible, but the complete article's trial-level statistics and baseline protocols could not be verified. What should a roboticist measure before deciding that a spectacular sim-to-real demonstration is a control primitive worth deploying?

ENPIRE: Can Coding Agents Improve Real-World Robot Manipulation?

Shaoqing Tan — Wed, 07 Oct 2026 05:15:12 -0700

Can coding agents improve real-world robot manipulation without a researcher rewriting every experiment? This episode explains what ENPIRE automates, what remains engineered, and how to interpret its headline results. We examine three questions: what 99% pass@8 means for recovery and first-attempt reliability; whether parallel robot workers improve efficiency or primarily reduce wall-clock time; and how much of the demonstrated system another lab can reproduce. ENPIRE combines code-based control and policy learning with automatic resets, outcome verification, and physical feedback, using bimanual YAM robots for manipulation alongside RoboCasa simulation experiments. Its public release provides useful infrastructure but leaves some task-specific robot-side components for users to supply. We connect the work to CaP-X, human-in-the-loop reinforcement learning, and specialist-to-generalist learning, then propose practical tests for verifier accuracy, held-out reliability, and operator effort. The central limitation is scope: autonomous iteration inside a prepared workcell does not establish general-purpose autonomy or eliminate human oversight. When should your next robotics investment be a better policy — and when should it be a better experiment?

GAS-Robo: Can Grid Actions Improve LLM-Driven Manipulation?

Shaoqing Tan — Wed, 07 Oct 2026 05:12:43 -0700

Can a grid-action interface make language models better robotic manipulation planners, rather than just better selectors of predefined skills? This episode examines GAS-Robo's proposed architecture, its relationship to earlier work, and the evidence needed to judge its performance. We cover three central questions: whether task-aware scene filtering preserves execution-critical geometry; whether discrete action generation offers an advantage over program, value-map, and constraint-based interfaces; and what a fair RLBench evaluation should control. The discussion connects the proposal to R2C, Code as Policies, VoxPoser, ReKep, and RoboPro, while separating related-work results from claims about GAS-Robo itself. The authors describe simulation evaluation and physical testing on a Franka manipulator, but the full manuscript and experimental tables could not be retrieved for this review. That is the episode's most important limitation: it offers a source-limited technical assessment, not a verified numerical verdict. For practitioners, the focus is concrete: representation ablations, perception assumptions, trial counts, latency, reproducibility, and guarded deployment. When does a spatial grid genuinely simplify robot planning—and when does it merely move the hardest work into another component?

Four Keypoints, One Careful Pour: Connecting Language to Precise Robot Motion

Shaoqing Tan — Tue, 06 Oct 2026 05:14:43 -0700

Bridging VLM and KMP: Enabling Fine-Grained Robotic Manipulation via Semantic Keypoints Representation

PEAfowl: Helping Two Arms Agree on What They See

Shaoqing Tan — Tue, 06 Oct 2026 05:12:56 -0700

PEAfowl: Perception-Enhanced Multi-View Vision-Language-Action for Bimanual Manipulation

MotorMind: Giving a Frozen VLM the Controls

Shaoqing Tan — Mon, 05 Oct 2026 05:19:03 -0700

MotorMind leverages a frozen VLM with mid-level prospective actions and asynchronous feedback to control robots zero-shot on manipulation tasks, requiring no task-specific training. This advances robot foundation models by demonstrating strong generalization across manipulation benchmarks without fine-tuning.

MAPF-World: Why a Little Foresight Can Beat a Bigger Policy

Shaoqing Tan — Mon, 05 Oct 2026 05:14:24 -0700

MAPF-World: Action World Model for Multi-Agent Path Finding

Digit Leaves the Lab: The Real Lesson of 500 Demonstrations

Shaoqing Tan — Sun, 04 Oct 2026 05:14:01 -0700

An end-to-end policy trained on ~500 teleoperation demonstrations combined with a sim-based whole-body controller achieves 80–90% success on unseen, chaotic real-world environments without any zero-shot fine-tuning. Demonstrates strong generalization of learned manipulation policies on a full humanoid platform.

VLS: Steering Frozen Robot Policies Without Retraining

Shaoqing Tan — Sun, 04 Oct 2026 05:13:54 -0700

VLS steers frozen diffusion and flow-matching robot policies using vision-language models to handle out-of-distribution changes such as novel obstacles and lighting conditions, requiring no additional training data. This advances the adaptability of robot foundation models at inference time without retraining.

CAST: Making Language Matter to Robot Policies

Shaoqing Tan — Sat, 03 Oct 2026 05:17:26 -0700

CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models In this episode of Embodied AI 101, we explore "CAST: Making Language Matter to Robot Policies". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When the Command Disappears: What Counts as Robot Intelligence?

Shaoqing Tan — Sat, 03 Oct 2026 05:11:20 -0700

Robot Independent Intelligence: When Language Vanishes in Vision-Language-Action Models In this episode of Embodied AI 101, we explore "When the Command Disappears: What Counts as Robot Intelligence?". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Fast Plans Are Not Enough: Does the Robot Actually Use Its Plan?

Shaoqing Tan — Fri, 02 Oct 2026 05:13:14 -0700

Introduces a hierarchical VLA architecture that better bridges high-level language planning and low-level robot execution, reducing the gap between what a model plans and what it physically does. This advances robot foundation models by improving real-world action fidelity from vision-language reasoning. In this episode of Embodied AI 101, we explore "Fast Plans Are Not Enough: Does the Robot Actually Use Its Plan?". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Atlas's New Hands: Built to Learn, Not Just to Look Human

Shaoqing Tan — Fri, 02 Oct 2026 05:09:23 -0700

Atlas is upgraded with 13 DoF dexterous hands featuring tactile sensing, direct drive actuation, and sim-to-real RL training, designed for real tool use and in-hand manipulation at scale. Represents a significant hardware-software milestone for humanoid dexterous manipulation in production. In this episode of Embodied AI 101, we explore "Atlas's New Hands: Built to Learn, Not Just to Look Human". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Astra Gets a Body: What Zero-Shot Humanoid Control Actually Transfers

Shaoqing Tan — Thu, 01 Oct 2026 05:23:28 -0700

Astra demonstrates zero-shot open-ended pick-and-place on a physical humanoid robot, bridging simulation tooling directly to real-world execution without task-specific fine-tuning. This represents a meaningful step in sim-to-real generalization for humanoid manipulation. In this episode of Embodied AI 101, we explore "Astra Gets a Body: What Zero-Shot Humanoid Control Actually Transfers". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Physical Coding: Robots That Check Their Work—and Keep the Fixes

Shaoqing Tan — Wed, 30 Sep 2026 05:37:59 -0700

A new paper introduces self-evolving coding agents for robotics that represent task state and execution as code, enabling robots to verify, recover from failures, and evolve policies through real-world experience via the HexaAnything harness. This advances sim-to-real transfer by grounding robot learning in executable, verifiable code representations. In this episode of Embodied AI 101, we explore "Physical Coding: Robots That Check Their Work—and Keep the Fixes". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

SpatialClaw: Give the VLM a Geometry Notebook

Shaoqing Tan — Wed, 30 Sep 2026 05:10:56 -0700

SpatialClaw is an NVIDIA spatial reasoning agent that uses Python-based code generation to perform 3D spatial reasoning, achieving +13.6 average improvement across 20 benchmarks with open vision-language models. The approach advances how embodied agents reason about 3D structure and spatial relationships without task-specific fine-tuning. In this episode of Embodied AI 101, we explore "SpatialClaw: Give the VLM a Geometry Notebook". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Real-Time EXPO-FT: Slow VLA, Fast Corrections, Better Control

Shaoqing Tan — Tue, 29 Sep 2026 05:16:20 -0700

Async RL fine-tuning via EXPO-FT enables π0.5 to handle highly dynamic manipulation tasks such as ball balancing and striking in real time. The approach demonstrates that post-training with RL can unlock dynamic capabilities not achievable through imitation learning alone. In this episode of Embodied AI 101, we explore "Real-Time EXPO-FT: Slow VLA, Fast Corrections, Better Control". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Aether: From Plausible Futures to Feasible Robot Plans

Shaoqing Tan — Tue, 29 Sep 2026 05:12:38 -0700

Aether's unified world model jointly learns environment dynamics and long-term physical state evolution under physical constraints, enabling robots to plan feasible trajectories over extended horizons. The demo highlights how constraint-aware world modeling can prevent physically implausible plans that plague standard generative approaches. In this episode of Embodied AI 101, we explore "Aether: From Plausible Futures to Feasible Robot Plans". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

ARLI: Fixing the Missing State in Asynchronous Robot RL

Shaoqing Tan — Mon, 28 Sep 2026 05:26:52 -0700

Proposes a framework that restores the Markov property for RL fine-tuning of VLAs despite asynchronous inference delays, enabling effective RL training on large vision-language-action models. This directly unblocks post-training RL for frontier robot policies operating under real-world latency constraints. In this episode of Embodied AI 101, we explore "ARLI: Fixing the Missing State in Asynchronous Robot RL". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Flex-π: Teach Geometry, Choose How Much to Imagine

Shaoqing Tan — Mon, 28 Sep 2026 05:15:33 -0700

A 6B-parameter world-action model that jointly predicts 3D pointmaps and DINO features alongside RGB frames, improving demonstration efficiency, generalization, and long-horizon task performance in robotics. Integrating structured 3D world predictions into the action loop represents a meaningful advance in robot foundation model design. In this episode of Embodied AI 101, we explore "Flex-π: Teach Geometry, Choose How Much to Imagine". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

SkipVLA: Let the Planner Move, Let the VLA Manipulate

Shaoqing Tan — Sun, 27 Sep 2026 05:12:01 -0700

SkipVLA proposes a hybrid architecture combining a classical planner with a Vision-Language-Action model for manipulation tasks, achieving 2.5x faster execution and 59.6% lower latency on a physical robot arm. The approach demonstrates that structured planning can significantly reduce the computational burden of end-to-end VLA inference. In this episode of Embodied AI 101, we explore "SkipVLA: Let the Planner Move, Let the VLA Manipulate". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

ForeDrive: A World Model Is Only Useful If It Changes the Plan

Shaoqing Tan — Sun, 27 Sep 2026 05:11:12 -0700

ForeDrive introduces a planning-relevant latent world model that guides autonomous driving through foresight, enabling the agent to anticipate future states before acting. The approach advances world-model-based driving by tightly coupling latent prediction with planning objectives. In this episode of Embodied AI 101, we explore "ForeDrive: A World Model Is Only Useful If It Changes the Plan". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

DO AS I DO: Making Human Video Executable

Shaoqing Tan — Sat, 26 Sep 2026 05:12:32 -0700

A new algorithm reconstructs and retargets monocular RGB (or synthetically generated) videos to high-DoF robot hands, enabling scalable dexterous manipulation data collection that outperforms current state-of-the-art retargeting methods. This matters because it removes the need for expensive motion capture or teleoperation hardware to generate training data for dexterous robots. In this episode of Embodied AI 101, we explore "DO AS I DO: Making Human Video Executable". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

GAM: Ground the Target Before You Learn the Action

Shaoqing Tan — Sat, 26 Sep 2026 05:09:24 -0700

GAM is a robot foundation model that conditions action chunk prediction on 3D grounded inputs — including language, 3D point clouds, and bounding boxes — alongside robot history, achieving strong results on LIBERO benchmarks and real-robot tasks. The 3D grounding mechanism advances how embodied agents spatially understand and act in their environment. In this episode of Embodied AI 101, we explore "GAM: Ground the Target Before You Learn the Action". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Tune Slowly, Control Quickly: Learning a Better Robot Navigation Stack

Shaoqing Tan — Fri, 25 Sep 2026 05:14:02 -0700

Autonomous navigation in complex, unstructured environments poses a significant challenge, with traditional planners lacking adaptability and end-to-end learning methods hindered by data dependency or training instability. This paper proposes a hierarchical learning-based hybrid method with sim-to-real transfer. In this episode of Embodied AI 101, we explore "Tune Slowly, Control Quickly: Learning a Better Robot Navigation Stack". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

THAW-VLA: World-Model Features Without World-Model Latency

Shaoqing Tan — Thu, 24 Sep 2026 05:13:26 -0700

This work distills frozen world-model representations into compact VLAs via a single feature-alignment loss at training time, with the teacher cached once and absent at inference; a 0.8B student achieves 97.9% on LIBERO and lifts RoboCasa-GR1 humanoid performance from 48.2% to 50.5% at zero extra runtime cost (32 ms / 1.86 GB on RTX 5090). The approach generalizes across scales and architectures on both single-arm and bimanual real hardware. In this episode of Embodied AI 101, we explore "THAW-VLA: World-Model Features Without World-Model Latency". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Joga: Why a Soccer Humanoid Needs to Control Its Gaze

Shaoqing Tan — Thu, 24 Sep 2026 05:11:31 -0700

Joga demonstrates agile humanoid soccer on a Unitree G1 using an actuated neck and residual models to close the sim-to-real gap for both perception and actuation, enabling dribbling, tight turns, and receive→dribble→shoot sequences with fully onboard perception. This is a significant step toward general-purpose humanoid locomotion and active spatial perception in unstructured environments. In this episode of Embodied AI 101, we explore "Joga: Why a Soccer Humanoid Needs to Control Its Gaze". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

AthenaZero: Why Dynamic Manipulation Starts With Lower Inertia

Shaoqing Tan — Wed, 23 Sep 2026 05:16:49 -0700

AthenaZero is a bimanual manipulator designed to minimize inertia without compromising control authority. By using quasidirect drive actuation and transmission remotization techniques, the system achieves an effective end-point mass comparable to state-of-the-art systems while enabling dynamic, contact-rich manipulation. A significant hardware advance for dexterous embodied manipulation research. In this episode of Embodied AI 101, we explore "AthenaZero: Why Dynamic Manipulation Starts With Lower Inertia". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

FROA-Drive and the Hard Part of Targeted Policy Repair

Shaoqing Tan — Wed, 23 Sep 2026 05:12:42 -0700

Vision–Language–Action (VLA) models have shown strong potential for end-to-end autonomous driving, yet their post-training commonly relies on expensive simulator interaction or global policy updates. FROA-Drive introduces a failure-routed offline adaptation method that enables lightweight, targeted policy improvement for already-competent pretrained VLA driving policies without costly retraining or simulator access. In this episode of Embodied AI 101, we explore "FROA-Drive and the Hard Part of Targeted Policy Repair". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Teach the Robot Where to Act—Then Make It Fast

Shaoqing Tan — Tue, 22 Sep 2026 05:17:45 -0700

We present a framework for assistive robot manipulation that addresses two fundamental challenges: efficient adaptation of large-scale models for scene affordance understanding and effective learning of robot actions by grounding... In this episode of Embodied AI 101, we explore "Teach the Robot Where to Act—Then Make It Fast". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

HEAR: The Sound Your Robot Missed Between Actions

Shaoqing Tan — Tue, 22 Sep 2026 05:17:18 -0700

Humans and animals use sound as a crucial cue for interacting with the physical world, as acoustic events can reveal contact, completion, hidden contents, or process state. Embodied agents should leverage sound for manipulation tasks. This paper presents the HEAR framework for sound-centric robot manipulation. In this episode of Embodied AI 101, we explore "HEAR: The Sound Your Robot Missed Between Actions". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

VS-Splat: Learning Where 3D Detail Belongs

Shaoqing Tan — Sun, 20 Sep 2026 05:14:37 -0700

A new feed-forward Gaussian Splatting method that selectively processes voxels for efficient 3D scene reconstruction, targeting real-time perception stacks for embodied agents. Advances the efficiency frontier of 3D scene representation without per-scene optimization. In this episode of Embodied AI 101, we explore "VS-Splat: Learning Where 3D Detail Belongs". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Spatial Search Comes to Quest — Beyond Pretty Gaussian Splats

Shaoqing Tan — Sun, 20 Sep 2026 05:07:22 -0700

Adapts a training-free Spatial Search prototype to Meta Quest's spatial data pipeline, enabling natural-language search over Gaussian Splatting and mesh scenes in real time on-device. Demonstrates practical deployment of 3D spatial intelligence for consumer AR/VR hardware. In this episode of Embodied AI 101, we explore "Spatial Search Comes to Quest — Beyond Pretty Gaussian Splats". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Occamy-1.0: Teaching a 35B Agent to Finish the Job

Shaoqing Tan — Sat, 19 Sep 2026 05:10:56 -0700

Introduces a 35B cost-efficient model positioned at the Pareto frontier for collaborative long-horizon agent tasks, achieving performance competitive with significantly larger systems. Advances the capability of embodied and agentic AI to handle extended multi-step cooperative workflows. In this episode of Embodied AI 101, we explore "Occamy-1.0: Teaching a 35B Agent to Finish the Job". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Helix 2.5: The House Changed. The Policy Didn't.

Shaoqing Tan — Fri, 18 Sep 2026 05:07:35 -0700

Figure's Helix 2.5 humanoid robot enters 30 unseen Bay Area rental homes with no additional training and performs useful whole-body tasks for 4+ hours, demonstrating strong zero-shot generalization across diverse real-world environments. This is a notable advance in robot foundation model generalization for home deployment. In this episode of Embodied AI 101, we explore "Helix 2.5: The House Changed. The Policy Didn't.". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Hard Part Was the Stack: PI's Robot Goes to Work

Shaoqing Tan — Wed, 16 Sep 2026 05:12:55 -0700

Physical Intelligence's mobile π robot runs fully autonomous for hours in a real production environment stacking boxes at Dandelion Chocolate, exposing generalization and reliability gaps between lab demos and production utility. Represents a significant milestone in deploying robot foundation models in unstructured real-world settings. In this episode of Embodied AI 101, we explore "The Hard Part Was the Stack: PI's Robot Goes to Work". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

ZDTaichu5.0-9B: Better Spatial Reasoning, an Unfinished Edge Story

Shaoqing Tan — Wed, 16 Sep 2026 05:10:52 -0700

ZDTaichu5.0-9B is a 9B edge-deployable multimodal model built on a Qwen3.5 backbone with C-RADIOv4 vision encoder that leads open sub-10B VLMs on spatial-reasoning benchmarks including ViewSpatial and MMSI-Bench, while supporting embodied AI and tool-use agent tasks. Its strong spatial reasoning performance at edge scale makes it particularly relevant for on-device robot perception. In this episode of Embodied AI 101, we explore "ZDTaichu5.0-9B: Better Spatial Reasoning, an Unfinished Edge Story". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Light-Loco-Parkour: Learning the Skill Is Only Half the Problem

Shaoqing Tan — Tue, 15 Sep 2026 05:20:44 -0700

Light-Loco-Parkour presents a multi-skill distillation approach for learning perceptive whole-body parkour and locomotion skills deployable on real robots, enabling versatile agile movement across diverse terrains. The method advances embodied locomotion by combining perception and whole-body control into a unified distilled policy. In this episode of Embodied AI 101, we explore "Light-Loco-Parkour: Learning the Skill Is Only Half the Problem". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

OM-1: Human Skills, Many Bodies

Shaoqing Tan — Tue, 15 Sep 2026 05:18:41 -0700

OM-1 is a robot foundation model trained exclusively on human manipulation data (no teleoperation or robot demonstrations) that achieves near human-level dexterity and zero-shot generalization across tabletop arms, industrial arms, and humanoids, including multi-robot collaboration. Its ability to transfer from human video data alone to diverse robot morphologies marks a notable step toward scalable robot learning. In this episode of Embodied AI 101, we explore "OM-1: Human Skills, Many Bodies". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Show-Harness Gives VLMs a Robot Keyboard

Shaoqing Tan — Mon, 14 Sep 2026 05:21:16 -0700

Uses discrete semantic actions paired with embodiment-specific interpreters to turn any pretrained VLM into a robot controller without a dedicated policy network or additional pretraining. Demonstrates strong zero-shot generalization across diverse tasks and environments. In this episode of Embodied AI 101, we explore "Show-Harness Gives VLMs a Robot Keyboard". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

BridgeVLA++: Keep the Geometry, Give the Robot a Memory

Shaoqing Tan — Mon, 14 Sep 2026 05:14:36 -0700

Renders point clouds as three orthographic 2D views fed into a pretrained PaliGemma VLM, predicting heatmaps to recover 6-DoF actions with temporal/spatial memory modules. Achieves 95.4% success on 13 real Franka tasks with only 3 demos per task, outperforming π0, RVT-2, and 3D Diffuser Actor, with fully open weights and code. In this episode of Embodied AI 101, we explore "BridgeVLA++: Keep the Geometry, Give the Robot a Memory". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Sim-to-Real Gap Inside the Gearbox

Shaoqing Tan — Sun, 13 Sep 2026 05:11:04 -0700

Reinforcement learning (RL) has become a powerful tool for quadrupedal locomotion, and a sim-to-real approach is widely adopted to avoid hardware damage during training. However, the sim-to-real gap remains a significant challenge. In this episode of Embodied AI 101, we explore "The Sim-to-Real Gap Inside the Gearbox". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

A Stable Grasp Isn't Enough: Giving 6-DoF Grasping a Purpose

Shaoqing Tan — Sun, 13 Sep 2026 05:09:30 -0700

Robotic grasping in cluttered scenes is dominated by methods that optimise either stability or a predefined task category, leaving a gap between 'can this grasp hold the object' and 'can this grasp serve the task'. This paper bridges that gap using foundation model semantic embeddings. In this episode of Embodied AI 101, we explore "A Stable Grasp Isn't Enough: Giving 6-DoF Grasping a Purpose". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

HiDream-O1-Embodied: A World Model Must Earn Its Actions

Shaoqing Tan — Sat, 12 Sep 2026 21:49:11 -0700

HiDream-O1-Embodied is a unified native architecture that ingests text, images, and video and directly outputs physical actions, positioning itself as a step beyond passive world modeling toward active embodied interaction. The architecture aims to close the loop between perception and action within a single model framework. In this episode of Embodied AI 101, we explore "HiDream-O1-Embodied: A World Model Must Earn Its Actions". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Axis's Real2Sim2Real Bet: Make the Data Prove Itself

Shaoqing Tan — Sat, 12 Sep 2026 21:48:09 -0700

Axis introduced a Real2Sim2Real pipeline that converts real objects and scenes into high-fidelity physics-accurate simulations and validates learned policies back on hardware, paired with a performance-driven active trajectory curation engine. This data flywheel approach targets improved robot learning policy quality by systematically closing the sim-to-real gap. In this episode of Embodied AI 101, we explore "Axis's Real2Sim2Real Bet: Make the Data Prove Itself". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

A Rubik's Cube, Two Robot Hands, and One Category Error

Shaoqing Tan — Sat, 12 Sep 2026 05:08:52 -0700

GPT-6 Astra is demonstrated solving a Rubik's Cube using dexterous robot hands in a real physical setup, showcasing a significant leap in vision-language-action integration for fine-grained dexterous manipulation. This demo highlights the frontier of embodied AI combining large foundation models with real-world dexterity. In this episode of Embodied AI 101, we explore "A Rubik's Cube, Two Robot Hands, and One Category Error". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

UnifoLM: Predict the Interaction, Not the Whole World

Shaoqing Tan — Sat, 12 Sep 2026 05:08:23 -0700

Unitree's UnifoLM is a 6B-parameter world model trained on 2,500 hours of real robot data that predicts only dynamic pixels and optical flow rather than full frames, enabling efficient robot action prediction. This interaction-centric design reduces computational overhead while maintaining high fidelity for embodied control. In this episode of Embodied AI 101, we explore "UnifoLM: Predict the Interaction, Not the Whole World". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Stop Making the Video Model Remember the World

Shaoqing Tan — Fri, 11 Sep 2026 05:10:51 -0700

A new world model architecture that decouples world-state evolution from rendering: an agent writes executable rules, a persistent engine tracks entities, and a video model renders the output. This separation enables more controllable and interpretable world simulation for embodied agents. In this episode of Embodied AI 101, we explore "Stop Making the Video Model Remember the World". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Atlas Turns Phone Scans Into Robot Worlds—But Not Yet Into Physics

Shaoqing Tan — Fri, 11 Sep 2026 05:06:49 -0700

Atlas is a unified spatial intelligence model that reconstructs real scenes from 2–3 images, generates 1440p video along camera paths, and supports real-to-sim robotics workflows using Gaussian Splats to create photorealistic 3D digital twins from phone scans. This represents a significant step toward scalable spatial understanding for physical AI and robot training. In this episode of Embodied AI 101, we explore "Atlas Turns Phone Scans Into Robot Worlds—But Not Yet Into Physics". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

W²G-Net: Carrying Instance Identity from Shiny Scrap to the Gripper

Shaoqing Tan — Thu, 10 Sep 2026 05:12:02 -0700

Purpose This study aims to develop a robust vision-guided framework for instance-level recognition and grasp-based classification of metal waste in dense, multi-scale and highly reflective industrial environments, addressing the challenges... In this episode of Embodied AI 101, we explore "W²G-Net: Carrying Instance Identity from Shiny Scrap to the Gripper". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When Avoidance Is Not Enough: T-CARE Gives UAV Swarms a Right-of-Way

Shaoqing Tan — Thu, 10 Sep 2026 05:11:16 -0700

Multi-agent UAV navigation in cluttered environments is challenged by deadlock, starvation-like persistent yielding, oscillation, and unsafe congestion, particularly in narrow passages and dynamically emerging bottlenecks. Existing approaches often rely on... In this episode of Embodied AI 101, we explore "When Avoidance Is Not Enough: T-CARE Gives UAV Swarms a Right-of-Way". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Teaching Robots What "Better" Means

Shaoqing Tan — Wed, 09 Sep 2026 05:09:07 -0700

A method for preference-based learning applied to dexterous robotic manipulation, enabling robots to learn from human preferences in freeform settings. Discussed publicly around September 8, 2026. In this episode of Embodied AI 101, we explore "Teaching Robots What "Better" Means". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

TANGO: Stop Treating a Humanoid Like a Disk on the Floor

Shaoqing Tan — Wed, 09 Sep 2026 05:09:04 -0700

TANGO directly predicts 29-DoF joint actions from RGB observations for zero-shot real-world humanoid navigation, enabling coordinated arms, torso, and gait control in cluttered 3D spaces. Presented at CoRL 2026. In this episode of Embodied AI 101, we explore "TANGO: Stop Treating a Humanoid Like a Disk on the Floor". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Stop Teaching the VLA Your Camera: Robot-Centric Pointmaps as an Action-Aligned Visual Interface

Shaoqing Tan — Tue, 08 Sep 2026 05:11:21 -0700

Vision-language-action (VLA) models predict robot actions from visual observations and language instructions. These actions are defined in the robot's own 3D coordinate frame, yet most VLAs observe the scene in 2D. This paper proposes robot-centric pointmaps to bridge this gap for vision-language-action models. In this episode of Embodied AI 101, we explore "Stop Teaching the VLA Your Camera: Robot-Centric Pointmaps as an Action-Aligned Visual Interface". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

One Policy, Many Bodies: Qwen-VLA's Bid to Unify Embodied AI

Shaoqing Tan — Tue, 08 Sep 2026 05:09:25 -0700

Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments. In this episode of Embodied AI 101, we explore "One Policy, Many Bodies: Qwen-VLA's Bid to Unify Embodied AI". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When the Simulator Lies About Friction: Physics as a Transfer Prior for Touch

Shaoqing Tan — Mon, 07 Sep 2026 05:10:07 -0700

Physics-Regularized Learning for Robust Sim-to-Real Tactile Friction Estimation In this episode of Embodied AI 101, we explore "When the Simulator Lies About Friction: Physics as a Transfer Prior for Touch". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Seeing Around Corners, Acting at the Edge: REACT as a VLA Safety Co-Pilot

Shaoqing Tan — Mon, 07 Sep 2026 05:08:57 -0700

Edge-based multimodal sensor data fusion with Vision-Language-Action (VLA) model for real-time autonomous vehicle accident avoidance In this episode of Embodied AI 101, we explore "Seeing Around Corners, Acting at the Edge: REACT as a VLA Safety Co-Pilot". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

RoboTok: Turning the Internet Into a Motion Search Engine for Dexterous Robots

Shaoqing Tan — Sun, 06 Sep 2026 05:09:17 -0700

RoboTok retrieves human hand-pose trajectories from internet videos to automatically generate scalable robot training data for dexterous manipulation tasks. This approach enables large-scale data collection without manual annotation or teleoperation. In this episode of Embodied AI 101, we explore "RoboTok: Turning the Internet Into a Motion Search Engine for Dexterous Robots". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

VLAct: The VLA Backbone Is Not Just Plumbing

Shaoqing Tan — Sun, 06 Sep 2026 05:09:09 -0700

VLAct proposes a representation-centric continued pre-training approach for Vision-Language-Action models that converts limited robot data into transferable action knowledge, achieving 92.5% success on RoboTwin with only 20% of full training data. The method and weights are fully open-sourced. In this episode of Embodied AI 101, we explore "VLAct: The VLA Backbone Is Not Just Plumbing". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Two Models, One Gradient: Making World-Model RL Work for Contact-Rich Robots

Shaoqing Tan — Sat, 05 Sep 2026 05:08:42 -0700

Breaks intractable full world-model RL into a heavy global trajectory model combined with a lightweight latent local dynamics approximator, enabling scalable RL for contact-rich humanoid skills without backpropagating through the full model. In this episode of Embodied AI 101, we explore "Two Models, One Gradient: Making World-Model RL Work for Contact-Rich Robots". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

GigaBrain-0.7: Scaling Robot Intelligence by Separating Acting, Thinking, and Imagining

Shaoqing Tan — Sat, 05 Sep 2026 05:07:13 -0700

Introduces a three-system architecture for scaling embodied foundation models, representing a new approach to large-scale embodied AI. In this episode of Embodied AI 101, we explore "GigaBrain-0.7: Scaling Robot Intelligence by Separating Acting, Thinking, and Imagining". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Geometry Is the Contact Plan: Bringing Surface Reasoning Into MPC

Shaoqing Tan — Fri, 04 Sep 2026 05:08:42 -0700

Geometry-Informed Contact Optimization and Model Predictive Control for Efficient Contact-Rich Manipulation In this episode of Embodied AI 101, we explore "Geometry Is the Contact Plan: Bringing Surface Reasoning Into MPC". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

A Swarm That Knows Its Margin: Safe Multi-Drone Transport With CBF-RL

Shaoqing Tan — Fri, 04 Sep 2026 05:06:42 -0700

Safe and Scalable Multi-Drone Payload Transport via CBF-Based Reinforcement Learning With Zero-Shot Sim-to-Real Transfer In this episode of Embodied AI 101, we explore "A Swarm That Knows Its Margin: Safe Multi-Drone Transport With CBF-RL". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Optimizer Becomes a Policy: Inside RP-1's Learned Planner for Robot World Models

Shaoqing Tan — Thu, 03 Sep 2026 15:34:33 -0700

Pantheon Industries introduces Reinforced Planning (RP-1), the first fully learned planner that iteratively improves robot action plans from scratch using Reinforcement Learning over a frozen World Model. RP-1 reimagines planning as a learned policy: starting with a candidate action sequence, it uses a frozen World Model to imagine outcomes, a learned critic to evaluate closeness to goal (value-based scoring instead of latent distance), and a neural planner to revise the sequence over multiple iterations. This replaces hand-designed search heuristics (CEM, MPPI, Adam) with learned, reusable plan-improvement rules. Key results across ThreeRoom, OGBench, and Reacher benchmarks: RP-1 beats SOTA latent methods on 48 of 48 head-to-head comparisons, achieves 2.2x higher success rate on hard and long-horizon tasks, is 67x faster at scale (50 robot arms on H200 GPUs), and requires 1000x fewer World Model queries than MPPI/CEM. In this episode of Embodied AI 101, we explore "The Optimizer Becomes a Policy: Inside RP-1's Learned Planner for Robot World Models". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

SAGA: Ground the Affordance, Then Learn the Motion

Shaoqing Tan — Thu, 03 Sep 2026 05:14:16 -0700

SAGA: Open-World Mobile Manipulation via Structured Affordance Grounding In this episode of Embodied AI 101, we explore "SAGA: Ground the Affordance, Then Learn the Motion". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

REMAC: Turning Robot Failures into Better Team Plans

Shaoqing Tan — Thu, 03 Sep 2026 05:07:14 -0700

REMAC: Self-reflective and self-evolving multi-agent collaboration for long-horizon robot manipulation In this episode of Embodied AI 101, we explore "REMAC: Turning Robot Failures into Better Team Plans". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Restore the Sensor Contract: Training-Free Robustness for Brittle VLA Policies

Shaoqing Tan — Wed, 02 Sep 2026 05:17:50 -0700

Vision–Language–Action (VLA) policies remain brittle under modest distribution shift. On LIBERO-Plus, contemporary models that solve clean tasks at high rates can fall below 30% success when the camera's viewpoint or background changes. This paper proposes a training-free method to improve out-of-distribution (OOD) robustness for VLA policies. In this episode of Embodied AI 101, we explore "Restore the Sensor Contract: Training-Free Robustness for Brittle VLA Policies". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Before the Robot Touches Anything: Physics-Grounded Deliberation with Embodied Tree of Thoughts

Shaoqing Tan — Wed, 02 Sep 2026 05:13:01 -0700

Embodied Tree of Thoughts: Deliberate Manipulation Planning With Embodied World Model In this episode of Embodied AI 101, we explore "Before the Robot Touches Anything: Physics-Grounded Deliberation with Embodied Tree of Thoughts". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

MemoAct: Why Robot Memory Needs Both a Scratchpad and an Archive

Shaoqing Tan — Tue, 01 Sep 2026 05:09:03 -0700

MemoAct: Atkinson–Shiffrin-Inspired Hierarchical Memory-Augmented Policy for Robotic Manipulation In this episode of Embodied AI 101, we explore "MemoAct: Why Robot Memory Needs Both a Scratchpad and an Archive". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When Touch Means Something: OmniVTLA and the Missing Contact Layer in Robot Foundation Models

Shaoqing Tan — Tue, 01 Sep 2026 05:09:00 -0700

OmniVTLA: Vision-Tactile-Language-Action Models With Semantic-Aligned Tactile Sensing In this episode of Embodied AI 101, we explore "When Touch Means Something: OmniVTLA and the Missing Contact Layer in Robot Foundation Models". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Gaussians as State, Futures as Supervision: Inside GSP-3D

Shaoqing Tan — Mon, 31 Aug 2026 05:15:32 -0700

Introduction Learning robust visuomotor policies for bimanual manipulation remains challenging due to the stringent requirements for precise coordination between arms and the ability to generalize across diverse environmental conditions. Existing approaches struggle with generalization across diverse environmental conditions for long-horizon bimanual manipulation tasks. In this episode of Embodied AI 101, we explore "Gaussians as State, Futures as Supervision: Inside GSP-3D". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

SIRModel and the Missing Geometry Layer in Vision–Language Manipulation

Shaoqing Tan — Mon, 31 Aug 2026 05:11:08 -0700

Long-horizon robotic manipulation requires a policy to bridge task-level semantic reasoning with metric three-dimensional interaction geometry. Existing vision–language–action policies usually acquire geometry implicitly from visual tokens or introduce deterministic intermediate representations that lack spatial grounding. This paper proposes SIRModel, which learns a spatial intermediate representation to parameter-efficiently fine-tune a vision language model for robotic manipulation tasks. In this episode of Embodied AI 101, we explore "SIRModel and the Missing Geometry Layer in Vision–Language Manipulation". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Robot That Knows When to Think Twice: Inside τ0-VLA

Shaoqing Tan — Sun, 30 Aug 2026 05:18:02 -0700

Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks. Most hierarchical vision-language-action (VLA) models make each such decision with a fixed compute budget. τ0-VLA introduces world-model-guided test-time computation to a hierarchical VLA, enabling more deliberate planning for long-horizon tasks. In this episode of Embodied AI 101, we explore "The Robot That Knows When to Think Twice: Inside τ0-VLA". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When Should a Robot Stop Imagining? RISE and Adaptive World-Action Planning

Shaoqing Tan — Sun, 30 Aug 2026 05:10:22 -0700

Introduces dynamic Roll/Stop decision-making in world action models that adapts imagination based on planning benefit, risk, and compute cost for more efficient robot planning. Enables world action models to selectively engage imagination only when beneficial. In this episode of Embodied AI 101, we explore "When Should a Robot Stop Imagining? RISE and Adaptive World-Action Planning". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Crib as a Causal Laboratory: How a Playing iCub Learns "I Did That"

Shaoqing Tan — Sat, 29 Aug 2026 05:22:38 -0700

Learning sensorimotor contingencies—that is, the link between one's actions and their sensory effects—is fundamental to developing body knowledge, understanding causality, and developing a sense of agency. In developmental psychology, this... In this episode of Embodied AI 101, we explore "The Crib as a Causal Laboratory: How a Playing iCub Learns "I Did That"". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Feeling Around Corners: STRP Compresses Fingertip Force into Directional Touch

Shaoqing Tan — Sat, 29 Aug 2026 05:09:06 -0700

High-voltage grid maintenance and chemical processing demand dexterous manipulation in confined, visually occluded environments. Existing teleoperation systems present a severe cost–fidelity trade-off: industrial platforms exceed USD 60,000 yet lack spatial force feedback. STRP addresses this with a low-cost platform featuring spatial force feedback. In this episode of Embodied AI 101, we explore "Feeling Around Corners: STRP Compresses Fingertip Force into Directional Touch". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Give the Swarm a Pheromone Compass

Shaoqing Tan — Fri, 28 Aug 2026 05:08:18 -0700

Search-and-rescue (SAR) operations in disaster environments require drone swarms to coordinate efficiently despite incomplete information and potential communication failures. Existing stigmergy-based approaches provide low-bandwidth coordination but rely on fixed rules, while multi-agent reinforcement learning (MARL) offers adaptive behavior but struggles with scalability and coordination. This paper proposes a novel approach combining directional pheromone gradient observations with decentralized MARL for swarm drone SAR missions. In this episode of Embodied AI 101, we explore "Give the Swarm a Pheromone Compass". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When Every Weld Looks the Same: Language-Labeled Graphs for Industrial Localization

Shaoqing Tan — Fri, 28 Aug 2026 05:07:16 -0700

Abstract As industries increasingly move towards automation, the ability to visually localize objects and accurately estimate their pose, including rigid and non-rigid transformations, becomes critical. Conventional pose estimation methods are... In this episode of Embodied AI 101, we explore "When Every Weld Looks the Same: Language-Labeled Graphs for Industrial Localization". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When Robot Eyes Converge: Active Stereo for Humanoid Depth

Shaoqing Tan — Thu, 27 Aug 2026 05:13:18 -0700

The design of robotic binocular camera systems has been inspired by human vision, as has their use in humanoid robots. One aspect of this inspiration has yet to play a role in robotics: convergent binocular stereo, where the cameras are angled inward to overlap their fields of view for improved depth perception. In this episode of Embodied AI 101, we explore "When Robot Eyes Converge: Active Stereo for Humanoid Depth". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Two Clocks for Touch: How T-Rex Gives Dexterous VLAs a Reflex Arc

Shaoqing Tan — Thu, 27 Aug 2026 05:08:24 -0700

Introduces a full-stack co-designed tactile-reactive policy with asynchronous mixture-of-transformers (slow visuomotor planning + fast 4× tactile corrections), trained on the largest open 50-hour / 5,500-episode synchronized robot tactile play dataset collected on 22-DoF hands; bridges egocentric video pretraining to contact-rich manipulation via EgoScale mid-training. In this episode of Embodied AI 101, we explore "Two Clocks for Touch: How T-Rex Gives Dexterous VLAs a Reflex Arc". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Let the VLM Decide, Let the Planner Drive: Inside Senna

Shaoqing Tan — Wed, 26 Aug 2026 05:10:18 -0700

Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving In this episode of Embodied AI 101, we explore "Let the VLM Decide, Let the Planner Drive: Inside Senna". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Seeing the Nearly Invisible: Stereo Vision for Cable-Safe Agricultural UAVs

Shaoqing Tan — Wed, 26 Aug 2026 05:07:36 -0700

This study develops a binocular stereo vision system to enhance the operational safety of agricultural unmanned aerial vehicles (UAVs) by detecting and localizing thin cables—one of the most prevalent hazards in low-altitude flight. In this episode of Embodied AI 101, we explore "Seeing the Nearly Invisible: Stereo Vision for Cable-Safe Agricultural UAVs". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Taming the Flying Tip: Constrained MPC for Dynamic Deformable Linear Objects

Shaoqing Tan — Tue, 25 Aug 2026 05:10:00 -0700

Deformable linear objects (DLOs) exhibit highly nonlinear dynamic behavior, complicating their control during high-speed maneuvers. Furthermore, the lack of a generic spatial representation and the difficulty of real-time state estimation... In this episode of Embodied AI 101, we explore "Taming the Flying Tip: Constrained MPC for Dynamic Deformable Linear Objects". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Replay Is Not Reality: Embodied RL When Both Policy and Physics Shift

Shaoqing Tan — Tue, 25 Aug 2026 05:08:31 -0700

Abstract Embodied agents must continuously adapt to the physical world using interaction data collected across varying timescales, controllers, and environmental conditions. However, standard reinforcement learning assumes stationary dynamics and on-policy data collection, which breaks down when agents must learn from heterogeneous data sources with non-stationary dynamics and off-policy behavior. In this episode of Embodied AI 101, we explore "Replay Is Not Reality: Embodied RL When Both Policy and Physics Shift". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

PPO Beyond the Simulator: What a TurtleBot3 Mapless Navigation Study Really Demonstrates

Shaoqing Tan — Mon, 24 Aug 2026 05:20:46 -0700

Objective: In the field of mobile robotics, autonomous navigation in dynamic environments is one of the most challenging tasks in these environments: traditional methods based on pre-mapping and geometric planning... In this episode of Embodied AI 101, we explore "PPO Beyond the Simulator: What a TurtleBot3 Mapless Navigation Study Really Demonstrates". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

VLAs in the Wild: What 142 Papers Really Say About Generalist Robot Intelligence

Shaoqing Tan — Sun, 23 Aug 2026 05:08:41 -0700

Generalist robots need to perform diverse tasks while operating in dynamic, uncertain, and unstructured environments, often around human beings. Vision-language-action (VLA) models have recently emerged as a promising and flexible approach to enabling generalist robot behavior. This is a systematic literature review of VLA models for generalist robots. In this episode of Embodied AI 101, we explore "VLAs in the Wild: What 142 Papers Really Say About Generalist Robot Intelligence". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

VEDA and the Moment Vision Stops Being Enough

Shaoqing Tan — Sat, 22 Aug 2026 05:11:22 -0700

Industrial humanoid robots are increasingly expected to execute high-precision manipulation tasks, such as assembly, welding, and material handling, under dynamic and contact-rich working conditions. In this episode of Embodied AI 101, we explore "VEDA and the Moment Vision Stops Being Enough". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Three Ways to Buy Back Robot Data: Augment the Demonstration, Import the Semantics, Benchmark the Physics

Shaoqing Tan — Fri, 21 Aug 2026 05:08:59 -0700

Abstract Robots have the potential to assist people in homes, hospitals, warehouses, and factories, but today's systems still struggle to learn robust manipulation skills from limited data, especially in contact-rich tasks. This paper explores data-efficient robot learning approaches for contact-rich manipulation. In this episode of Embodied AI 101, we explore "Three Ways to Buy Back Robot Data: Augment the Demonstration, Import the Semantics, Benchmark the Physics". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

UI-Mate: Show the Desktop Agent Once, Then Let It Replan

Shaoqing Tan — Thu, 20 Aug 2026 05:21:49 -0700

A 27B model that achieves one-shot doubling of strict success rates on long-horizon office automation tasks via demonstration-conditioned policy learning. In this episode of Embodied AI 101, we explore "UI-Mate: Show the Desktop Agent Once, Then Let It Replan". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Frozen VLM That Learns Anyway: Inside Spatial Memory Agent

Shaoqing Tan — Thu, 20 Aug 2026 05:09:24 -0700

A parameter-update-free framework that distills verified experience into transferable spatial memory for frozen VLMs, achieving consistent gains across 4 VLMs and 5 spatial reasoning benchmarks. In this episode of Embodied AI 101, we explore "The Frozen VLM That Learns Anyway: Inside Spatial Memory Agent". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The LLM Proposes, the Planner Disposes: VLA-SP for Verifiable Robot Manipulation

Shaoqing Tan — Wed, 19 Aug 2026 05:14:39 -0700

Abstract Cross-modal foundation models are increasingly used for robotic task understanding and planning. However, connecting multimodal observations and natural language instructions to symbolic planners and executable robot actions remains a challenge. This paper addresses long-horizon robotic manipulation tasks using LLM-driven symbolic planning. In this episode of Embodied AI 101, we explore "The LLM Proposes, the Planner Disposes: VLA-SP for Verifiable Robot Manipulation". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

When Vision Cannot Know: MulPlanLM Lets Robot Planners Lift, Weigh, and Decide

Shaoqing Tan — Wed, 19 Aug 2026 05:12:55 -0700

MulPlanLM: multimodal robotic task planning with vision-language models and physical feedback In this episode of Embodied AI 101, we explore "When Vision Cannot Know: MulPlanLM Lets Robot Planners Lift, Weigh, and Decide". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Reading Trust in Motion: A Multimodal Perception Stack for Industrial HRC

Shaoqing Tan — Tue, 18 Aug 2026 05:16:33 -0700

Abstract Industry 5.0 promotes collaborative manufacturing environments where humans and robots work together, leveraging their complementary strengths to improve productivity and safety. In such settings, trust is fundamental to ensuring effective collaboration between humans and robots. This paper presents a multi-modal model for trust evaluation in industrial human-robot collaboration settings. In this episode of Embodied AI 101, we explore "Reading Trust in Motion: A Multimodal Perception Stack for Industrial HRC". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Two MPCs, One Robot: Bézier Curves for Fast, Compliant Mobile Manipulation

Shaoqing Tan — Tue, 18 Aug 2026 05:13:59 -0700

Dual-arm mobile manipulators can transport and manipulate large objects with simple end-effectors. Interacting with dynamic environments subject to strict safety and compliance requirements, achieving whole-body motion planning online while maintaining compliance is challenging. This paper presents an efficient whole-body model predictive control approach for online compliant dual-arm mobile manipulation. In this episode of Embodied AI 101, we explore "Two MPCs, One Robot: Bézier Curves for Fast, Compliant Mobile Manipulation". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Reward Behind the Gait: Learning Transferable Locomotion from a Few Stick-Insect Steps

Shaoqing Tan — Mon, 17 Aug 2026 05:20:35 -0700

Insect locomotion exhibits remarkable adaptability and flexibility despite the limited scale of its nervous system. This paper extracts and models underlying leg coordination principles via adversarial inverse reinforcement learning, enabling transferable robot locomotion from limited biological data. In this episode of Embodied AI 101, we explore "The Reward Behind the Gait: Learning Transferable Locomotion from a Few Stick-Insect Steps". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Beyond Looking Human: Bio-Functional Robot Design for Cross-Scale Dexterity and Tool Use

Shaoqing Tan — Mon, 17 Aug 2026 05:16:18 -0700

Dexterous hands and robotic manipulators as physical interfaces for diverse environments. Proposes bio-functional mimicry beyond structural biomimicry, enabling cross-scale manipulation and limb-tool integration for versatile robotic tasks. In this episode of Embodied AI 101, we explore "Beyond Looking Human: Bio-Functional Robot Design for Cross-Scale Dexterity and Tool Use". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

LDA-1B and the Case for Never Throwing Robot Data Away

Shaoqing Tan — Sun, 16 Aug 2026 05:07:35 -0700

Scales a latent dynamics action model to 1B parameters by ingesting diverse embodied datasets into a unified training pipeline for generalist robot policies. In this episode of Embodied AI 101, we explore "LDA-1B and the Case for Never Throwing Robot Data Away". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Flex-π: One Robot Policy, Three Futures, Fifty-Six Ways to Run It

Shaoqing Tan — Wed, 12 Aug 2026 14:08:45 -0700

Jointly predicts future RGB frames, 3D pointmaps, DINO semantics, and actions within a unified world-action model. Can be deployed flexibly as a VLA, full world-action model, or hybrid from a single checkpoint. In this episode of Embodied AI 101, we explore "Flex-π: One Robot Policy, Three Futures, Fifty-Six Ways to Run It". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Imagined Motion, Grounded Geometry: How Rest2Art Makes Closed Objects Articulate

Shaoqing Tan — Wed, 12 Aug 2026 14:07:41 -0700

An ECCV 2026 paper that converts a single static closed-state image into a simulation-ready articulated asset using video diffusion for joint hypothesis generation, requiring no motion data. Enables articulated object reconstruction without dynamic observations. In this episode of Embodied AI 101, we explore "Imagined Motion, Grounded Geometry: How Rest2Art Makes Closed Objects Articulate". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

ComBodied Agents: When the Human Becomes the World Model

Shaoqing Tan — Wed, 12 Aug 2026 05:12:03 -0700

Proposes a new agent architecture that unifies digital and physical (embodied) action spaces, supporting human-state trajectories with explicit handling of consent, uncertainty, and reversibility for safer human-robot interaction. In this episode of Embodied AI 101, we explore "ComBodied Agents: When the Human Becomes the World Model". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

A Sphere Every Robot Hand Can Speak

Shaoqing Tan — Tue, 11 Aug 2026 14:08:14 -0700

Introduces a unified hand action space that enables seamless cross-embodiment transfer for dexterous manipulation policies across different robot hands. In this episode of Embodied AI 101, we explore "A Sphere Every Robot Hand Can Speak". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Soft Gripper Pareto Frontier: Adaptability Without Surrendering Payload

Shaoqing Tan — Tue, 11 Aug 2026 05:10:20 -0700

Soft gripper systems have inherent flexibility and remarkable interactivity security. However, balancing load-bearing capacity, deformation, complexity, and adaptability in a single gripper remains a substantial challenge. In this episode of Embodied AI 101, we explore "The Soft Gripper Pareto Frontier: Adaptability Without Surrendering Payload". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Dyna-2's Million-Hour Bet: When Human Video Starts Improving Robots

Shaoqing Tan — Tue, 11 Aug 2026 05:08:30 -0700

Pre-trained on 1 million hours of human video, Dyna-2 demonstrates scaling laws for world-action models across four orders of magnitude on human data, with emergent scaling on unseen robot data and joint video + action objectives enabling cross-embodiment transfer. In this episode of Embodied AI 101, we explore "Dyna-2's Million-Hour Bet: When Human Video Starts Improving Robots". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Think Deeply Only When It Matters: DySL-VLA and Action-Aware Dynamic Depth

Shaoqing Tan — Mon, 10 Aug 2026 14:10:38 -0700

Vision-Language-Action (VLA) models have shown remarkable success in robotic tasks like manipulation by fusing a language model's reasoning with a vision model's 3D understanding. However, their high computational cost remains... In this episode of Embodied AI 101, we explore "Think Deeply Only When It Matters: DySL-VLA and Action-Aware Dynamic Depth". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The 100,000-Hour Bet: How Xiaomi-Robotics-1 Tries to Make Robot Scaling Real

Shaoqing Tan — Mon, 10 Aug 2026 14:09:24 -0700

We present Xiaomi-Robotics-1, a foundational vision-language-action (VLA) model capable of (1) following diverse language instructions to perform a wide range of mobile manipulation tasks in unseen environments out-of-the-box, and (2) rapidly adapting to new tasks with minimal demonstrations through efficient fine-tuning. In this episode of Embodied AI 101, we explore "The 100,000-Hour Bet: How Xiaomi-Robotics-1 Tries to Make Robot Scaling Real". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

Reason First, Move Second: A Modular Alternative to End-to-End Robot Policies

Shaoqing Tan — Mon, 10 Aug 2026 05:10:58 -0700

Explicit task reasoning empowering robotic manipulation In this episode of Embodied AI 101, we explore "Reason First, Move Second: A Modular Alternative to End-to-End Robot Policies". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

From BIM to the Build: A Multimodal Agent for Conversational Construction Robotics

Shaoqing Tan — Mon, 10 Aug 2026 05:10:32 -0700

A multimodal foundation model-enabled agent for human–robot collaboration in construction In this episode of Embodied AI 101, we explore "From BIM to the Build: A Multimodal Agent for Conversational Construction Robotics". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

The Twin Watches, the QP Dodges: Predictive Safety for a UR10e

Shaoqing Tan — Sun, 09 Aug 2026 14:08:23 -0700

Safe human–robot collaboration remains a critical challenge in manufacturing. Traditional safety approaches, such as cages and proximity sensors, are often insufficient for dynamic human interaction. This paper presents a digital twin approach using deep learning for collision avoidance in industrial collaborative robot manipulation. In this episode of Embodied AI 101, we explore "The Twin Watches, the QP Dodges: Predictive Safety for a UR10e". We break down the research, methodology, and real-world implications for robotics, AI, and physical intelligence. Embodied AI 101 covers the latest research at the intersection of AI and physical intelligence — robotics, manipulation, world models, and the path from digital intelligence to embodied agents.

One Shot, Twenty Screws: Robot Vision for the Dirty Reality of Appliance Recycling

Shaoqing Tan — Sun, 09 Aug 2026 14:08:02 -0700

Industrial-grade robust robot vision system for screw detection and removal under uneven conditions.

Better Geometry at the Neural–Symbolic Boundary: DAIoU for Robot Manipulation

Shaoqing Tan — Sun, 09 Aug 2026 05:10:53 -0700

Distance–area–IoU fusion loss for better neuro-symbolic robot manipulation

REL Hand: A Mechanical Shortcut to More Human-Like Humanoid Dexterity

Shaoqing Tan — Sun, 09 Aug 2026 05:10:30 -0700

Design and evaluation of a tendon-and-linkage hybrid-driven humanoid dexterous hand

When the Appliance Becomes the API: Part–Function–State Models for Robot Planning

Shaoqing Tan — Sat, 08 Aug 2026 14:09:13 -0700

Appliances Describe Themselves: Part–Function–State Modeling for Appliance Manipulation Planning

CISMG-Nav: When a Navigation Agent Stops Forgetting the Building

Shaoqing Tan — Sat, 08 Aug 2026 05:12:12 -0700

CISMG-Nav: A Cross-Task Incremental Semantic Memory Graph-Driven Vision-and-Language Navigation Framework

ReaDy-Go: Teaching Robots to Dodge Photorealistic People in Gaussian-Splat Digital Twins

Shaoqing Tan — Sat, 08 Aug 2026 05:10:12 -0700

ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation With Moving Obstacles

The Action Is Missing: Turning Human Video into Robot Control

Shaoqing Tan — Fri, 07 Aug 2026 14:07:43 -0700

Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models. However, most existing approaches rely on large collections of robot demonstrations, which are costly. This survey covers scalable vision-language-action learning using human-centric data, particularly human videos, as an alternative to expensive robot demonstrations.

Look Where You Mean: Gaze2Act Gives VLAs a Live Intent Channel

Shaoqing Tan — Fri, 07 Aug 2026 05:41:28 -0700

Vision-Language-Action (VLA) models have recently shown strong potential for robot learning by following language instructions. However, in practice, language alone is often insufficient to precisely convey human intent.

HARP-VLA: Align the Eyes Before the Actions

Shaoqing Tan — Fri, 07 Aug 2026 05:17:16 -0700

Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions.

Make Geometry the Policy: GAM's Bet on 3D-Native Robot Control

Shaoqing Tan — Thu, 06 Aug 2026 14:08:33 -0700

Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent VLAs and video world-action models lack explicit 3D geometric reasoning.

π0.7: When a Robot Foundation Model Learns How to Behave

Shaoqing Tan — Thu, 06 Aug 2026 14:08:20 -0700

We present a new robotic foundation model, π0.7, that can enable strong out-of-the-box performance in a wide range of scenarios, following diverse language instructions in unseen environments.

EgoHumanoid: Turning Human First-Person Experience into Humanoid Loco-Manipulation

Shaoqing Tan — Thu, 06 Aug 2026 05:06:51 -0700

Introduces a dataset and method for learning loco-manipulation policies directly from egocentric human videos without any robot data collection, enabling real-world humanoid deployment in unstructured environments.

CycleRL: Teaching a Bicycle to Balance in Simulation—and Trusting It on Asphalt

Shaoqing Tan — Wed, 05 Aug 2026 14:09:42 -0700

CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control

Feeling the World Without a Force Sensor: UniFP for Legged Loco-Manipulation

Shaoqing Tan — Wed, 05 Aug 2026 14:08:36 -0700

Proposes a unified RL policy that jointly learns force and position control in simulation without force sensors, enabling compliant behaviors and force-aware imitation learning. Includes a force estimator trained as an auxiliary task and integration with Diffusion Policy for contact-rich tasks on quadrupeds and humanoids.

The Robot Needs a Memory: Multi-Agent Reasoning Inside a Digital Twin

Shaoqing Tan — Wed, 05 Aug 2026 05:13:34 -0700

Digital Twin-enabled adaptive robotics: multi-agent reasoning over language, vision, and structured database

Language Is Not Just a Prompt: Four Ways Robots Turn Words into Action

Shaoqing Tan — Wed, 05 Aug 2026 05:12:40 -0700

Language-conditioned robot manipulation is an emerging field aimed at enabling seamless communication and cooperation between humans and robotic agents by teaching robots to comprehend and execute instructions conveyed in natural language.

Becoming the Path: Planning Motion, Shape, and Stiffness Together

Shaoqing Tan — Wed, 05 Aug 2026 03:12:44 -0700

Autonomous operation in confined spaces demands robots that can simultaneously navigate tight passages and manipulate objects. However, conventional mobile manipulators are often too large, while compliant soft manipulators are typically too weak.

Meaning, Motion, and Two Hands: A Task Graph That Actually Plans

Shaoqing Tan — Wed, 05 Aug 2026 03:09:56 -0700

Semantic–Geometric Task Representations for Bimanual Manipulation From Human Demonstrations to Robot Action Planning

Safety Before Failure: Giving Robot Plans a One-Second Reflex

Shaoqing Tan — Tue, 04 Aug 2026 14:10:05 -0700

Robots that execute language-conditioned tasks in dynamic environments often rely on feedback only after an action has failed, which can be insufficient when failures involve collisions or workspace conflicts.

VL-GRiP3 and the Case for Keeping VLMs Away from the Motor Loop

Shaoqing Tan — Tue, 04 Aug 2026 14:09:46 -0700

VL-GRiP3: A hierarchical pipeline leveraging vision-language models for autonomous robotic 3D grasping

Think Late, Act Now: TIC-VLA Turns Reasoning Latency into a Control Variable

Shaoqing Tan — Tue, 04 Aug 2026 05:09:18 -0700

Robots in dynamic, human-centric environments must follow language instructions while maintaining real-time reactive control. Vision-language-action (VLA) models offer a promising framework, but they assume temporally aligned reasoning and control.

One Policy, Many Bodies: Inside Qwen-VLA's Generalist Robot Model

Shaoqing Tan — Tue, 04 Aug 2026 05:08:53 -0700

VLAbot: A Human-in-the-Loop Control Plane for Long-Horizon Robot Assembly

Shaoqing Tan — Tue, 04 Aug 2026 03:13:39 -0700

VLAbot: A human Vision–Language–Action models interaction framework for robotic assembly

Teaching Isaac Sim to Feel: A Real-Time Digital Twin for Capacitive Touch

Shaoqing Tan — Tue, 04 Aug 2026 03:12:11 -0700

With advances in robotic manipulation in recent years, tactile sensing has become increasingly important in scenarios where visual information is unreliable or insufficient.

Stop Starting From Noise: Action-to-Action Flow Matching for Fast Robot Control

Shaoqing Tan — Mon, 03 Aug 2026 14:08:02 -0700

Proposes a new flow-matching approach for robot action generation that improves efficiency in learning and repurposing generalist policies. Targets improved scalability and adaptability of robot action models.

ACE-Data-0: Turning a Home into a Multimodal Robot-Learning Instrument

Shaoqing Tan — Mon, 03 Aug 2026 05:19:45 -0700

An embodied data engine releasing 150 hours of synchronized egocentric video, full-body motion, hand tracking, object trajectories, audio, and tactile signals from everyday household activities.

Before the Robot Feels It: How N₀-VTLA Predicts Contact to Control It

Shaoqing Tan — Mon, 03 Aug 2026 05:17:21 -0700

A vision-tactile-language-action model that predicts latent tactile tokens for fine-grained contact-rich manipulation, achieving wins on all 9 real-robot tasks and 63.8% on 20 simulation tasks.

Metis Gives Foundation Models a Native, Mutable Memory

Shaoqing Tan — Mon, 03 Aug 2026 03:10:26 -0700

Equips foundation models with persistent, dynamically evolving native memory state and gradient-free online updates, enabling continual adaptation without retraining.

TurboVLA and the Case for Taking the LLM Out of the Servo Loop

Shaoqing Tan — Mon, 03 Aug 2026 03:08:47 -0700

Achieves high-speed inference for vision-language-action policies at 32 Hz on an RTX 4090 with less than 1 GB VRAM using a 0.2B parameter model, enabling real-time VLA deployment on consumer hardware.

Catching on Optical Expansion: Inside Pixel2Catch's Monocular Sim-to-Real Pipeline

Shaoqing Tan — Sun, 02 Aug 2026 14:09:45 -0700

Pixel2Catch: Multi-Agent Sim-to-Real Transfer for Agile Manipulation With a Single RGB Camera

Calibrate the Robot, Randomize the World: PACE for Legged Sim-to-Real

Shaoqing Tan — Sun, 02 Aug 2026 14:07:58 -0700

Legged robots must achieve both robust locomotion and energy efficiency to be practical in real-world environments. Yet controllers trained in simulation often fail to transfer reliably, and most existing approaches are limited to specific robot morphologies. This paper presents a systematic framework for sim-to-real transfer across diverse legged robot platforms.

HumanCLAW: The Model Sees the Chair but Loses the Body

Shaoqing Tan — Sun, 02 Aug 2026 03:09:15 -0700

Introduces a benchmark with 1,218 embodied episodes that decouples high-level VLM action decisions from low-level motor control; no current VLM solves the tasks, with the best achieving only 16.8% success, highlighting lack of embodied self-awareness.

Get Up, Adapt, Survive: Recovery Skills for Small Quadrupeds

Shaoqing Tan — Sat, 01 Aug 2026 14:08:06 -0700

Real-to-Sim-to-Real: Learning Agile and Robust Recovery Skills With Terrain Imagination via Adversarial Imitation

A Patch Only Needs a Glimpse: Breaking VLA Robots Under Partial Observability

Shaoqing Tan — Sat, 01 Aug 2026 14:07:12 -0700

Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics

A Thousand Tasks, One Day, and the Case Against Learning Everything End to End

Shaoqing Tan — Sat, 01 Aug 2026 05:21:06 -0700

Presents a scalable approach to robot task learning enabling acquisition of a large number of tasks within a single day. Details were discussed in a podcast episode released July 30, 2026.

HUG: Turning Everyday Human Grasps Into a Robot-Ready Prior

Shaoqing Tan — Sat, 01 Aug 2026 05:14:41 -0700

Collects 1M frames (27.8 hours) of egocentric human grasping video and trains a flow-matching model to predict and retarget hand poses to robots. Achieves large zero-shot gains on diverse everyday grasping tasks without requiring robot-specific training data.

SkillPlug: Teaching Robot Policies to Reuse the Moves Between Tasks

Shaoqing Tan — Sat, 01 Aug 2026 03:12:26 -0700

SkillPlug: Unsupervised Skill Mining for Few-Shot Adaptation in Robotic Manipulation

Expand Before You Compress: TEAM-VLA's Training-Free Shortcut to Faster Robot Policies

Shaoqing Tan — Sat, 01 Aug 2026 03:09:27 -0700

Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models

One Policy, Three Bodies, Zero Messages: Inside CHORUS

Shaoqing Tan — Wed, 29 Jul 2026 14:12:07 -0700

Trains a single VLA policy that enables multiple heterogeneous robots to collaborate on complex manipulation tasks using only local observations and robot IDs, avoiding centralized controllers.

Anchor the Semantics, Align the Actions: A Better Way to Fine-Tune VLAs

Shaoqing Tan — Wed, 29 Jul 2026 14:10:43 -0700

Proposes a VLA fine-tuning technique that preserves pretrained VLM representations while aligning language to actions, raising real-robot generalization success from 28% to 54%.

Learning the Rule, Not the Gait: Stick-Insect Behavior as a Portable Robot Objective

Shaoqing Tan — Wed, 29 Jul 2026 05:14:23 -0700

Insect locomotion exhibits remarkable adaptability and flexibility despite the limited scale of its nervous system. However, the underlying principles that govern leg coordination remain difficult to extract and model. This paper presents an adversarial inverse reinforcement learning approach to infer transferable locomotor principles from limited insect behavioral data, enabling robust robot locomotion.

Two Brains Under Pressure: How UnderwaterVLA Connects Language Models to AUV Control

Shaoqing Tan — Wed, 29 Jul 2026 05:13:33 -0700

The UnderwaterVLA dual-brain vision-language-action architecture enables robust autonomous underwater navigation

From Demo to Duty: RL-100's Real-World Reinforcement Learning Flywheel

Shaoqing Tan — Tue, 28 Jul 2026 18:31:25 -0700

Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass skilled human operators. We present a real-world reinforcement learning (RL) framework, RL-100, for achieving performant robotic manipulation through real-world RL training.

Distilling a Swarm into 158 Million Parameters: Inside MiniUAV-VLA

Shaoqing Tan — Tue, 28 Jul 2026 18:30:26 -0700

Coordinating multiple unmanned aerial vehicles (UAVs) for cooperative missions requires agents that perceive their environment, reason about objectives, and generate joint actions. Vision–language–action (VLA) models unify these capabilities but lack efficiency for real-world multi-UAV deployment. This episode explores MiniUAV-VLA, a compact 158M-parameter model that uses MARL expert distillation to enable cooperative multi-UAV search and elimination missions.

Imagining Locomotion: Learning a Neural World Model for Legged Robots

Shaoqing Tan — Thu, 23 Jul 2026 05:17:17 -0700

A neural dynamics world model paired with model-free RL policies for quadruped and humanoid locomotion in IsaacLab, enabling long-horizon autoregressive prediction and imagined rollouts that outperform pure model-based RL in prediction accuracy, policy learning, and sim-to-real transfer.

One Model to See, Plan, and Act: Introducing EO-1 for Embodied AI

Shaoqing Tan — Thu, 23 Jul 2026 03:11:34 -0700

A 3B parameter unified decoder-only transformer that interleaves vision, text, and action tokens for perception, planning, reasoning, and control in a single model. Trained on the 1.5M-sample EO-1.5M dataset with strong results across manipulation tasks and benchmarks.

Video-Action Models for Robot Learning

Shaoqing Tan — Thu, 23 Jul 2026 03:09:32 -0700

Introduces Video-Action Models (VAMs) that leverage pretrained internet-scale video models such as Cosmos-Predict2 as backbones instead of VLMs, paired with a flow-matching action decoder. Claims approximately 10x sample efficiency gains over standard vision-language-action models on real-world pick-and-place tasks.

FastDSAC: Unlocking the Potential of Maximum Entropy RL in High-Dimensional Humanoid Control

Shaoqing Tan — Wed, 22 Jul 2026 14:18:44 -0700

Introduces FastDSAC, a method that scales maximum entropy reinforcement learning to high-dimensional humanoid control tasks, improving sample efficiency and policy performance.

A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation

Shaoqing Tan — Wed, 22 Jul 2026 14:16:14 -0700

Proposes a minimalist pipeline using retargeting-guided RL for dexterous manipulation in humanoid robots. Focuses on simplifying the training recipe while maintaining strong manipulation performance.

DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation

Shaoqing Tan — Wed, 22 Jul 2026 03:22:38 -0700

DexNDM bridges the sim-to-real gap for stable in-hand rotation of complex objects, enabling learning from biased real-world data without requiring any successful demonstrations.

Learning Multi-Modal Trajectory Policies for Data-Efficient Robotic Manipulation

Shaoqing Tan — Wed, 22 Jul 2026 03:15:37 -0700

Proposes learning multi-modal trajectory policies to improve data efficiency in robotic manipulation tasks.

ACE: Agentic Control for Embodied Manipulation via Zero-shot Workflow Reasoning

Shaoqing Tan — Tue, 21 Jul 2026 05:17:04 -0700

A zero-shot workflow reasoning approach for agentic control of embodied manipulation, enabling robots to perform complex tasks without task-specific training.

Let the Body Follow: Coupled Egocentric Control for Whole-Body Robot Teleoperation

Shaoqing Tan — Tue, 21 Jul 2026 05:12:31 -0700

A coupled egocentric control approach for whole-body robot teleoperation that coordinates body and limb movements from an egocentric perspective, submitted to Humanoids 2026.

GigaWorld-Policy-0.5: An Efficient Mixture-of-Transformers Policy for Real-Time Robot Control

Shaoqing Tan — Mon, 20 Jul 2026 14:09:20 -0700

A Mixture-of-Transformers robot policy that achieves 85 ms inference on RTX 4090 by separating visual dynamics from action generation. Demonstrates efficient real-time robot policy execution through architectural decomposition.

Xiaomi-Robotics-1: A Scalable Vision-Language-Action Foundation Model for Mobile Manipulation

Shaoqing Tan — Mon, 20 Jul 2026 05:06:36 -0700

A scalable vision-language-action (VLA) foundation model pretrained on over 100k hours of real-world manipulation trajectories, enabling out-of-the-box mobile manipulation capabilities.

Local Policies Enable Zero-shot Long-Horizon Manipulation

Shaoqing Tan — Sun, 19 Jul 2026 14:16:30 -0700

Proposes training local policies in simulation that transfer zero-shot to real-world long-horizon robotic manipulation tasks. Addresses the challenge of generalizing learned behaviors across extended task sequences.

Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids

Shaoqing Tan — Sun, 19 Jul 2026 14:10:54 -0700

Trains perception-driven dexterous manipulation policies in simulation for zero-shot real-world transfer on humanoid robots. Focuses on bridging the sim-to-real gap for vision-based dexterous tasks.

UniTracker: A Universal Motion Tracking Framework for Humanoid Robots

Shaoqing Tan — Sun, 19 Jul 2026 05:18:06 -0700

A framework enabling humanoid robots to execute diverse motions within physical limits, with video demonstrations and open-source code. Targets generalizable motion tracking across varied humanoid morphologies.

ViTacFormer: Dexterous Manipulation via Active Vision and High-Resolution Touch Sensing

Shaoqing Tan — Sun, 19 Jul 2026 05:12:38 -0700

A pipeline combining active vision and high-resolution tactile sensing for dexterous manipulation on high-DoF robot hands, achieving approximately 2.5 minutes of continuous autonomous control on real-world tasks such as food preparation.

Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Language

Shaoqing Tan — Sun, 19 Jul 2026 03:10:22 -0700

Uses language and demonstrations to learn which aspects of behavior matter for reward in imitation learning, enabling 5× faster learning by avoiding copying irrelevant motion details. Accepted at ICRA 2026.

FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space

Shaoqing Tan — Sun, 19 Jul 2026 03:09:36 -0700

Steers frozen robot foundation models including VLAs, diffusion policies, and world models using human corrections via action inversion in latent space without fine-tuning the base policy. Outperforms supervised fine-tuning and latent RL with only 5–20 human interventions.

RoboTTT: End-to-End Robotic Assembly with Test-Time Training

Shaoqing Tan — Sat, 18 Jul 2026 05:13:30 -0700

Presents a single end-to-end policy that performs precise, unscripted assembly of complex objects, measuring every grasp and alignment without speed-ups or human intervention.

Xiaomi-Robotics-U0: A 38B World-Foundation Model for Unified Embodied Perception and Synthesis

Shaoqing Tan — Sat, 18 Jul 2026 05:07:10 -0700

38B autoregressive world foundation model unifying text-to-image, multi-view scene generation, embodied transfer, and video generation; ranks #1 on World Arena.

SpatialPoint: Spatial-Aware Point Prediction for Embodied Localization

Shaoqing Tan — Sat, 18 Jul 2026 03:19:51 -0700

A spatial-aware point prediction method for embodied localization tasks. Addresses grounding and spatial reasoning in embodied AI settings.

Multi-AUV Scene-Adaptive Embodied Intelligence for Multi-Target Tracking

Shaoqing Tan — Sat, 18 Jul 2026 03:19:44 -0700

An embodied AI framework for multi-AUV (Autonomous Underwater Vehicle) multi-target tracking in ad-hoc underwater networks. Incorporates scene-adaptive policies for dynamic underwater environments.

Speedup Patch: Learning a Plug-and-Play Policy to Accelerate Embodied Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 14:09:14 -0700

Proposes a plug-and-play policy module designed to accelerate existing embodied manipulation policies without retraining them from scratch.

R3DP: Real-Time 3D-Aware Policy for Embodied Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 14:08:01 -0700

Introduces a real-time 3D-aware policy framework for embodied manipulation tasks, enabling spatially grounded action generation.

VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 05:11:30 -0700

Presents a vision-language-action model grounded in 3D Gaussian representations that integrates geometric and semantic awareness for improved robotic manipulation.

SKooP: Symmetric Koopman Predictions for Legged Robot Reinforcement Learning

Shaoqing Tan — Fri, 17 Jul 2026 05:10:38 -0700

Leverages symmetric Koopman operator predictions within a reinforcement learning framework to achieve faster training and better generalization for legged robot locomotion.

RobotTT: A Tactile Transformer for Dexterous Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 03:22:51 -0700

A tactile foundation model and sim-to-real pipeline designed for dexterous manipulation tasks. Combines transformer-based tactile sensing with a sim-to-real transfer approach.

TAC-LOCO: Unified Whole-Body Control for Contact-Aware Locomotion and Manipulation

Shaoqing Tan — Thu, 16 Jul 2026 14:11:13 -0700

A tactile-informed whole-body loco-manipulation controller for quadrupedal robots that unifies locomotion and manipulation using tactile sensing. Enables compliant and contact-aware whole-body control.

TACTIC: Contact-Centric Control for Whole-Arm Manipulation

Shaoqing Tan — Thu, 16 Jul 2026 14:07:52 -0700

A contact-centric whole-arm manipulation framework that conditions control on both tactile and visual inputs. Targets dexterous manipulation tasks requiring rich contact feedback.

SIEVE: Structure-Aware Data Selection for Imitation Learning with VLAs

Shaoqing Tan — Thu, 16 Jul 2026 05:17:43 -0700

SIEVE introduces structure-aware data selection specifically for imitation learning with Vision-Language-Action models, aiming to improve training efficiency and policy quality.

DexMachina: RL for Long-Horizon Bimanual Dexterous Policies

Shaoqing Tan — Thu, 16 Jul 2026 05:10:47 -0700

An RL algorithm that learns long-horizon bimanual dexterous policies for any robot hand from a single human demonstration, emphasizing generalization across hands, objects, and complex motions.

Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation

Shaoqing Tan — Thu, 16 Jul 2026 03:17:13 -0700

A unified platform comprising GE-Base (video diffusion model trained on 1M+ manipulation episodes), GE-Act (flow-matching action model), and GE-Sim (neural world simulator for closed-loop control). All code, models, and benchmarks are open-sourced.

RoboTTT: Test-Time Training for Visuomotor Policies

Shaoqing Tan — Wed, 15 Jul 2026 14:09:54 -0700

Introduces test-time training (TTT) inside the policy to natively scale visuomotor context to 8K timesteps at constant inference cost, enabling one-shot imitation from human video demos, online self-recovery from errors, and long-horizon assembly tasks.

HY-Embodied-VLM-1.0: Efficient Physical-World Agents

Shaoqing Tan — Wed, 15 Jul 2026 14:08:55 -0700

An embodied vision-language-action model with released weights, code, and paper targeting robotic manipulation and embodied AI tasks.

Learning Unified Force and Position Control for Legged Loco-Manipulation

Shaoqing Tan — Wed, 15 Jul 2026 05:11:01 -0700

Introduces a unified RL policy that jointly handles force and position control on quadrupedal and humanoid robots without force sensors, enabling position+force tracking, compliant behaviors, and force-aware imitation learning for contact-rich tasks.

HapticVLA: Extending Vision-Language-Action Models to Contact-Rich Tasks Without Touch Sensors

Shaoqing Tan — Tue, 14 Jul 2026 14:15:54 -0700

Enables contact-rich robotic manipulation using a VLA model trained with tactile sensing data but requiring no tactile input at inference time. Distills haptic knowledge into the vision-language-action policy.

AnoleVLA: Lightweight VLA with Deep State Space Models for Mobile Manipulation

Shaoqing Tan — Tue, 14 Jul 2026 14:09:29 -0700

Proposes a lightweight VLA model leveraging deep state space models (SSMs) instead of transformers for efficient mobile manipulation. Targets resource-constrained deployment scenarios with competitive performance.

ABot-N1: Visual Language Navigation Foundation Model

Shaoqing Tan — Tue, 14 Jul 2026 03:19:00 -0700

Decouples cognition from control in a VLM-based navigation policy, delivering 35% POI arrival gains and over 92% success rates in complex indoor and outdoor scenes.

ABot-AgentOS: A General-Purpose Robotic Agent Operating System

Shaoqing Tan — Tue, 14 Jul 2026 03:14:45 -0700

Provides scene-conditioned planning, context-isolated skill execution, multi-modal memory, and self-evolution capabilities for long-horizon embodied tasks.

Trust Region Policy Distillation (TOP-D)

Shaoqing Tan — Mon, 13 Jul 2026 14:13:47 -0700

Transforms unstable on-policy distillation into a stable training paradigm via dynamic proximal teacher construction, improving sample efficiency without additional computational overhead.

HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers

Shaoqing Tan — Mon, 13 Jul 2026 14:13:05 -0700

A new humanoid whole-body control method that distills complementary teachers for agentic task-space control on humanoids. The approach enables robust whole-body coordination for complex manipulation tasks.

EVA-Client: A Unified Framework for Real-Robot Policy Iteration

Shaoqing Tan — Mon, 13 Jul 2026 05:07:06 -0700

A unified open framework for real-robot policy iteration that integrates teleoperation data collection, model training, deployment, and evaluation into a single closed-loop pipeline.

UniVR-34B: A Vision-Only Foundation Model for Physical Tasks

Shaoqing Tan — Mon, 13 Jul 2026 03:22:53 -0700

First large-scale model to learn complex physical dynamics, visual reasoning, and long-horizon planning directly from visual demonstrations without text chains; released with 310k SFT and 3k RL samples across 16 sources alongside the VR-X benchmark.

CLAP: Converting Vision-Language Models into Vision-Language-Action Models via Language-Prompted Actions

Shaoqing Tan — Mon, 13 Jul 2026 03:19:11 -0700

Converts any pretrained vision-language model into a vision-language-action model with zero architectural changes by prepending natural-language action descriptions; reaches 90.8% success on LIBERO with a 2B model after less than 6 hours of post-training.

InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action

Shaoqing Tan — Sun, 12 Jul 2026 14:07:51 -0700

InternVLA-A15 is a new Vision-Language-Action (VLA) model released by InternRobotics, targeting robotic manipulation and control tasks. The model is accompanied by a Hugging Face collection and an associated technical paper.

MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation

Shaoqing Tan — Sun, 12 Jul 2026 09:36:13 -0700

Proposes foundation world-action models targeting real-time humanoid loco-manipulation, integrating world modeling with action generation for unified locomotion and manipulation control.

On-Device Diffusion Transformer Policy for Efficient Robot Manipulation

Shaoqing Tan — Sun, 12 Jul 2026 09:31:23 -0700

LightDP introduces a framework to accelerate Diffusion Transformer-based policies for real-time, on-device deployment in robot manipulation tasks, achieving efficiency for visuomotor control without sacrificing performance. Accepted to ICCV 2025.

Robotic World Model: Neural Dynamics for Locomotion

Shaoqing Tan — Sun, 12 Jul 2026 05:09:14 -0700

A neural dynamics world model paired with model-free RL for quadruped and humanoid locomotion in IsaacLab, enabling long-horizon autoregressive prediction with policies trained in imagined rollouts that outperform model-based baselines on sim-to-real transfer.

LingBot-Vision: Self-Supervised ViT with Masked Boundary Modeling

Shaoqing Tan — Sat, 11 Jul 2026 14:07:31 -0700

A self-supervised ViT backbone pretrained with masked boundary modeling for dense spatial perception, achieving strong zero-shot results on depth estimation, segmentation, and embodied tasks.

LaMem-VLA: Latent-Memory-Native VLA Framework

Shaoqing Tan — Sat, 11 Jul 2026 14:07:00 -0700

A VLA framework that curates experience into short- and long-term memory vaults, condenses them to latent tokens, and integrates directly into VLA reasoning, evaluated on SimplerEnv and LIBERO benchmarks.

LingBot-VA 2.0: Native Video-Action Foundation Model for Robot Control

Shaoqing Tan — Sat, 11 Jul 2026 05:10:02 -0700

A native video-action foundation model pretrained from scratch with a semantic visual-action tokenizer and foresight reasoning, enabling real-time robot control at ≤150 Hz on consumer GPUs without relying on retrofitted VLMs or video generators.

GigaWorld-1: Large-Scale World Model for Robot Policy Evaluation

Shaoqing Tan — Sat, 11 Jul 2026 05:07:46 -0700

A large-scale world model trained on 12,980 hours of data and benchmarked across 324,000+ simulated rollouts, designed for comprehensive robot policy evaluation.

Learning Unified Force and Position Control for Legged Loco-Manipulation

Shaoqing Tan — Fri, 10 Jul 2026 14:14:20 -0700

Proposes a unified RL policy trained in simulation that jointly handles force and position control without force sensors, enabling compliant behaviors and improved force-aware imitation learning on quadrupedal and humanoid robots.

Robix: Unified Vision-Language Model for Robotic Reasoning and Planning

Shaoqing Tan — Fri, 10 Jul 2026 05:18:52 -0700

A single vision-language model that unifies high-level reasoning, long-horizon planning, and human-robot interaction, serving as a cognitive layer above low-level vision-language-action models.

EmbodiedOneVision (EO-1): A Unified Decoder-Only Transformer for General Robot Control

Shaoqing Tan — Thu, 09 Jul 2026 14:20:24 -0700

A 3B-parameter open-source unified decoder-only transformer for general robot control that jointly handles perception, planning, reasoning, and action via interleaved vision-text-action pretraining on a 1.5M-sample dataset. Achieves strong results across manipulation tasks and benchmarks with fully open-sourced model, code, and dataset.

mimic-video: Video-Action Models for Robot Learning

Shaoqing Tan — Thu, 09 Jul 2026 14:12:13 -0700

Introduces Video-Action Models (VAMs) that replace standard VLMs in VLAs with a pretrained internet-scale video backbone (Cosmos-Predict2) combined with a flow-matching action decoder. Achieves approximately 10× better sample efficiency on real-world pick-and-place tasks.

Lowering the Barrier: The GEM Open-Source Arm

Shaoqing Tan — Thu, 09 Jul 2026 05:14:22 -0700

An open-source, sub-$500 7-DOF 3D-printed robotic arm with 1.2 kg payload, wrist and head cameras, and full LeRobot integration, designed to lower the barrier to entry for real manipulation research.

RynnWorld-4D: A 4D Embodied World Model for Bimanual Robot Control

Shaoqing Tan — Wed, 08 Jul 2026 14:11:44 -0700

A 4D embodied world model that predicts RGB, depth, and optical flow from RGB-D input and instructions using a tri-branch diffusion architecture. Enables closed-loop bimanual robot control by bridging world prediction and policy execution.

RynnWorld-Teleop: Digital Teleoperation via World Model Rendering

Shaoqing Tan — Wed, 08 Jul 2026 14:08:55 -0700

A digital teleoperation system that uses hand-pose streams to drive a world model for real-time (40+ FPS) high-fidelity robot video rendering from a single image. Removes physical robots from the training loop entirely.

VLA-Corrector: Adaptive Action Horizons through Latent Visual Monitoring

Shaoqing Tan — Wed, 08 Jul 2026 05:14:06 -0700

A lightweight plug-in that monitors latent visual dynamics in action-chunked policies, drops stale actions on drift, and replans on-the-fly achieving +17.7 success points while keeping the VLA backbone frozen.

Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization

Shaoqing Tan — Wed, 08 Jul 2026 05:09:22 -0700

A feed-forward model that decomposes unposed images into instance-structured 3D token groups without annotations, enabling unified reconstruction, segmentation, and manipulation.

Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation

Shaoqing Tan — Tue, 07 Jul 2026 14:09:34 -0700

A proprioceptive contact feedback method leveraging motor current signals as a touch proxy to enable compliant dexterous manipulation without dedicated tactile sensors.

DART: One-Shot VLA Policy Adaptation via Weight-Space Arithmetic

Shaoqing Tan — Tue, 07 Jul 2026 05:16:14 -0700

Uses weight-space arithmetic to isolate domain shifts from task knowledge, enabling one-shot VLA policy adaptation to new cameras or embodiments.

Does VLA Even Know the Basics? Act2Answer Benchmark

Shaoqing Tan — Tue, 07 Jul 2026 05:12:02 -0700

Introduces a benchmark showing VLAs lose 20–40 points in commonsense/world knowledge versus source VLMs after robotics fine-tuning, evaluated via action-based answering.

Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding

Shaoqing Tan — Mon, 06 Jul 2026 14:07:09 -0700

Learns dexterous manipulation policies that explicitly ground actions in generative contact predictions from visuotactile observations, improving robustness on contact-rich tasks.

Freeform Preference Learning (FPL) for Robotic Manipulation

Shaoqing Tan — Mon, 06 Jul 2026 05:09:07 -0700

Introduces multi-axis preference supervision to learn dense, language-conditioned rewards across speed/precision/subtask axes without segmentation; enables compositional generalization and better long-horizon credit assignment than single-reward baselines.

Orca: The World is in Your Mind

Shaoqing Tan — Sun, 05 Jul 2026 14:09:11 -0700

Proposes a general world foundation model leveraging Next-State-Prediction to jointly generate text, images, and embodied actions within a unified framework.

Qwen-RobotNav: A Scalable Unified Navigation Model for Agentic Robotics

Shaoqing Tan — Sun, 05 Jul 2026 14:08:08 -0700

A unified 2B–8B parameter model for robot navigation tasks (VLN, ObjectNav, tracking, autonomous driving) via a configurable observation protocol, with demonstrated zero-shot deployment on real quadruped robots using agentic planners.

Scaling Robot Skills from Cheap Human Videos

Shaoqing Tan — Wed, 01 Jul 2026 14:08:01 -0700

Replaces noisy 6-DoF hand poses with relative wrist translation as a shared action space between humans and bimanual robots, enabling scalable skill acquisition from inexpensive video data. This approach outperforms full-pose baselines for robot skill learning.

ABC: An Open Behavior Cloning Stack for Bimanual Manipulation

Shaoqing Tan — Wed, 01 Jul 2026 05:16:54 -0700

Large-scale open-source framework for real-world robotic manipulation using behavior cloning, including the ABC-130K dataset with 3,500 hours and 130K+ episodes across 195 tasks. Provides hardware setups, simulators, and training recipes for Diffusion Transformers and Vision-Language-Action models.

Qwen-RobotManip: Alignment Unlocks Scale for Robotic Manipulation Foundation Models

Shaoqing Tan — Wed, 01 Jul 2026 05:12:07 -0700

Presents alignment techniques to scale robotic manipulation foundation models, building on the Qwen model family for dexterous robot control.

ASPIRE: Automated Skill Discovery for Robotics

Shaoqing Tan — Tue, 30 Jun 2026 14:11:15 -0700

Introduces the first automated system that continuously discovers, evolves, and accumulates reusable sensorimotor skills via evolutionary search over control programs, enabling compounding multi-task, sim-to-real, and cross-embodiment transfer without retraining end-to-end policies.

SERF: 4D Latent Mapping for Long-Horizon Mobile Manipulation

Shaoqing Tan — Tue, 30 Jun 2026 05:48:09 -0700

Embeds both the robot and environment into a shared 4D latent space augmented with forward-kinematics robot points, enabling a vision-language-action model to handle dynamic scenes and long-horizon memory. Outperforms image-only VLA baselines on the BEHAVIOR-1K benchmark for mobile manipulation.

ViserDex: Visual Sim-to-Real for Robust Dexterous In-Hand Reorientation

Shaoqing Tan — Tue, 30 Jun 2026 05:13:30 -0700

A single-camera sim-to-real framework that uses physically consistent 3D Gaussian Splatting augmentations to achieve zero-shot transfer of dexterous in-hand reorientation policies to an Allegro hand. The approach trains entirely on consumer hardware while maintaining high fidelity to real-world dynamics.

DexSkin: A High-Coverage, Conformable "Electronic Skin" for Robot Fingers

Shaoqing Tan — Tue, 30 Jun 2026 03:12:49 -0700

Introduces a high-coverage, conformable robotic skin hardware system designed to improve data collection and policy learning for contact-rich, dexterous manipulation tasks. The system provides rich tactile sensing coverage to enable more capable robot manipulation policies.

EBench: A Diagnostic Benchmark for Generalist Manipulation Policies

Shaoqing Tan — Tue, 30 Jun 2026 03:11:09 -0700

A CAT-scan style diagnostic benchmark for robot foundation models that evaluates policies such as π0, π0.5, and Qwen-RobotManip beyond single success rates. The benchmark is designed to distinguish genuine generalization from overfitting to demonstrations in generalist manipulation policies.

VITRA: A Foundation for Dexterous VLA via Human Video Pretraining

Shaoqing Tan — Mon, 29 Jun 2026 14:13:41 -0700

A scalable VLA pretraining pipeline that converts unstructured egocentric human videos into robot training data, trains a dexterous hand VLA, and fine-tunes on robot data, achieving strong zero-shot generalization and real-robot dexterous manipulation.

DexWM: A Dexterous Manipulation World Model from Human Videos

Shaoqing Tan — Mon, 29 Jun 2026 14:08:49 -0700

A dexterous manipulation world model pretrained on 829 hours of EgoDex human data and DROID robot data using conditioned diffusion transformers, enabling open-loop rollouts and sim-to-real transfer with minimal robot fine-tuning.

PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning

Shaoqing Tan — Mon, 29 Jun 2026 05:14:05 -0700

Introduces PoLAR, a method that factorizes latent action representations into extent and mode components to improve robot policy learning efficiency and generalization.

Continual Robot Policy Learning via Variational Neural Dynamics

Shaoqing Tan — Mon, 29 Jun 2026 05:13:48 -0700

Proposes a variational neural dynamics framework for continual robot policy learning, enabling robots to acquire new skills without forgetting previously learned ones.

PhysisForcing: Physics-Reinforced World Models for Robotic Manipulation

Shaoqing Tan — Mon, 29 Jun 2026 03:24:42 -0700

Plug-and-play training framework that enforces physical plausibility in robotic video generation models, achieving SOTA on R-Bench, PAI-Bench, and EZS-Bench. Lifts WorldArena success rate from 16% to 24% with zero extra inference cost.

Translation as a Bridging Action

Shaoqing Tan — Mon, 29 Jun 2026 03:11:02 -0700

Replaces noisy 6DoF hand poses with relative wrist translation as a shared action space between cheap human videos and bimanual robots. Scales data-efficiently and outperforms full-pose baselines on manipulation tasks.

Play2Perfect: Dexterous Play Pretraining for Precise Assembly

Shaoqing Tan — Sun, 28 Jun 2026 14:13:36 -0700

Pre-trains a dexterous hand via unstructured 'play' interactions with objects, then fine-tunes for precise assembly tasks including 0.5 mm clearance insertions and furniture screwing, achieving 33x better sample efficiency than RL from scratch.

Dexora: Open-Source VLA for High-DoF Bimanual Dexterity

Shaoqing Tan — Sun, 28 Jun 2026 14:11:01 -0700

First open-source Vision-Language-Action (VLA) model for dual-arm, dual-hand 36-DoF dexterous manipulation, trained on 100K simulated and 10K real trajectories with strong cross-embodiment transfer capabilities.

WorldVLA: Towards Autoregressive Action World Model

Shaoqing Tan — Sun, 28 Jun 2026 05:09:08 -0700

Unifies VLA and world-modeling in a single autoregressive transformer that predicts both future images and actions. Outperforms separate VLA or world models on LIBERO simulation benchmarks.

HumDex: Humanoid Dexterous Manipulation Made Easy

Shaoqing Tan — Sun, 28 Jun 2026 05:08:49 -0700

HumDex targets humanoid dexterous manipulation, aiming to simplify the development of dexterous manipulation capabilities for humanoid robots.

ForceBand: Learning Forceful Manipulation with sEMG

Shaoqing Tan — Sat, 27 Jun 2026 14:11:53 -0700

Presents an open-source, low-cost sEMG wristband framework that extracts force signals from human muscle activity in videos, enabling zero-shot human-to-robot transfer of forceful manipulation policies across any robot, camera, or environment.

In-Context World Modeling for Robotic Control

Shaoqing Tan — Sat, 27 Jun 2026 14:08:16 -0700

Introduces ICWM, a method that learns world dynamics from just seconds of a robot's self-generated interaction data, enabling zero-shot adaptation to unseen cameras and new robot morphologies without any fine-tuning.

WOLF-VLA: Vision-Language-Action for Humanoid Walking

Shaoqing Tan — Sat, 27 Jun 2026 05:15:30 -0700

Introduces a framework integrating vision-language-action models for whole-body humanoid locomotion, addressing optimal control and learning for complex bipedal behaviors. Combines VLA learning with locomotion-specific control for humanoid robots.

Motion-Focused Latent Action for Cross-Embodiment VLA from Human Videos

Shaoqing Tan — Sat, 27 Jun 2026 05:14:57 -0700

Proposes a motion-focused latent action representation for cross-embodiment vision-language-action policies learned from human videos, accepted to IROS 2026.

ManiFlow: Manipulation via Rectified Flow

Shaoqing Tan — Sat, 27 Jun 2026 03:11:30 -0700

ManiFlow is a visuomotor imitation learning policy using consistency flow matching with a DiT-X architecture that generates high-quality actions in 1–2 steps. It works across single-arm, bimanual, and humanoid platforms using RGB or point cloud inputs.

RL-100: Toward Highly Reliable Real-World Robot Reinforcement Learning

Shaoqing Tan — Sat, 27 Jun 2026 03:10:48 -0700

RL-100 demonstrates highly reliable real-world RL manipulation achieving 900/900 success rates across 7 tasks with up to 250 consecutive trials without failure. It also shows strong robustness to disturbances and zero/few-shot adaptation capabilities.

Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation

Shaoqing Tan — Fri, 26 Jun 2026 14:10:00 -0700

A video-diffusion world model trained on over 1 million manipulation episodes (3,000 hours) that includes an action model and neural simulator for closed-loop robotic manipulation control, with all code and models open-sourced.

Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning for Long-Horizon Contact-Rich Manipulation

Shaoqing Tan — Fri, 26 Jun 2026 05:12:44 -0700

Proposes a hierarchical imitation learning framework using bilateral control, subtask-level progress tracking, and keyframe memory to handle long-horizon, contact-rich manipulation tasks.

From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation

Shaoqing Tan — Fri, 26 Jun 2026 05:08:58 -0700

Uses reinforcement learning to improve process reasoning capabilities in robotic manipulation policies, shifting the model from passive observation to active critique.

ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning

Shaoqing Tan — Fri, 26 Jun 2026 03:12:15 -0700

ROVE leverages reinforcement learning to enable humanoid robots to benefit from human interventions during manipulation tasks.

ConstrainedMimic: Safe Humanoid Robot Motion Tracking

Shaoqing Tan — Fri, 26 Jun 2026 03:11:55 -0700

A control framework for safe humanoid robot motion tracking using RL policies with real-time constraint enforcement via kinematics, dynamics, and control barrier functions.

REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering

Shaoqing Tan — Thu, 25 Jun 2026 14:13:34 -0700

Introduces spatio-temporal policy learning combined with physics-guided filtering to achieve robust and extremely agile robot control.

HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching

Shaoqing Tan — Thu, 25 Jun 2026 14:08:48 -0700

Introduces a tokenization-free autoregressive policy learning framework using flow matching across scales for robotic control.

Reactive Diffusion Policy: Slow-Fast Visual-Tactile Learning for Contact-Rich Manipulation

Shaoqing Tan — Thu, 25 Jun 2026 05:15:32 -0700

Introduces a slow-fast imitation learning framework combining diffusion-based planning with reactive tactile/force feedback for contact-rich manipulation tasks. Also includes TactAR, an AR-based teleoperation system with tactile sensing.

SARM2 + SPIRAL: Multi-Task Reward Models and RL Refinement for Long-Horizon Dexterous Manipulation

Shaoqing Tan — Thu, 25 Jun 2026 05:12:28 -0700

Combines scalable autonomous reward modeling with RL-based refinement to improve vision-language-action policies on long-horizon dexterous manipulation tasks via autonomous rollouts. Demonstrates significant gains over imitation learning baselines.

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm VLA Systems

Shaoqing Tan — Thu, 25 Jun 2026 03:34:15 -0700

Introduces coordination-aware structured action modeling for dual-arm robotic systems within a VLA framework. Addresses the unique challenges of bimanual manipulation through specialized action representations.

ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation

Shaoqing Tan — Thu, 25 Jun 2026 03:31:11 -0700

Proposes interleaved vision and language reasoning for robotic manipulation within a VLA framework. Aims to improve instruction following and task performance through integrated multimodal reasoning.

Playful Agentic Robot Learning

Shaoqing Tan — Thu, 25 Jun 2026 03:22:48 -0700

Self-directed play combined with Code-as-Policy for reusable skill acquisition and downstream manipulation tasks.

Learning Unified Force and Position Control for Legged Loco-Manipulation

Shaoqing Tan — Wed, 24 Jun 2026 14:14:01 -0700

A unified RL policy for quadrupeds and humanoids that jointly handles force and position control without force sensors, enabling compliant behaviors, force-aware imitation learning, and contact-rich tasks.

Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies

Shaoqing Tan — Wed, 24 Jun 2026 14:11:02 -0700

Explores imitation learning approaches for multi-robot systems, focusing on policy coupling through sequential asymmetric imitation to enable collaborative robot behaviors.

AstraBrain-WBC 0.5: A Humanoid Robot Cerebellum Foundation Model

Shaoqing Tan — Wed, 24 Jun 2026 05:17:07 -0700

A humanoid robot 'cerebellum' foundation model trained on 20,000 hours of human motion data that demonstrates scaling laws for robot motion control and enables zero-shot execution of unseen motions on real humanoids.

SRL: Combining SLIP Model and Reinforcement Learning for Agile Robotic Jumping

Shaoqing Tan — Wed, 24 Jun 2026 05:14:53 -0700

Combines the Spring-Loaded Inverted Pendulum (SLIP) model with reinforcement learning to achieve agile jumping behaviors in robotic systems.

DataClaw0: Agentic Tailoring for Raw Multimodal Streams

Shaoqing Tan — Wed, 24 Jun 2026 03:20:11 -0700

A 9B model that filters noise from videos, GUI, and embodied data streams, reorganizing them into dense supervision via factual anchors and semantic synthesis; trained with SFT + GRPO across five domains with benchmarks.

ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining

Shaoqing Tan — Wed, 24 Jun 2026 03:16:42 -0700

Converts 6K+ hours of mixed human/robot egocentric video into robot pseudo-actions via camera-space alignment and reliability-aware loss, achieving 72.8% on RoboCasa and 91.1% on RoboTwin.

VERA: Video-to-Action World Model Policy

Shaoqing Tan — Wed, 24 Jun 2026 00:03:04 -0700

A 14B-parameter video world model that converts predicted visual futures into embodiment-agnostic actions via Jacobian inverse-dynamics, enabling zero-shot cross-robot transfer across a Panda arm and 16-DoF hand with open-sourced weights and training code.

GEN-1: Scaled Dexterous Manipulation Foundation Model

Shaoqing Tan — Wed, 24 Jun 2026 00:01:21 -0700

A dexterous manipulation foundation model trained on 500k hours of real-world bimanual data that handles deformable objects such as cardboard folding and screw packing, featuring online retry and adaptation capabilities.

Efficient Hybrid SE(3)-Equivariant Visuomotor Flow Policy via Spherical Harmonics for Robot Manipulation

Shaoqing Tan — Mon, 22 Jun 2026 05:16:39 -0700

Develops an SE(3)-equivariant flow-based visuomotor policy leveraging spherical harmonics for efficient and geometrically consistent robot manipulation.

Cortical Policy: A Dual-Stream View Transformer for Robotic Manipulation

Shaoqing Tan — Mon, 22 Jun 2026 05:10:43 -0700

Introduces a dual-stream transformer architecture inspired by cortical visual processing for learning robotic manipulation policies.

VisualClaw: A Self-Evolving Wearable Vision Agent

Shaoqing Tan — Mon, 22 Jun 2026 03:05:59 -0700

An edge-filtered video streaming agent that evolves skills from memory and runs on smart glasses, reducing API costs by 98%, accompanied by the VisualClawArena benchmark dataset.

Kairos: A Native World Model Stack for Physical AI

Shaoqing Tan — Sun, 21 Jun 2026 14:23:15 -0700

A 4B unified architecture for world understanding, generation, and action with hybrid linear attention enabling real-time edge inference across embodiments, outperforming 14B models on embodied benchmarks.

DragMesh-2: A Contact-Driven Framework for Dexterous Hand–Object Interaction

Shaoqing Tan — Sun, 21 Jun 2026 14:12:42 -0700

A framework that trains a 51-DoF dexterous hand to open drawers and doors using only physical contact without requiring tactile sensors.

Guava: A Universal Harness for Robot Manipulation

Shaoqing Tan — Sun, 21 Jun 2026 05:10:01 -0700

A 4B open-source VLA-style model trained on fewer than 2K simulation trajectories that matches closed frontier systems on real-world manipulation tasks with zero-shot generalization to novel objects and long-horizon behaviors, including failure-recovery demonstrations.

Geometric Action Model for Robot Policies

Shaoqing Tan — Sun, 21 Jun 2026 05:09:28 -0700

A new geometric action model for robot manipulation policies that focuses on structured action representations to improve policy learning and generalization.

ENPIRE: Physical AutoResearch with a Fleet of 8 Robots

Shaoqing Tan — Sun, 21 Jun 2026 03:14:25 -0700

ENPIRE demonstrates fully autonomous physical AutoResearch where Codex agents control a fleet of 8 robots overnight, self-improving through real hardware rollouts on tasks like zip-tie tying and GPU installation, while discovering physical scaling laws with built-in safety harnesses and frozen reward classifiers derived from demonstrations.

MolmoAct2: An Open Foundation Model for Real-World Robotics

Shaoqing Tan — Sun, 21 Jun 2026 00:43:08 -0700

An open VLA-style robotics foundation model featuring open weights, open dataset, open action tokenizer, and a depth-reasoning variant; designed to enable community experiments on real robots for manipulation and generalist policies.

Hy-Embodied-0.5-VLA: A Massive Bimanual Teleoperation Dataset for Vision-Language-Action

Shaoqing Tan — Mon, 15 Jun 2026 05:27:45 -0700

Released a massive bimanual robot manipulation dataset with 2,163 hours and 250K+ episodes across 70+ tasks, along with a compatible VLA model for multi-view egocentric teleop. The dataset and model are fully compatible with LeRobot v3.0.

Q-Guided Flow: Test-Time Gradient Guidance of Flow Policies

Shaoqing Tan — Sun, 14 Jun 2026 14:23:14 -0700

New framework for guided flow-matching policies that improves long-horizon robotic control and sample efficiency.

Flow Reversal Steering: Guiding Diffusion-Based Robot Policies with High-Level Reasoning

Shaoqing Tan — Sun, 14 Jun 2026 14:13:15 -0700

Introduces flow reversal steering to guide diffusion-based vision-language-action models with high-level VLM reasoning and enables RL directly in the diffusion noise space.

Test-Time Compute Scaling for Robot Policies (DIRECT)

Shaoqing Tan — Sun, 14 Jun 2026 05:26:08 -0700

Larger models + more thinking + more context improve performance on some prompts but not others; a learned router enables better performance/latency trade-offs.

LabVLA: Bringing Vision-Language-Action to the Chemistry Lab

Shaoqing Tan — Sun, 14 Jun 2026 05:16:11 -0700

RoboGenesis generates 10K+ lab scenes across 16 robot embodiments; LabVLA (Qwen3-VL + DiT flow-matching) achieves 71.1% success on LabUtopia and transfers to real Franka arms.

Humanoid-GPT: A Foundation Model for Zero-Shot Humanoid Control

Shaoqing Tan — Sat, 13 Jun 2026 14:17:57 -0700

GPT-style Transformer pretrained on 2 billion motion frames that achieves agile, generalist zero-shot control on a real Unitree G1 humanoid for tasks like soccer, dancing, and digging. Requires no fine-tuning or task-specific adaptation.

CHORUS: Decentralized Multi-Robot Collaboration with a Single Shared VLA Model

Shaoqing Tan — Sat, 13 Jun 2026 14:07:47 -0700

Finetunes a single Vision-Language-Action (VLA) foundation model so that any robot in a team can control any other. Outperforms both per-robot specialists and a monolithic centralized policy while scaling to large teams.

RISE: Self-Improving Robot Policy with Compositional World Model

Shaoqing Tan — Sat, 13 Jun 2026 05:18:47 -0700

Trains a compositional world model on real robot data to enable closed-loop policy improvement via future prediction and progress evaluation, bypassing both risky real-world RL and traditional sim-to-real gaps.

EmbodiedOneVision: Interleaved Vision-Text-Action Pretraining for General Robot Control

Shaoqing Tan — Fri, 12 Jun 2026 14:34:55 -0700

Open-source 3B unified embodied foundation model trained on 1.5M interleaved vision-text-action samples for perception, planning, and acting.

Robix: A Unified Model for Robot Interaction, Reasoning and Planning

Shaoqing Tan — Fri, 12 Jun 2026 14:15:36 -0700

A single vision-language model that unifies reasoning, task planning, and human-robot interaction for complex instructions and long-horizon tasks.

Robotic World Model: Learning to Simulate for Robust Robot Control

Shaoqing Tan — Fri, 12 Jun 2026 05:07:03 -0700

Presents a neural network-based world model for model-based reinforcement learning in robotics, focusing on sim-to-real transfer for quadrupedal and humanoid robots. Enables robust policy optimization through learned environment simulation.

AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization

Shaoqing Tan — Wed, 10 Jun 2026 14:21:45 -0700

Embodied egocentric simulation framework that controls first-person worlds with 3D human motion and customizes evolving scenes via pose-anchored views.

ArtiFixer: Few-Step Diffusion for 3D Scene Reconstruction

Shaoqing Tan — Wed, 10 Jun 2026 14:10:57 -0700

Few-step auto-regressive diffusion model that converts broken 3D reconstructions into fully realized scenes, outperforming prior methods by 1-3 dB PSNR.

Deployment-Time Memorization in Foundation-Model Agents

Shaoqing Tan — Wed, 10 Jun 2026 05:17:10 -0700

Examines memorization phenomena that occur during deployment of foundation model agents in practical applications.

Adversarial Machine Learning: Taxonomy, Threat Models, and Mitigation Strategies in Deep Neural Networks

Shaoqing Tan — Tue, 09 Jun 2026 14:08:24 -0700

Focuses on security aspects of deep neural networks, providing taxonomy and mitigation strategies for adversarial attacks.

SoCRATES: Evaluating LLM Mediators in Conflict Scenarios

Shaoqing Tan — Tue, 09 Jun 2026 05:38:49 -0700

First comprehensive framework for evaluating LLM mediators in real-time, emotional, socio-cognitive scenarios.

Unembedding Matrix as a Feature Lens: Unlocking Better Text Embeddings

Shaoqing Tan — Tue, 09 Jun 2026 05:32:33 -0700

Improves embedding quality without extra training by using the unembedding matrix as a feature lens for text embeddings.

LeanMarathon: Autonomous Formalization of Math Proofs on Erdős Problems

Shaoqing Tan — Mon, 08 Jun 2026 14:38:40 -0700

Presents an autonomous system for formalizing mathematical proofs, specifically targeting Erdős problems. Demonstrates automated proof formalization capabilities in the Lean theorem prover.

Deep Research Agents: Survey and Roadmap for Autonomous AI Research

Shaoqing Tan — Mon, 08 Jun 2026 14:26:32 -0700

Provides a comprehensive taxonomy, benchmarks, and future directions for autonomous AI research agents. Examines systematic approaches to developing agents capable of conducting independent research.

Cosmos 3: Omnimodal World Models for Physical AI

Shaoqing Tan — Sun, 07 Jun 2026 14:08:40 -0700

Omnimodal world models explicitly designed for Physical AI and robotics applications. Enables improved simulation and control for robotic systems through multimodal understanding.

Humanoid-GPT: GPT-Style Transformer for Zero-Shot Dynamic Humanoid Control

Shaoqing Tan — Sun, 07 Jun 2026 05:11:47 -0700

GPT-style Transformer trained on 2 billion motion frames enabling zero-shot dynamic humanoid control for tasks like soccer, dancing, and digging on real Unitree G1 robots without fine-tuning. Breaks the agility-generalization trade-off in humanoid robotics.

Bending Paper, Shaping Dexterity: The Robotic Origami Challenge

Shaoqing Tan — Fri, 05 Jun 2026 14:20:26 -0700

New IROS benchmark providing 500+ teleoperation episodes and physically accurate simulation assets for training policies that outperform human origami experts.

GraspGen-X: A Foundation Model for Zero-Shot 6-DoF Grasping

Shaoqing Tan — Fri, 05 Jun 2026 14:10:23 -0700

First foundation model for zero-shot grasping trained on billions of simulated grasps, enabling generalized manipulation without task-specific training.

When Does Deep RL Beat Calibrated Baselines?

Shaoqing Tan — Thu, 04 Jun 2026 05:24:54 -0700

Benchmark study examining when deep reinforcement learning outperforms calibrated baseline methods in adaptive resource control tasks.

Training Deep Networks as Random Effects: An Optimization–Inference Duality

Shaoqing Tan — Thu, 04 Jun 2026 05:10:17 -0700

Explores training dynamics of deep neural networks through a statistical lens, examining the duality between optimization and inference perspectives.

Generative Depth Supervision for Embodied Vision-Language Models

Shaoqing Tan — Tue, 02 Jun 2026 05:07:28 -0700

Vision-language model that adds generative depth prediction during pre-training for physical grounding; achieves SOTA on embodied benchiments and transfers directly to real-robot tasks.

PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation

Shaoqing Tan — Mon, 01 Jun 2026 05:12:12 -0700

Presents a 3D point-cloud-based world model trained on mixed real/sim data that enables zero-shot grasping and articulated object handling on real robots by explicitly modeling spatial structure.

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

Shaoqing Tan — Sun, 31 May 2026 16:41:21 -0700

NVIDIA Research presents LocateAnything, a unified generative grounding and detection framework based on Parallel Box Decoding (PBD). Unlike prior VLMs that serialize bounding boxes into sequential coordinate tokens, PBD treats each box as an atomic unit and predicts all coordinates in a single forward pass. This preserves intra-box geometric coherence while achieving 2.5x faster decoding throughput. The model supports diverse localization tasks including document understanding, GUI grounding, dense object detection, and OCR localization. Built on Moon-ViT vision encoder and Qwen2.5 language decoder. Trained on LocateAnything-Data with 138M language queries and 785M bounding boxes. Achieves state-of-the-art on LVIS, M6Doc, and ScreenSpot-Pro benchmarks. Models and demo available on HuggingFace.

LT2: Linear-Time Looped Transformers

Shaoqing Tan — Sun, 31 May 2026 05:21:02 -0700

Replaces quadratic softmax attention in looped architectures with linear/sparse mechanisms for iterative memory refinement, achieving parity with standard looped transformers at much lower cost.

One Learning Rate Doesn't Fit All: Layerwise Spectral Scheduling for Transformers

Shaoqing Tan — Sun, 31 May 2026 05:07:55 -0700

Shows that modern transformers are highly heterogeneous across layers and proposes layerwise learning rates based on weight spectrum shape, yielding up to 1.5× training speedup on LLaMA/GPT-style models.

SimToolReal: Procedural Tool Generation and a Universal Objective for Zero-Shot Tool Manipulation

Shaoqing Tan — Sat, 30 May 2026 14:19:08 -0700

Trains generalist policies in simulation on procedurally generated tools to move objects, enabling real-world tool use across varied shapes/sizes.

Robometer and the Future of Robotic Reward Modeling

Shaoqing Tan — Sat, 30 May 2026 14:07:00 -0700

New framework for scalable robotic reward modeling using trajectory comparisons to train general-purpose reward models.

Qwen-VLA: A Generalist Vision–Language–Action Robot Model

Shaoqing Tan — Fri, 29 May 2026 14:15:49 -0700

A single generalist VLA built on Qwen3.5-4B + 1.15B DiT flow-matching action decoder that unifies manipulation, navigation, and trajectory prediction across 11 embodiments via text-described embodiment prompts. Trained in four stages and outperforms task-specific specialists on real ALOHA and sim benchmarks without per-task fine-tuning.

EXPO-FT: Sample-Efficient Reinforcement Learning Fine-Tuning for Vision-Language-Action Models

Shaoqing Tan — Fri, 29 May 2026 05:13:54 -0700

Extends the EXPO method with real-world RL post-training for VLAs using image observations, action chunking, DAgger, and on-the-fly Q-value maximization. Achieves 30/30 success on 8 challenging manipulation tasks with only ~19 min of RL data on average.

RoboMeter: Learning Dense Rewards from Successes and Failures

Shaoqing Tan — Thu, 28 May 2026 22:00:45 -0700

RoboMeter trains dense reward models from both successful and failed robot trajectories, solving a key gap in prior methods that only learn from expert demos.

MobileGym: A Controllable, Parallel Sandbox for Mobile GUI Agents

Shaoqing Tan — Wed, 27 May 2026 05:34:07 -0700

Browser-hosted mobile environment with JSON state, deterministic judges, and 256 parallel rollouts. Reports +40.7 real-device points after GRPO training on 416 tasks for GUI agent development.

ANY2ANY: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking

Shaoqing Tan — Wed, 27 May 2026 05:19:30 -0700

Introduces a method to transfer a Unitree G1 foundation policy (Gear-Sonic) to LimX Oli/Luna humanoids using only 1% of the original compute/data. Achieves fast convergence and strong tracking performance for humanoid whole-body control.

TriSplat: Feed-Forward 3D Reconstruction with Triangulated Meshes

Shaoqing Tan — Tue, 26 May 2026 14:31:44 -0700

Outputs physics-engine-compatible triangle meshes directly from sparse, unposed images without Gaussian splatting or post-processing.

MIKASA-Robo-VLA: A Memory-Intensive Benchmark for Vision-Language-Action Robotics

Shaoqing Tan — Tue, 26 May 2026 14:11:34 -0700

Releases a benchmark suite for systematically evaluating memory in Vision-Language-Action policies on tabletop manipulation tasks.

PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation

Shaoqing Tan — Mon, 25 May 2026 14:12:29 -0700

Introduces large-scale 3D world models pretrained on diverse real-world video to enable robust robotic manipulation policies that generalize beyond simulation.

Bimanual Pegboard Manipulation: A Benchmark for Vision-Language-Action Models

Shaoqing Tan — Sun, 24 May 2026 14:14:31 -0700

New LeRobot-based bimanual pegboard manipulation dataset with 52 episodes, 30k frames, 3 camera views, and 14-DOF arms for VLA evaluation. Provides standardized benchmark for vision-language-action model assessment.

FutureSim: Replaying Real-World Events to Evaluate AI Forecasting Agents

Shaoqing Tan — Sun, 24 May 2026 05:31:47 -0700

A benchmark designed to test AI models' capabilities in making accurate 3-month future predictions.

AgentFloor: A Benchmark for Long-Horizon Agent Planning

Shaoqing Tan — Sun, 24 May 2026 05:18:22 -0700

A 30-task benchmark for evaluating long-horizon planning capabilities across 16 different AI models.

AlexNet: The Deep Convolutional Network That Transformed Vision

Shaoqing Tan — Sat, 23 May 2026 14:21:12 -0700

AlexNet paper that sparked the modern deep learning revolution through convolutional neural networks.

A Few Useful Things to Know About Machine Learning

Shaoqing Tan — Sat, 23 May 2026 14:13:24 -0700

Practical insights into ML pitfalls and best practices for machine learning practitioners.

SimToolReal: A Universal Dexterous Tool-Use Policy

Shaoqing Tan — Sat, 23 May 2026 05:26:37 -0700

Introduces an object-centric sim-to-real policy that enables zero-shot dexterous tool use on physical robots without task-specific fine-tuning. Leverages simulation data for robust real-world transfer.

Mimic-Video: Learning Physics Priors from Web-Scale Video for Robot Dexterity

Shaoqing Tan — Sat, 23 May 2026 05:13:14 -0700

Pretrains robot policies on large-scale web video to acquire dynamics and physics understanding instead of static images or VLMs. Yields faster training, better generalization, and superior dexterous manipulation results in real-world tasks.

Deep Residual Learning for Image Recognition (ResNet)

Shaoqing Tan — Sat, 23 May 2026 02:03:20 -0700

Introduced residual connections (ResNet) enabling training of very deep networks, still widely used in modern architectures.

Attention Is All You Need – The Transformer Revolution

Shaoqing Tan — Sat, 23 May 2026 01:49:20 -0700

Introduced the Transformer architecture based purely on attention mechanisms, becoming the foundation of nearly all modern large language models.

NVIDIA Cosmos: World Foundation Models for Physical AI

Shaoqing Tan — Wed, 20 May 2026 05:19:16 -0700

World foundation models for video and physics prediction with SynthID watermarking for responsible AI practices. Developed in collaboration with Google DeepMind.

LATENT: Teaching a Humanoid to Play Tennis from Imperfect Data

Shaoqing Tan — Tue, 19 May 2026 14:11:09 -0700

Introduces a three-stage pipeline that extracts a latent action space from noisy, low-quality human motion capture, then trains a high-level RL policy in simulation to compose and execute dynamic whole-body tennis skills. Achieves volleys at human-level performance on a humanoid robot.

CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models

Shaoqing Tan — Tue, 19 May 2026 05:26:22 -0700

Closed-loop framework coupling Vision-Language Models with Video Generation Models at step-level granularity. Mitigates long-horizon drift and mid-clip errors in goal-directed video reasoning for robotic planning.

World Action Models: The Next Frontier in Embodied AI

Shaoqing Tan — Tue, 19 May 2026 05:10:48 -0700

First systematic survey defining World Action Models (WAMs) as embodied foundation models that jointly predict future states and generate actions. Covers architectures, data ecosystems, and evaluation protocols.

Training a Whole-Body Control Foundation Model

Shaoqing Tan — Mon, 18 May 2026 14:26:59 -0700

Describes end-to-end learning of a foundation model for adaptive whole-body humanoid control via massive simulation variation. Combines proprioceptive perception and policy adaptation across embodiments.

DexJoCo: A Unified Benchmark for Task-Oriented Dexterous Manipulation

Shaoqing Tan — Mon, 18 May 2026 14:11:24 -0700

Releases an open-source MuJoCo-based benchmark with 11 dexterous tasks, low-cost teleoperation hardware, and 1.1K human demonstrations. Designed to evaluate and train modern VLA/robotic policies.

MMSkills: Building Multimodal Skill Libraries for Visual Agents

Shaoqing Tan — Mon, 18 May 2026 05:29:29 -0700

Skill library, demonstrations, and dataset for multi-modal robotic skill learning and manipulation tasks.

PhysBrain 1.0 VLA (TwinBrainVLA): Dual-Brain Vision-Language-Action with Physics-Grounded Learning

Shaoqing Tan — Mon, 18 May 2026 05:16:17 -0700

Introduces dual-brain fusion Vision-Language-Action model with LangForce physics-grounded training methodology.

MolmoAct2-LIBERO: An Open Vision-Language-Action Model for Robotics

Shaoqing Tan — Sun, 17 May 2026 14:24:27 -0700

Vision-Language-Action (VLA) model fine-tuned on the merged LIBERO robotics dataset (1,693 episodes, 273k+ frames) achieving 98.25% success rate on manipulation tasks. Released with both checkpoint and dataset for VLA finetuning.

SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Diffusion Transformers

Shaoqing Tan — Sun, 17 May 2026 14:12:10 -0700

A 2.6B-parameter open-source world model that generates coherent 720p, minute-long videos with precise 6-DoF camera control on a single GPU using a Hybrid Linear Diffusion Transformer + Gated DeltaNet for long-context efficiency. Targets controllable physics simulation.

WildClawBench: A Real-World, Long-Horizon Benchmark for AI Agents

Shaoqing Tan — Sun, 17 May 2026 05:24:48 -0700

New benchmark and dataset for robotic manipulation in unconstrained 'wild' environments. Includes standardized containers, leaderboards, and evaluation protocols for cross-embodiment policies.

MCP-Cosmos: Bring Your Own World Model

Shaoqing Tan — Sun, 17 May 2026 05:15:11 -0700

Introduces a latent-space world model framework that lets agents simulate state transitions and iteratively refine plans before real-world execution. Evaluated on 20+ MCP-Bench tasks with measurable gains in tool-use success.

OpenAI o1: Teaching LLMs to Think Slow and Deep

Shaoqing Tan — Sat, 16 May 2026 18:41:07 -0700

Details OpenAI's reasoning-focused o1 model and its 'long thought' approach using test-time compute scaling. Explores how extended reasoning during inference can improve model performance on complex tasks.

The Llama 3 Herd of Models

Shaoqing Tan — Sat, 16 May 2026 18:32:22 -0700

Comprehensive technical report on the Llama 3 family, covering architecture, training at scale, multimodal extensions, and real-world impact. Details the development of Meta's flagship open-source language model series.

LATENT: Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

Shaoqing Tan — Sat, 16 May 2026 18:03:21 -0700

Introduces a three-stage pipeline that extracts a latent action space from low-quality human tennis demonstrations, then trains a high-level policy in simulation via reinforcement learning. Enables dynamic whole-body humanoid tennis play with back-and-forth volleys at human level.

AnyFlow: Any-Step Video Diffusion for Predictive World Modeling

Shaoqing Tan — Thu, 14 May 2026 16:13:25 -0700

First any-step video diffusion framework using flow maps, allowing a single model to adapt to arbitrary inference budgets for scalable high-quality video generation relevant to predictive world modeling.

# Robotics: The Endgame

Shaoqing Tan — Thu, 14 May 2026 16:02:02 -0700

Technical roadmap mirroring LLM scaling: critiques VLAs, advocates video world models as second pretraining phase, introduces World Action Models (WAM), manipulation data flywheels, EgoScale with new Dexterity Scaling Law, and DreamDojo end-to-end neural physics engine for sim RL.

Claw-Eval: Toward Trustworthy and Transparent Evaluation of Autonomous Agents

Shaoqing Tan — Wed, 08 Apr 2026 07:19:18 -0700

Benchmark with 2,159 rubric items across 300 tasks using trajectory-aware grading and 3-trial Pass^3 scoring to mitigate luck. Evaluates agent reliability in real-world robotics settings.

LIBERO-Para: Paraphrase Robustness in Robotic Manipulation

Shaoqing Tan — Wed, 08 Apr 2026 07:18:01 -0700

Reveals paraphrase fragility in VLAs causing 22-52% success drops due to task misidentification. Introduces PRIDE metric weighting success by paraphrase difficulty on LIBERO benchmark manipulation tasks.

YOR: Your Own Mobile Manipulator for Generalizable Robotics

Shaoqing Tan — Tue, 07 Apr 2026 07:41:37 -0700

Low-cost mobile manipulator design and training strategies for broad generalization in real-world tasks.

EgoSim: Egocentric World Simulator for Embodied Interaction Generation

Shaoqing Tan — Tue, 07 Apr 2026 07:29:11 -0700

Closed-loop egocentric video simulator maintaining persistent 3D scene state for consistent interactions, enabling cross-embodiment transfer from human videos to robotic manipulation.

Accelerating Video World Models: From Generative Videos to Real-Time Simulators

Shaoqing Tan — Mon, 06 Apr 2026 22:17:58 -0700

Comprehensive survey taxonomizing efficient architectures/algorithms for video world models as simulators, targeting compute bottlenecks in embodied AI, autonomous driving, and games with techniques like short-window attention for real-time long-horizon prediction.

From Tokens to Thoughts: Continuous Latent Reasoning in Large Models and Robot Control

Shaoqing Tan — Mon, 06 Apr 2026 22:14:05 -0700

Curated collection of 100+ works surveying shift to continuous latent spaces in LLMs/VLMs/VLAs for improved reasoning over discrete tokens, with relevance to robotics action modeling.

CaP-X: Coding Agents for Physical eXecution

Shaoqing Tan — Mon, 06 Apr 2026 07:11:45 -0700

CaP-X is an open-source agentic robotics framework where LLMs/VLMs generate code to call perception and control APIs for execution across diverse simulated and real robots in CaP-Gym's 187 manipulation tasks. The framework includes CaP-Bench for evaluating frontier models and CaP-RL, which boosts a 7B model's success from 20% to 72% with minimal sim-to-real gap.

DoRA: Weight-Decomposed Low-Rank Adaptation

Shaoqing Tan — Sun, 05 Apr 2026 22:30:22 -0700

An upgrade over LoRA for parameter-efficient fine-tuning, enabling better performance in LLMs by decomposing weights into magnitude and direction components.

AI Model Collapse: What Happens When AI Trains on Its Own Outputs

Shaoqing Tan — Sun, 05 Apr 2026 22:15:47 -0700

Seminal work showing how training on AI-generated data leads to 'model collapse' in neural networks, with urgent implications for future scaling.

PhAIL: Benchmarking Vision-Language-Action Models on Real-World Bin-Picking

Shaoqing Tan — Sun, 05 Apr 2026 07:19:40 -0700

Real-world hardware evaluation of VLAs on blind bin-to-bin picking, achieving max 64 picks/hour across hundreds of runs, with full videos/data exposing gaps in production-scale robotic manipulation reliability.

Co-training Large Behavior Models: Data Modalities and Training Strategies for Robot Manipulation

Shaoqing Tan — Sat, 04 Apr 2026 22:42:13 -0700

Comprehensive evaluation of 89 policies showing optimal co-training practices mixing real robot data with sim/egocentric human videos to boost diversity and performance in large robotics foundation models.

HyDRA: Hybrid Memory for Dynamic Video World Models

Shaoqing Tan — Sat, 04 Apr 2026 22:31:30 -0700

Novel memory system preserving dynamic object identity and motion continuity across occlusions in video world models, addressing frozen/vanishing issues for improved predictive physics in embodied AI.

# WildWorld: Dynamic World Modeling with Actions and Explicit State

Shaoqing Tan — Sat, 04 Apr 2026 07:29:53 -0700

Massive dataset enabling dynamic world models with explicit states and actions, supporting predictive modeling for cross-embodiment robotic control.

Omni-WorldBench: Evaluating Interactive 4D World Models

Shaoqing Tan — Sat, 04 Apr 2026 07:18:25 -0700

New benchmark assessing world models on interaction tasks, pushing predictive physics and video modeling towards robotics applications with action-conditioned evaluation.

SIMART: From Static Meshes to Sim-Ready Articulated Models

Shaoqing Tan — Fri, 03 Apr 2026 22:37:59 -0700

Unified MLLM framework with Sparse 3D VQ-VAE (70% token reduction) for part-level mesh decomposition and kinematic chain prediction, enabling physics-based robotic simulation from monolithic assets.

EgoSim: An Egocentric World Simulator for Embodied Interaction

Shaoqing Tan — Fri, 03 Apr 2026 22:23:00 -0700

Closed-loop egocentric simulator persistently updating 3D scene state to generate spatially consistent interaction videos for continuous simulation, enabling cross-embodiment transfer from human videos to robotic manipulation tasks.

Digit's New Motor Cortex: Sim-to-Real RL for Whole-Body Control

Shaoqing Tan — Fri, 03 Apr 2026 07:13:57 -0700

AI-trained capabilities for new whole-body motions using mocap/teleop data and sim-to-real reinforcement learning, deployable overnight on hardware.

EgoNav: Diffusion-Based Humanoid Navigation from Human Egocentric Video

Shaoqing Tan — Thu, 02 Apr 2026 22:32:11 -0700

Diffusion-based humanoid navigation trained solely on 5 hours of human egocentric video data, enabling zero-shot deployment on Unitree G1 for complex behaviors like handling glass walls, crowds, and dynamic obstacles via 360° visual memory and hybrid trajectory sampling; upcoming release of dataset, models, and code.

CaP-X: A Code-as-Policy Framework for Robot Manipulation

Shaoqing Tan — Thu, 02 Apr 2026 22:19:27 -0700

Comprehensive open-source agentic robotics framework treating VLMs/LLMs as code-generating APIs for perception (SAM3, Molmo) and control (IK, grasping), with CaP-Gym benchmark of 187 diverse manipulation tasks (tabletop, bimanual, mobile; sim/real) and CaP-Bench evaluating 12 frontier models; demonstrates rapid RL gains (7B model from 20% to 72% success) with strong sim-to-real transfer.

Embodied Intelligence Breakthrough: Generalist AI’s GEN-1 Robots

Shaoqing Tan — Thu, 02 Apr 2026 12:58:30 -0700

We've created GEN-1, our latest milestone in scaling robot learning. We believe it to be the first general-purpose AI model that crosses a new performance threshold: mastery of simple physical tasks. It improves average success rates to 99% on tasks where previous models achieve 64%, completes tasks roughly 3x faster than state of the art, and requires only 1 hour of robot data for each of these results. GEN-1 unlocks commercial viability across a broad range of applications—and while it cannot solve all tasks today, it is a significant step towards our mission of creating generalist intelligence for the physical world.

CaP-X: LMs' First Physical Exam

Shaoqing Tan — Thu, 02 Apr 2026 12:43:57 -0700

A novel benchmark that evaluates language models on physical examination tasks, testing their ability to understand and perform clinical physical exam procedures in simulated environments. This work introduces a comprehensive evaluation framework for AI systems in medical/clinical settings.

AI Model Collapse: The Danger of Training on AI-Generated Data

Shaoqing Tan — Tue, 31 Mar 2026 07:36:21 -0700

Demonstrated that LLMs trained recursively on AI-generated data suffer model collapse, a degenerative process where they lose grasp of true data distributions. Sparked critical debates on data provenance and the importance of preserving human-generated training data.

High-Level Automated Reasoning with Qwen2.5-7B

Shaoqing Tan — Tue, 31 Mar 2026 07:35:15 -0700

Qwen2.5-7B achieved 79.6% on MATH benchmark, surpassing GPT-4o, by employing atomic reasoning actions combined with Monte Carlo Tree Search. Demonstrated that strategic reasoning architectures can enable smaller models to outperform much larger ones.

Co-Training Large Behavior Models: Multimodal Data for Robot Manipulation

Shaoqing Tan — Mon, 30 Mar 2026 22:19:22 -0700

Explores data modalities and co-training strategies to enhance large behavior models (foundation models) for improved performance in robot manipulation tasks, supporting end-to-end learning and cross-embodiment generalization.

HyDRA: Hybrid Memory for Dynamic Video World Models

Shaoqing Tan — Sun, 29 Mar 2026 22:20:50 -0700

Memory architecture preserving identity and motion continuity for out-of-view dynamic subjects, addressing frozen/vanishing issues in video world models.

DexWM: Leveraging Human Videos for Dexterous Robot World Models

Shaoqing Tan — Sun, 29 Mar 2026 22:18:43 -0700

Dataset of robot trajectories designed for training world models to learn dexterous hand-object interactions directly from human videos.

World Models in Robotics

Shaoqing Tan — Sun, 29 Mar 2026 07:14:29 -0700

Technical survey categorizing world models into action-conditioned, video-inverse dynamics, and joint world-action models (WAMs), discussing their generalization, video data leverage, and trends for closing the robotics data gap.

SIMART: Decomposing Monolithic Meshes into Sim-Ready Articulated Assets

Shaoqing Tan — Sat, 28 Mar 2026 07:21:21 -0700

Unified MLLM framework with Sparse 3D VQ-VAE that reduces tokens by 70% for efficient part-level decomposition and kinematic prediction in physics-based robotic simulations.

LeWorldModel: A Stable JEPA World Model from Pixels

Shaoqing Tan — Fri, 27 Mar 2026 22:16:24 -0700

Stable end-to-end JEPA world model trained directly from pixels using simple MSE prediction loss and SIGReg anti-collapse regularization, enabling efficient latent planning under 1 second on 15M params with emergent spatial structure outperforming prior methods.

World Models for Robots: The Next Big Leap?

Shaoqing Tan — Fri, 27 Mar 2026 07:39:49 -0700

Technical overview defining world models in robotics, their potential to solve diverse problems via video prediction, and key enablers like scale.

Harnessing Long-Running AI in Embodied Systems

Shaoqing Tan — Fri, 27 Mar 2026 00:11:43 -0700

As AI moves from quick Q&A to marathon tasks, designers grapple with continuity. This episode explores how Anthropics harness design principles translate to embodied AI - robots that need to maintain context across long-running missions.

HoMMI: Learning Whole-Body Mobile Manipulation from Human Demonstrations

Shaoqing Tan — Wed, 25 Mar 2026 22:29:11 -0700

Whole-Body Mobile Manipulation Interface (HoMMI) that learns bimanual and whole-body manipulation, long-horizon navigation, and active perception directly from egocentric human demonstrations without teleoperation.

TurboQuant: Redefining AI Efficiency with Extreme Compression

Shaoqing Tan — Wed, 25 Mar 2026 17:52:48 -0700

This episode explores TurboQuant, a revolutionary set of quantization algorithms from Google Research that redefines AI efficiency through extreme compression.

We dive deep into how TurboQuant addresses one of AI's most pressing challenges: the memory bottleneck created by high-dimensional vectors in key-value caches. The research introduces theoretically grounded quantization methods that enable massive compression for large language models and vector search engines without sacrificing performance.

Key topics covered:

The theoretical foundations of TurboQuant's quantization algorithms
How extreme compression works for LLMs and vector search engines
Impact on high-dimensional vectors and key-value cache memory bottlenecks
Performance metrics and comparisons with existing methods
Practical implications for AI deployment and efficiency

Links:
Paper: https://arxiv.org/pdf/2504.19874
Blog: https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/

DexWM: Learning Dexterous Object Manipulation from Human Videos

Shaoqing Tan — Wed, 25 Mar 2026 07:19:46 -0700

Dataset of robot trajectories designed for training world models that learn dexterous hand-object interactions from human videos, released on Hugging Face.

FlashAttention-3: Fast & Accurate Attention with Asynchrony & Low-Precision

Shaoqing Tan — Tue, 24 Mar 2026 22:54:40 -0700

Major efficiency leap for Transformer attention mechanisms, enabling faster training/inference on long sequences with low-precision compute.

When AI Trains on Its Own Output: The Model Collapse Problem

Shaoqing Tan — Tue, 24 Mar 2026 22:39:06 -0700

Warns of "model collapse" in LLMs trained on synthetic data from prior models, urging preservation of human-generated data. One of 2024's most influential papers.

MolmoBot: A Vision-Language Model for Zero-Shot Robot Manipulation

Shaoqing Tan — Tue, 24 Mar 2026 07:22:35 -0700

Vision-language model (VLM) for zero-shot robot manipulation, trained entirely in simulation without real-world data; achieves 79.2% success rate on real-world tabletop tasks, outperforming π₀.₅ baseline at 39.2%.

LeWorldModel: Stable End-to-End JEPA from Pixels

Shaoqing Tan — Tue, 24 Mar 2026 01:12:52 -0700

A stable end-to-end Joint Embedding Predictive Architecture (JEPA) trained directly from pixels that enables robust world modeling for embodied AI systems.

EgoVerse: An Egocentric Data Ecosystem for Scaling Robot Learning

Shaoqing Tan — Mon, 23 Mar 2026 22:18:26 -0700

Ecosystem with over 1300 hours of egocentric human video data spanning 240 scenes and 2000+ tasks, designed for scalable robot policy training via behavior cloning; includes cloud infrastructure, data viewer, and human-to-robot transfer algorithms to enable cross-embodiment learning without teleoperation.

HSImul3R: Physics-Driven Reconstruction of Human–Scene Interactions

Shaoqing Tan — Mon, 23 Mar 2026 22:15:11 -0700

Physics-in-the-loop bi-directional optimization pipeline reconstructing stable, simulation-ready 3D human-scene interactions from casual videos, deployable directly to humanoid robots for world modeling and manipulation.

MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation

Shaoqing Tan — Mon, 23 Mar 2026 11:48:08 -0700

Open-source suite of large-scale simulation environments and benchmarks designed for advancing end-to-end learning in robot navigation and manipulation across multiple embodiments.

DreamZero: World Action Models Are Zero-Shot Policies

Shaoqing Tan — Mon, 23 Mar 2026 11:36:32 -0700

Introduces World Action Models (WAMs), a family of 14B-parameter autoregressive diffusion models that jointly predict video and robotic actions to enable zero-shot generalization across manipulation tasks, outperforming fine-tuned Vision-Language-Action models on benchmarks like MolmoSpaces and RoboArena.

Kinema4D: A 4D Generative Simulator for Embodied AI

Shaoqing Tan — Sun, 22 Mar 2026 19:16:00 -0700

An action-conditioned 4D generative robotic simulator that disentangles precise kinematic control from environmental dynamics, facilitating physically-plausible simulations of complex robot-world interactions for training and world modeling.

VEGA-3D: Teaching multimodal LLMs spatial reasoning through video generation

Shaoqing Tan — Sun, 22 Mar 2026 19:02:22 -0700

A plug-and-play framework extracts implicit 3D priors from video diffusion models to enhance multimodal LLMs with spatial reasoning capabilities, enabling improved geometric scene understanding and embodied decision-making without explicit 3D supervision.