Embodied AI 101

Taming the Flying Tip: Constrained MPC for Dynamic Deformable Linear Objects

Shaoqing Tan — Tue, 25 Aug 2026 05:10:00 -0700

Deformable linear objects (DLOs) exhibit highly nonlinear dynamic behavior, complicating their control during high-speed maneuvers. Furthermore, the lack of a generic spatial representation and the difficulty of real-time state estimation...

Replay Is Not Reality: Embodied RL When Both Policy and Physics Shift

Shaoqing Tan — Tue, 25 Aug 2026 05:08:31 -0700

Abstract Embodied agents must continuously adapt to the physical world using interaction data collected across varying timescales, controllers, and environmental conditions. However, standard reinforcement learning assumes stationary dynamics and on-policy data collection, which breaks down when agents must learn from heterogeneous data sources with non-stationary dynamics and off-policy behavior.

PPO Beyond the Simulator: What a TurtleBot3 Mapless Navigation Study Really Demonstrates

Shaoqing Tan — Mon, 24 Aug 2026 05:20:46 -0700

Objective: In the field of mobile robotics, autonomous navigation in dynamic environments is one of the most challenging tasks in these environments: traditional methods based on pre-mapping and geometric planning...

VLAs in the Wild: What 142 Papers Really Say About Generalist Robot Intelligence

Shaoqing Tan — Sun, 23 Aug 2026 05:08:41 -0700

Generalist robots need to perform diverse tasks while operating in dynamic, uncertain, and unstructured environments, often around human beings. Vision-language-action (VLA) models have recently emerged as a promising and flexible approach to enabling generalist robot behavior. This is a systematic literature review of VLA models for generalist robots.

VEDA and the Moment Vision Stops Being Enough

Shaoqing Tan — Sat, 22 Aug 2026 05:11:22 -0700

Industrial humanoid robots are increasingly expected to execute high-precision manipulation tasks, such as assembly, welding, and material handling, under dynamic and contact-rich working conditions.

Three Ways to Buy Back Robot Data: Augment the Demonstration, Import the Semantics, Benchmark the Physics

Shaoqing Tan — Fri, 21 Aug 2026 05:08:59 -0700

Abstract Robots have the potential to assist people in homes, hospitals, warehouses, and factories, but today's systems still struggle to learn robust manipulation skills from limited data, especially in contact-rich tasks. This paper explores data-efficient robot learning approaches for contact-rich manipulation.

UI-Mate: Show the Desktop Agent Once, Then Let It Replan

Shaoqing Tan — Thu, 20 Aug 2026 05:21:49 -0700

A 27B model that achieves one-shot doubling of strict success rates on long-horizon office automation tasks via demonstration-conditioned policy learning.

The Frozen VLM That Learns Anyway: Inside Spatial Memory Agent

Shaoqing Tan — Thu, 20 Aug 2026 05:09:24 -0700

A parameter-update-free framework that distills verified experience into transferable spatial memory for frozen VLMs, achieving consistent gains across 4 VLMs and 5 spatial reasoning benchmarks.

The LLM Proposes, the Planner Disposes: VLA-SP for Verifiable Robot Manipulation

Shaoqing Tan — Wed, 19 Aug 2026 05:14:39 -0700

Abstract Cross-modal foundation models are increasingly used for robotic task understanding and planning. However, connecting multimodal observations and natural language instructions to symbolic planners and executable robot actions remains a challenge. This paper addresses long-horizon robotic manipulation tasks using LLM-driven symbolic planning.

When Vision Cannot Know: MulPlanLM Lets Robot Planners Lift, Weigh, and Decide

Shaoqing Tan — Wed, 19 Aug 2026 05:12:55 -0700

MulPlanLM: multimodal robotic task planning with vision-language models and physical feedback

Reading Trust in Motion: A Multimodal Perception Stack for Industrial HRC

Shaoqing Tan — Tue, 18 Aug 2026 05:16:33 -0700

Abstract Industry 5.0 promotes collaborative manufacturing environments where humans and robots work together, leveraging their complementary strengths to improve productivity and safety. In such settings, trust is fundamental to ensuring effective collaboration between humans and robots. This paper presents a multi-modal model for trust evaluation in industrial human-robot collaboration settings.

Two MPCs, One Robot: Bézier Curves for Fast, Compliant Mobile Manipulation

Shaoqing Tan — Tue, 18 Aug 2026 05:13:59 -0700

Dual-arm mobile manipulators can transport and manipulate large objects with simple end-effectors. Interacting with dynamic environments subject to strict safety and compliance requirements, achieving whole-body motion planning online while maintaining compliance is challenging. This paper presents an efficient whole-body model predictive control approach for online compliant dual-arm mobile manipulation.

The Reward Behind the Gait: Learning Transferable Locomotion from a Few Stick-Insect Steps

Shaoqing Tan — Mon, 17 Aug 2026 05:20:35 -0700

Insect locomotion exhibits remarkable adaptability and flexibility despite the limited scale of its nervous system. This paper extracts and models underlying leg coordination principles via adversarial inverse reinforcement learning, enabling transferable robot locomotion from limited biological data.

Beyond Looking Human: Bio-Functional Robot Design for Cross-Scale Dexterity and Tool Use

Shaoqing Tan — Mon, 17 Aug 2026 05:16:18 -0700

Dexterous hands and robotic manipulators as physical interfaces for diverse environments. Proposes bio-functional mimicry beyond structural biomimicry, enabling cross-scale manipulation and limb-tool integration for versatile robotic tasks.

LDA-1B and the Case for Never Throwing Robot Data Away

Shaoqing Tan — Sun, 16 Aug 2026 05:07:35 -0700

Scales a latent dynamics action model to 1B parameters by ingesting diverse embodied datasets into a unified training pipeline for generalist robot policies.

Flex-π: One Robot Policy, Three Futures, Fifty-Six Ways to Run It

Shaoqing Tan — Wed, 12 Aug 2026 14:08:45 -0700

Jointly predicts future RGB frames, 3D pointmaps, DINO semantics, and actions within a unified world-action model. Can be deployed flexibly as a VLA, full world-action model, or hybrid from a single checkpoint.

Imagined Motion, Grounded Geometry: How Rest2Art Makes Closed Objects Articulate

Shaoqing Tan — Wed, 12 Aug 2026 14:07:41 -0700

An ECCV 2026 paper that converts a single static closed-state image into a simulation-ready articulated asset using video diffusion for joint hypothesis generation, requiring no motion data. Enables articulated object reconstruction without dynamic observations.

ComBodied Agents: When the Human Becomes the World Model

Shaoqing Tan — Wed, 12 Aug 2026 05:12:03 -0700

Proposes a new agent architecture that unifies digital and physical (embodied) action spaces, supporting human-state trajectories with explicit handling of consent, uncertainty, and reversibility for safer human-robot interaction.

A Sphere Every Robot Hand Can Speak

Shaoqing Tan — Tue, 11 Aug 2026 14:08:14 -0700

Introduces a unified hand action space that enables seamless cross-embodiment transfer for dexterous manipulation policies across different robot hands.

The Soft Gripper Pareto Frontier: Adaptability Without Surrendering Payload

Shaoqing Tan — Tue, 11 Aug 2026 05:10:20 -0700

Soft gripper systems have inherent flexibility and remarkable interactivity security. However, balancing load-bearing capacity, deformation, complexity, and adaptability in a single gripper remains a substantial challenge.

Dyna-2's Million-Hour Bet: When Human Video Starts Improving Robots

Shaoqing Tan — Tue, 11 Aug 2026 05:08:30 -0700

Pre-trained on 1 million hours of human video, Dyna-2 demonstrates scaling laws for world-action models across four orders of magnitude on human data, with emergent scaling on unseen robot data and joint video + action objectives enabling cross-embodiment transfer.

Think Deeply Only When It Matters: DySL-VLA and Action-Aware Dynamic Depth

Shaoqing Tan — Mon, 10 Aug 2026 14:10:38 -0700

Vision-Language-Action (VLA) models have shown remarkable success in robotic tasks like manipulation by fusing a language model's reasoning with a vision model's 3D understanding. However, their high computational cost remains...

The 100,000-Hour Bet: How Xiaomi-Robotics-1 Tries to Make Robot Scaling Real

Shaoqing Tan — Mon, 10 Aug 2026 14:09:24 -0700

We present Xiaomi-Robotics-1, a foundational vision-language-action (VLA) model capable of (1) following diverse language instructions to perform a wide range of mobile manipulation tasks in unseen environments out-of-the-box, and (2) rapidly adapting to new tasks with minimal demonstrations through efficient fine-tuning.

Reason First, Move Second: A Modular Alternative to End-to-End Robot Policies

Shaoqing Tan — Mon, 10 Aug 2026 05:10:58 -0700

Explicit task reasoning empowering robotic manipulation

From BIM to the Build: A Multimodal Agent for Conversational Construction Robotics

Shaoqing Tan — Mon, 10 Aug 2026 05:10:32 -0700

A multimodal foundation model-enabled agent for human–robot collaboration in construction

The Twin Watches, the QP Dodges: Predictive Safety for a UR10e

Shaoqing Tan — Sun, 09 Aug 2026 14:08:23 -0700

Safe human–robot collaboration remains a critical challenge in manufacturing. Traditional safety approaches, such as cages and proximity sensors, are often insufficient for dynamic human interaction. This paper presents a digital twin approach using deep learning for collision avoidance in industrial collaborative robot manipulation.

One Shot, Twenty Screws: Robot Vision for the Dirty Reality of Appliance Recycling

Shaoqing Tan — Sun, 09 Aug 2026 14:08:02 -0700

Industrial-grade robust robot vision system for screw detection and removal under uneven conditions.

Better Geometry at the Neural–Symbolic Boundary: DAIoU for Robot Manipulation

Shaoqing Tan — Sun, 09 Aug 2026 05:10:53 -0700

Distance–area–IoU fusion loss for better neuro-symbolic robot manipulation

REL Hand: A Mechanical Shortcut to More Human-Like Humanoid Dexterity

Shaoqing Tan — Sun, 09 Aug 2026 05:10:30 -0700

Design and evaluation of a tendon-and-linkage hybrid-driven humanoid dexterous hand

When the Appliance Becomes the API: Part–Function–State Models for Robot Planning

Shaoqing Tan — Sat, 08 Aug 2026 14:09:13 -0700

Appliances Describe Themselves: Part–Function–State Modeling for Appliance Manipulation Planning

CISMG-Nav: When a Navigation Agent Stops Forgetting the Building

Shaoqing Tan — Sat, 08 Aug 2026 05:12:12 -0700

CISMG-Nav: A Cross-Task Incremental Semantic Memory Graph-Driven Vision-and-Language Navigation Framework

ReaDy-Go: Teaching Robots to Dodge Photorealistic People in Gaussian-Splat Digital Twins

Shaoqing Tan — Sat, 08 Aug 2026 05:10:12 -0700

ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation With Moving Obstacles

The Action Is Missing: Turning Human Video into Robot Control

Shaoqing Tan — Fri, 07 Aug 2026 14:07:43 -0700

Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models. However, most existing approaches rely on large collections of robot demonstrations, which are costly. This survey covers scalable vision-language-action learning using human-centric data, particularly human videos, as an alternative to expensive robot demonstrations.

Look Where You Mean: Gaze2Act Gives VLAs a Live Intent Channel

Shaoqing Tan — Fri, 07 Aug 2026 05:41:28 -0700

Vision-Language-Action (VLA) models have recently shown strong potential for robot learning by following language instructions. However, in practice, language alone is often insufficient to precisely convey human intent.

HARP-VLA: Align the Eyes Before the Actions

Shaoqing Tan — Fri, 07 Aug 2026 05:17:16 -0700

Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions.

Make Geometry the Policy: GAM's Bet on 3D-Native Robot Control

Shaoqing Tan — Thu, 06 Aug 2026 14:08:33 -0700

Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world. Recent VLAs and video world-action models lack explicit 3D geometric reasoning.

π0.7: When a Robot Foundation Model Learns How to Behave

Shaoqing Tan — Thu, 06 Aug 2026 14:08:20 -0700

We present a new robotic foundation model, π0.7, that can enable strong out-of-the-box performance in a wide range of scenarios, following diverse language instructions in unseen environments.

EgoHumanoid: Turning Human First-Person Experience into Humanoid Loco-Manipulation

Shaoqing Tan — Thu, 06 Aug 2026 05:06:51 -0700

Introduces a dataset and method for learning loco-manipulation policies directly from egocentric human videos without any robot data collection, enabling real-world humanoid deployment in unstructured environments.

CycleRL: Teaching a Bicycle to Balance in Simulation—and Trusting It on Asphalt

Shaoqing Tan — Wed, 05 Aug 2026 14:09:42 -0700

CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control

Feeling the World Without a Force Sensor: UniFP for Legged Loco-Manipulation

Shaoqing Tan — Wed, 05 Aug 2026 14:08:36 -0700

Proposes a unified RL policy that jointly learns force and position control in simulation without force sensors, enabling compliant behaviors and force-aware imitation learning. Includes a force estimator trained as an auxiliary task and integration with Diffusion Policy for contact-rich tasks on quadrupeds and humanoids.

The Robot Needs a Memory: Multi-Agent Reasoning Inside a Digital Twin

Shaoqing Tan — Wed, 05 Aug 2026 05:13:34 -0700

Digital Twin-enabled adaptive robotics: multi-agent reasoning over language, vision, and structured database

Language Is Not Just a Prompt: Four Ways Robots Turn Words into Action

Shaoqing Tan — Wed, 05 Aug 2026 05:12:40 -0700

Language-conditioned robot manipulation is an emerging field aimed at enabling seamless communication and cooperation between humans and robotic agents by teaching robots to comprehend and execute instructions conveyed in natural language.

Becoming the Path: Planning Motion, Shape, and Stiffness Together

Shaoqing Tan — Wed, 05 Aug 2026 03:12:44 -0700

Autonomous operation in confined spaces demands robots that can simultaneously navigate tight passages and manipulate objects. However, conventional mobile manipulators are often too large, while compliant soft manipulators are typically too weak.

Meaning, Motion, and Two Hands: A Task Graph That Actually Plans

Shaoqing Tan — Wed, 05 Aug 2026 03:09:56 -0700

Semantic–Geometric Task Representations for Bimanual Manipulation From Human Demonstrations to Robot Action Planning

Safety Before Failure: Giving Robot Plans a One-Second Reflex

Shaoqing Tan — Tue, 04 Aug 2026 14:10:05 -0700

Robots that execute language-conditioned tasks in dynamic environments often rely on feedback only after an action has failed, which can be insufficient when failures involve collisions or workspace conflicts.

VL-GRiP3 and the Case for Keeping VLMs Away from the Motor Loop

Shaoqing Tan — Tue, 04 Aug 2026 14:09:46 -0700

VL-GRiP3: A hierarchical pipeline leveraging vision-language models for autonomous robotic 3D grasping

Think Late, Act Now: TIC-VLA Turns Reasoning Latency into a Control Variable

Shaoqing Tan — Tue, 04 Aug 2026 05:09:18 -0700

Robots in dynamic, human-centric environments must follow language instructions while maintaining real-time reactive control. Vision-language-action (VLA) models offer a promising framework, but they assume temporally aligned reasoning and control.

One Policy, Many Bodies: Inside Qwen-VLA's Generalist Robot Model

Shaoqing Tan — Tue, 04 Aug 2026 05:08:53 -0700

Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments.

VLAbot: A Human-in-the-Loop Control Plane for Long-Horizon Robot Assembly

Shaoqing Tan — Tue, 04 Aug 2026 03:13:39 -0700

VLAbot: A human Vision–Language–Action models interaction framework for robotic assembly

Teaching Isaac Sim to Feel: A Real-Time Digital Twin for Capacitive Touch

Shaoqing Tan — Tue, 04 Aug 2026 03:12:11 -0700

With advances in robotic manipulation in recent years, tactile sensing has become increasingly important in scenarios where visual information is unreliable or insufficient.

Stop Starting From Noise: Action-to-Action Flow Matching for Fast Robot Control

Shaoqing Tan — Mon, 03 Aug 2026 14:08:02 -0700

Proposes a new flow-matching approach for robot action generation that improves efficiency in learning and repurposing generalist policies. Targets improved scalability and adaptability of robot action models.

ACE-Data-0: Turning a Home into a Multimodal Robot-Learning Instrument

Shaoqing Tan — Mon, 03 Aug 2026 05:19:45 -0700

An embodied data engine releasing 150 hours of synchronized egocentric video, full-body motion, hand tracking, object trajectories, audio, and tactile signals from everyday household activities.

Before the Robot Feels It: How N₀-VTLA Predicts Contact to Control It

Shaoqing Tan — Mon, 03 Aug 2026 05:17:21 -0700

A vision-tactile-language-action model that predicts latent tactile tokens for fine-grained contact-rich manipulation, achieving wins on all 9 real-robot tasks and 63.8% on 20 simulation tasks.

Metis Gives Foundation Models a Native, Mutable Memory

Shaoqing Tan — Mon, 03 Aug 2026 03:10:26 -0700

Equips foundation models with persistent, dynamically evolving native memory state and gradient-free online updates, enabling continual adaptation without retraining.

TurboVLA and the Case for Taking the LLM Out of the Servo Loop

Shaoqing Tan — Mon, 03 Aug 2026 03:08:47 -0700

Achieves high-speed inference for vision-language-action policies at 32 Hz on an RTX 4090 with less than 1 GB VRAM using a 0.2B parameter model, enabling real-time VLA deployment on consumer hardware.

Catching on Optical Expansion: Inside Pixel2Catch's Monocular Sim-to-Real Pipeline

Shaoqing Tan — Sun, 02 Aug 2026 14:09:45 -0700

Pixel2Catch: Multi-Agent Sim-to-Real Transfer for Agile Manipulation With a Single RGB Camera

Calibrate the Robot, Randomize the World: PACE for Legged Sim-to-Real

Shaoqing Tan — Sun, 02 Aug 2026 14:07:58 -0700

Legged robots must achieve both robust locomotion and energy efficiency to be practical in real-world environments. Yet controllers trained in simulation often fail to transfer reliably, and most existing approaches are limited to specific robot morphologies. This paper presents a systematic framework for sim-to-real transfer across diverse legged robot platforms.

HumanCLAW: The Model Sees the Chair but Loses the Body

Shaoqing Tan — Sun, 02 Aug 2026 03:09:15 -0700

Introduces a benchmark with 1,218 embodied episodes that decouples high-level VLM action decisions from low-level motor control; no current VLM solves the tasks, with the best achieving only 16.8% success, highlighting lack of embodied self-awareness.

Get Up, Adapt, Survive: Recovery Skills for Small Quadrupeds

Shaoqing Tan — Sat, 01 Aug 2026 14:08:06 -0700

Real-to-Sim-to-Real: Learning Agile and Robust Recovery Skills With Terrain Imagination via Adversarial Imitation

A Patch Only Needs a Glimpse: Breaking VLA Robots Under Partial Observability

Shaoqing Tan — Sat, 01 Aug 2026 14:07:12 -0700

Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics

A Thousand Tasks, One Day, and the Case Against Learning Everything End to End

Shaoqing Tan — Sat, 01 Aug 2026 05:21:06 -0700

Presents a scalable approach to robot task learning enabling acquisition of a large number of tasks within a single day. Details were discussed in a podcast episode released July 30, 2026.

HUG: Turning Everyday Human Grasps Into a Robot-Ready Prior

Shaoqing Tan — Sat, 01 Aug 2026 05:14:41 -0700

Collects 1M frames (27.8 hours) of egocentric human grasping video and trains a flow-matching model to predict and retarget hand poses to robots. Achieves large zero-shot gains on diverse everyday grasping tasks without requiring robot-specific training data.

SkillPlug: Teaching Robot Policies to Reuse the Moves Between Tasks

Shaoqing Tan — Sat, 01 Aug 2026 03:12:26 -0700

SkillPlug: Unsupervised Skill Mining for Few-Shot Adaptation in Robotic Manipulation

Expand Before You Compress: TEAM-VLA's Training-Free Shortcut to Faster Robot Policies

Shaoqing Tan — Sat, 01 Aug 2026 03:09:27 -0700

Token Expand-Merge: Training-Free Token Compression for Vision-Language-Action Models

One Policy, Three Bodies, Zero Messages: Inside CHORUS

Shaoqing Tan — Wed, 29 Jul 2026 14:12:07 -0700

Trains a single VLA policy that enables multiple heterogeneous robots to collaborate on complex manipulation tasks using only local observations and robot IDs, avoiding centralized controllers.

Anchor the Semantics, Align the Actions: A Better Way to Fine-Tune VLAs

Shaoqing Tan — Wed, 29 Jul 2026 14:10:43 -0700

Proposes a VLA fine-tuning technique that preserves pretrained VLM representations while aligning language to actions, raising real-robot generalization success from 28% to 54%.

Learning the Rule, Not the Gait: Stick-Insect Behavior as a Portable Robot Objective

Shaoqing Tan — Wed, 29 Jul 2026 05:14:23 -0700

Insect locomotion exhibits remarkable adaptability and flexibility despite the limited scale of its nervous system. However, the underlying principles that govern leg coordination remain difficult to extract and model. This paper presents an adversarial inverse reinforcement learning approach to infer transferable locomotor principles from limited insect behavioral data, enabling robust robot locomotion.

Two Brains Under Pressure: How UnderwaterVLA Connects Language Models to AUV Control

Shaoqing Tan — Wed, 29 Jul 2026 05:13:33 -0700

The UnderwaterVLA dual-brain vision-language-action architecture enables robust autonomous underwater navigation

From Demo to Duty: RL-100's Real-World Reinforcement Learning Flywheel

Shaoqing Tan — Tue, 28 Jul 2026 18:31:25 -0700

Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass skilled human operators. We present a real-world reinforcement learning (RL) framework, RL-100, for achieving performant robotic manipulation through real-world RL training.

Distilling a Swarm into 158 Million Parameters: Inside MiniUAV-VLA

Shaoqing Tan — Tue, 28 Jul 2026 18:30:26 -0700

Coordinating multiple unmanned aerial vehicles (UAVs) for cooperative missions requires agents that perceive their environment, reason about objectives, and generate joint actions. Vision–language–action (VLA) models unify these capabilities but lack efficiency for real-world multi-UAV deployment. This episode explores MiniUAV-VLA, a compact 158M-parameter model that uses MARL expert distillation to enable cooperative multi-UAV search and elimination missions.

Imagining Locomotion: Learning a Neural World Model for Legged Robots

Shaoqing Tan — Thu, 23 Jul 2026 05:17:17 -0700

A neural dynamics world model paired with model-free RL policies for quadruped and humanoid locomotion in IsaacLab, enabling long-horizon autoregressive prediction and imagined rollouts that outperform pure model-based RL in prediction accuracy, policy learning, and sim-to-real transfer.

One Model to See, Plan, and Act: Introducing EO-1 for Embodied AI

Shaoqing Tan — Thu, 23 Jul 2026 03:11:34 -0700

A 3B parameter unified decoder-only transformer that interleaves vision, text, and action tokens for perception, planning, reasoning, and control in a single model. Trained on the 1.5M-sample EO-1.5M dataset with strong results across manipulation tasks and benchmarks.

Video-Action Models for Robot Learning

Shaoqing Tan — Thu, 23 Jul 2026 03:09:32 -0700

Introduces Video-Action Models (VAMs) that leverage pretrained internet-scale video models such as Cosmos-Predict2 as backbones instead of VLMs, paired with a flow-matching action decoder. Claims approximately 10x sample efficiency gains over standard vision-language-action models on real-world pick-and-place tasks.

FastDSAC: Unlocking the Potential of Maximum Entropy RL in High-Dimensional Humanoid Control

Shaoqing Tan — Wed, 22 Jul 2026 14:18:44 -0700

Introduces FastDSAC, a method that scales maximum entropy reinforcement learning to high-dimensional humanoid control tasks, improving sample efficiency and policy performance.

A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation

Shaoqing Tan — Wed, 22 Jul 2026 14:16:14 -0700

Proposes a minimalist pipeline using retargeting-guided RL for dexterous manipulation in humanoid robots. Focuses on simplifying the training recipe while maintaining strong manipulation performance.

DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation

Shaoqing Tan — Wed, 22 Jul 2026 03:22:38 -0700

DexNDM bridges the sim-to-real gap for stable in-hand rotation of complex objects, enabling learning from biased real-world data without requiring any successful demonstrations.

Learning Multi-Modal Trajectory Policies for Data-Efficient Robotic Manipulation

Shaoqing Tan — Wed, 22 Jul 2026 03:15:37 -0700

Proposes learning multi-modal trajectory policies to improve data efficiency in robotic manipulation tasks.

ACE: Agentic Control for Embodied Manipulation via Zero-shot Workflow Reasoning

Shaoqing Tan — Tue, 21 Jul 2026 05:17:04 -0700

A zero-shot workflow reasoning approach for agentic control of embodied manipulation, enabling robots to perform complex tasks without task-specific training.

Let the Body Follow: Coupled Egocentric Control for Whole-Body Robot Teleoperation

Shaoqing Tan — Tue, 21 Jul 2026 05:12:31 -0700

A coupled egocentric control approach for whole-body robot teleoperation that coordinates body and limb movements from an egocentric perspective, submitted to Humanoids 2026.

GigaWorld-Policy-0.5: An Efficient Mixture-of-Transformers Policy for Real-Time Robot Control

Shaoqing Tan — Mon, 20 Jul 2026 14:09:20 -0700

A Mixture-of-Transformers robot policy that achieves 85 ms inference on RTX 4090 by separating visual dynamics from action generation. Demonstrates efficient real-time robot policy execution through architectural decomposition.

Xiaomi-Robotics-1: A Scalable Vision-Language-Action Foundation Model for Mobile Manipulation

Shaoqing Tan — Mon, 20 Jul 2026 05:06:36 -0700

A scalable vision-language-action (VLA) foundation model pretrained on over 100k hours of real-world manipulation trajectories, enabling out-of-the-box mobile manipulation capabilities.

Local Policies Enable Zero-shot Long-Horizon Manipulation

Shaoqing Tan — Sun, 19 Jul 2026 14:16:30 -0700

Proposes training local policies in simulation that transfer zero-shot to real-world long-horizon robotic manipulation tasks. Addresses the challenge of generalizing learned behaviors across extended task sequences.

Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids

Shaoqing Tan — Sun, 19 Jul 2026 14:10:54 -0700

Trains perception-driven dexterous manipulation policies in simulation for zero-shot real-world transfer on humanoid robots. Focuses on bridging the sim-to-real gap for vision-based dexterous tasks.

UniTracker: A Universal Motion Tracking Framework for Humanoid Robots

Shaoqing Tan — Sun, 19 Jul 2026 05:18:06 -0700

A framework enabling humanoid robots to execute diverse motions within physical limits, with video demonstrations and open-source code. Targets generalizable motion tracking across varied humanoid morphologies.

ViTacFormer: Dexterous Manipulation via Active Vision and High-Resolution Touch Sensing

Shaoqing Tan — Sun, 19 Jul 2026 05:12:38 -0700

A pipeline combining active vision and high-resolution tactile sensing for dexterous manipulation on high-DoF robot hands, achieving approximately 2.5 minutes of continuous autonomous control on real-world tasks such as food preparation.

Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Language

Shaoqing Tan — Sun, 19 Jul 2026 03:10:22 -0700

Uses language and demonstrations to learn which aspects of behavior matter for reward in imitation learning, enabling 5× faster learning by avoiding copying irrelevant motion details. Accepted at ICRA 2026.

FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space

Shaoqing Tan — Sun, 19 Jul 2026 03:09:36 -0700

Steers frozen robot foundation models including VLAs, diffusion policies, and world models using human corrections via action inversion in latent space without fine-tuning the base policy. Outperforms supervised fine-tuning and latent RL with only 5–20 human interventions.

RoboTTT: End-to-End Robotic Assembly with Test-Time Training

Shaoqing Tan — Sat, 18 Jul 2026 05:13:30 -0700

Presents a single end-to-end policy that performs precise, unscripted assembly of complex objects, measuring every grasp and alignment without speed-ups or human intervention.

Xiaomi-Robotics-U0: A 38B World-Foundation Model for Unified Embodied Perception and Synthesis

Shaoqing Tan — Sat, 18 Jul 2026 05:07:10 -0700

38B autoregressive world foundation model unifying text-to-image, multi-view scene generation, embodied transfer, and video generation; ranks #1 on World Arena.

SpatialPoint: Spatial-Aware Point Prediction for Embodied Localization

Shaoqing Tan — Sat, 18 Jul 2026 03:19:51 -0700

A spatial-aware point prediction method for embodied localization tasks. Addresses grounding and spatial reasoning in embodied AI settings.

Multi-AUV Scene-Adaptive Embodied Intelligence for Multi-Target Tracking

Shaoqing Tan — Sat, 18 Jul 2026 03:19:44 -0700

An embodied AI framework for multi-AUV (Autonomous Underwater Vehicle) multi-target tracking in ad-hoc underwater networks. Incorporates scene-adaptive policies for dynamic underwater environments.

Speedup Patch: Learning a Plug-and-Play Policy to Accelerate Embodied Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 14:09:14 -0700

Proposes a plug-and-play policy module designed to accelerate existing embodied manipulation policies without retraining them from scratch.

R3DP: Real-Time 3D-Aware Policy for Embodied Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 14:08:01 -0700

Introduces a real-time 3D-aware policy framework for embodied manipulation tasks, enabling spatially grounded action generation.

VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 05:11:30 -0700

Presents a vision-language-action model grounded in 3D Gaussian representations that integrates geometric and semantic awareness for improved robotic manipulation.

SKooP: Symmetric Koopman Predictions for Legged Robot Reinforcement Learning

Shaoqing Tan — Fri, 17 Jul 2026 05:10:38 -0700

Leverages symmetric Koopman operator predictions within a reinforcement learning framework to achieve faster training and better generalization for legged robot locomotion.

RobotTT: A Tactile Transformer for Dexterous Manipulation

Shaoqing Tan — Fri, 17 Jul 2026 03:22:51 -0700

A tactile foundation model and sim-to-real pipeline designed for dexterous manipulation tasks. Combines transformer-based tactile sensing with a sim-to-real transfer approach.

TAC-LOCO: Unified Whole-Body Control for Contact-Aware Locomotion and Manipulation

Shaoqing Tan — Thu, 16 Jul 2026 14:11:13 -0700

A tactile-informed whole-body loco-manipulation controller for quadrupedal robots that unifies locomotion and manipulation using tactile sensing. Enables compliant and contact-aware whole-body control.

TACTIC: Contact-Centric Control for Whole-Arm Manipulation

Shaoqing Tan — Thu, 16 Jul 2026 14:07:52 -0700

A contact-centric whole-arm manipulation framework that conditions control on both tactile and visual inputs. Targets dexterous manipulation tasks requiring rich contact feedback.

SIEVE: Structure-Aware Data Selection for Imitation Learning with VLAs

Shaoqing Tan — Thu, 16 Jul 2026 05:17:43 -0700

SIEVE introduces structure-aware data selection specifically for imitation learning with Vision-Language-Action models, aiming to improve training efficiency and policy quality.

DexMachina: RL for Long-Horizon Bimanual Dexterous Policies

Shaoqing Tan — Thu, 16 Jul 2026 05:10:47 -0700

An RL algorithm that learns long-horizon bimanual dexterous policies for any robot hand from a single human demonstration, emphasizing generalization across hands, objects, and complex motions.

Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation

Shaoqing Tan — Thu, 16 Jul 2026 03:17:13 -0700

A unified platform comprising GE-Base (video diffusion model trained on 1M+ manipulation episodes), GE-Act (flow-matching action model), and GE-Sim (neural world simulator for closed-loop control). All code, models, and benchmarks are open-sourced.

RoboTTT: Test-Time Training for Visuomotor Policies

Shaoqing Tan — Wed, 15 Jul 2026 14:09:54 -0700

Introduces test-time training (TTT) inside the policy to natively scale visuomotor context to 8K timesteps at constant inference cost, enabling one-shot imitation from human video demos, online self-recovery from errors, and long-horizon assembly tasks.

HY-Embodied-VLM-1.0: Efficient Physical-World Agents

Shaoqing Tan — Wed, 15 Jul 2026 14:08:55 -0700

An embodied vision-language-action model with released weights, code, and paper targeting robotic manipulation and embodied AI tasks.

Learning Unified Force and Position Control for Legged Loco-Manipulation

Shaoqing Tan — Wed, 15 Jul 2026 05:11:01 -0700

Introduces a unified RL policy that jointly handles force and position control on quadrupedal and humanoid robots without force sensors, enabling position+force tracking, compliant behaviors, and force-aware imitation learning for contact-rich tasks.

HapticVLA: Extending Vision-Language-Action Models to Contact-Rich Tasks Without Touch Sensors

Shaoqing Tan — Tue, 14 Jul 2026 14:15:54 -0700

Enables contact-rich robotic manipulation using a VLA model trained with tactile sensing data but requiring no tactile input at inference time. Distills haptic knowledge into the vision-language-action policy.

AnoleVLA: Lightweight VLA with Deep State Space Models for Mobile Manipulation

Shaoqing Tan — Tue, 14 Jul 2026 14:09:29 -0700

Proposes a lightweight VLA model leveraging deep state space models (SSMs) instead of transformers for efficient mobile manipulation. Targets resource-constrained deployment scenarios with competitive performance.

ABot-N1: Visual Language Navigation Foundation Model

Shaoqing Tan — Tue, 14 Jul 2026 03:19:00 -0700

Decouples cognition from control in a VLM-based navigation policy, delivering 35% POI arrival gains and over 92% success rates in complex indoor and outdoor scenes.

ABot-AgentOS: A General-Purpose Robotic Agent Operating System

Shaoqing Tan — Tue, 14 Jul 2026 03:14:45 -0700

Provides scene-conditioned planning, context-isolated skill execution, multi-modal memory, and self-evolution capabilities for long-horizon embodied tasks.

Trust Region Policy Distillation (TOP-D)

Shaoqing Tan — Mon, 13 Jul 2026 14:13:47 -0700

Transforms unstable on-policy distillation into a stable training paradigm via dynamic proximal teacher construction, improving sample efficiency without additional computational overhead.

HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers

Shaoqing Tan — Mon, 13 Jul 2026 14:13:05 -0700

A new humanoid whole-body control method that distills complementary teachers for agentic task-space control on humanoids. The approach enables robust whole-body coordination for complex manipulation tasks.

EVA-Client: A Unified Framework for Real-Robot Policy Iteration

Shaoqing Tan — Mon, 13 Jul 2026 05:07:06 -0700

A unified open framework for real-robot policy iteration that integrates teleoperation data collection, model training, deployment, and evaluation into a single closed-loop pipeline.

UniVR-34B: A Vision-Only Foundation Model for Physical Tasks

Shaoqing Tan — Mon, 13 Jul 2026 03:22:53 -0700

First large-scale model to learn complex physical dynamics, visual reasoning, and long-horizon planning directly from visual demonstrations without text chains; released with 310k SFT and 3k RL samples across 16 sources alongside the VR-X benchmark.

CLAP: Converting Vision-Language Models into Vision-Language-Action Models via Language-Prompted Actions

Shaoqing Tan — Mon, 13 Jul 2026 03:19:11 -0700

Converts any pretrained vision-language model into a vision-language-action model with zero architectural changes by prepending natural-language action descriptions; reaches 90.8% success on LIBERO with a 2B model after less than 6 hours of post-training.

InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action

Shaoqing Tan — Sun, 12 Jul 2026 14:07:51 -0700

InternVLA-A15 is a new Vision-Language-Action (VLA) model released by InternRobotics, targeting robotic manipulation and control tasks. The model is accompanied by a Hugging Face collection and an associated technical paper.

MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation

Shaoqing Tan — Sun, 12 Jul 2026 09:36:13 -0700

Proposes foundation world-action models targeting real-time humanoid loco-manipulation, integrating world modeling with action generation for unified locomotion and manipulation control.

On-Device Diffusion Transformer Policy for Efficient Robot Manipulation

Shaoqing Tan — Sun, 12 Jul 2026 09:31:23 -0700

LightDP introduces a framework to accelerate Diffusion Transformer-based policies for real-time, on-device deployment in robot manipulation tasks, achieving efficiency for visuomotor control without sacrificing performance. Accepted to ICCV 2025.

Robotic World Model: Neural Dynamics for Locomotion

Shaoqing Tan — Sun, 12 Jul 2026 05:09:14 -0700

A neural dynamics world model paired with model-free RL for quadruped and humanoid locomotion in IsaacLab, enabling long-horizon autoregressive prediction with policies trained in imagined rollouts that outperform model-based baselines on sim-to-real transfer.

LingBot-Vision: Self-Supervised ViT with Masked Boundary Modeling

Shaoqing Tan — Sat, 11 Jul 2026 14:07:31 -0700

A self-supervised ViT backbone pretrained with masked boundary modeling for dense spatial perception, achieving strong zero-shot results on depth estimation, segmentation, and embodied tasks.

LaMem-VLA: Latent-Memory-Native VLA Framework

Shaoqing Tan — Sat, 11 Jul 2026 14:07:00 -0700

A VLA framework that curates experience into short- and long-term memory vaults, condenses them to latent tokens, and integrates directly into VLA reasoning, evaluated on SimplerEnv and LIBERO benchmarks.

LingBot-VA 2.0: Native Video-Action Foundation Model for Robot Control

Shaoqing Tan — Sat, 11 Jul 2026 05:10:02 -0700

A native video-action foundation model pretrained from scratch with a semantic visual-action tokenizer and foresight reasoning, enabling real-time robot control at ≤150 Hz on consumer GPUs without relying on retrofitted VLMs or video generators.

GigaWorld-1: Large-Scale World Model for Robot Policy Evaluation

Shaoqing Tan — Sat, 11 Jul 2026 05:07:46 -0700

A large-scale world model trained on 12,980 hours of data and benchmarked across 324,000+ simulated rollouts, designed for comprehensive robot policy evaluation.

Learning Unified Force and Position Control for Legged Loco-Manipulation

Shaoqing Tan — Fri, 10 Jul 2026 14:14:20 -0700

Proposes a unified RL policy trained in simulation that jointly handles force and position control without force sensors, enabling compliant behaviors and improved force-aware imitation learning on quadrupedal and humanoid robots.

Robix: Unified Vision-Language Model for Robotic Reasoning and Planning

Shaoqing Tan — Fri, 10 Jul 2026 05:18:52 -0700

A single vision-language model that unifies high-level reasoning, long-horizon planning, and human-robot interaction, serving as a cognitive layer above low-level vision-language-action models.

EmbodiedOneVision (EO-1): A Unified Decoder-Only Transformer for General Robot Control

Shaoqing Tan — Thu, 09 Jul 2026 14:20:24 -0700

A 3B-parameter open-source unified decoder-only transformer for general robot control that jointly handles perception, planning, reasoning, and action via interleaved vision-text-action pretraining on a 1.5M-sample dataset. Achieves strong results across manipulation tasks and benchmarks with fully open-sourced model, code, and dataset.

mimic-video: Video-Action Models for Robot Learning

Shaoqing Tan — Thu, 09 Jul 2026 14:12:13 -0700

Introduces Video-Action Models (VAMs) that replace standard VLMs in VLAs with a pretrained internet-scale video backbone (Cosmos-Predict2) combined with a flow-matching action decoder. Achieves approximately 10× better sample efficiency on real-world pick-and-place tasks.

Lowering the Barrier: The GEM Open-Source Arm

Shaoqing Tan — Thu, 09 Jul 2026 05:14:22 -0700

An open-source, sub-$500 7-DOF 3D-printed robotic arm with 1.2 kg payload, wrist and head cameras, and full LeRobot integration, designed to lower the barrier to entry for real manipulation research.

RynnWorld-4D: A 4D Embodied World Model for Bimanual Robot Control

Shaoqing Tan — Wed, 08 Jul 2026 14:11:44 -0700

A 4D embodied world model that predicts RGB, depth, and optical flow from RGB-D input and instructions using a tri-branch diffusion architecture. Enables closed-loop bimanual robot control by bridging world prediction and policy execution.

RynnWorld-Teleop: Digital Teleoperation via World Model Rendering

Shaoqing Tan — Wed, 08 Jul 2026 14:08:55 -0700

A digital teleoperation system that uses hand-pose streams to drive a world model for real-time (40+ FPS) high-fidelity robot video rendering from a single image. Removes physical robots from the training loop entirely.

VLA-Corrector: Adaptive Action Horizons through Latent Visual Monitoring

Shaoqing Tan — Wed, 08 Jul 2026 05:14:06 -0700

A lightweight plug-in that monitors latent visual dynamics in action-chunked policies, drops stale actions on drift, and replans on-the-fly achieving +17.7 success points while keeping the VLA backbone frozen.

Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization

Shaoqing Tan — Wed, 08 Jul 2026 05:09:22 -0700

A feed-forward model that decomposes unposed images into instance-structured 3D token groups without annotations, enabling unified reconstruction, segmentation, and manipulation.

Current as Touch: Proprioceptive Contact Feedback for Compliant Dexterous Manipulation

Shaoqing Tan — Tue, 07 Jul 2026 14:09:34 -0700

A proprioceptive contact feedback method leveraging motor current signals as a touch proxy to enable compliant dexterous manipulation without dedicated tactile sensors.

DART: One-Shot VLA Policy Adaptation via Weight-Space Arithmetic

Shaoqing Tan — Tue, 07 Jul 2026 05:16:14 -0700

Uses weight-space arithmetic to isolate domain shifts from task knowledge, enabling one-shot VLA policy adaptation to new cameras or embodiments.

Does VLA Even Know the Basics? Act2Answer Benchmark

Shaoqing Tan — Tue, 07 Jul 2026 05:12:02 -0700

Introduces a benchmark showing VLAs lose 20–40 points in commonsense/world knowledge versus source VLMs after robotics fine-tuning, evaluated via action-based answering.

Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding

Shaoqing Tan — Mon, 06 Jul 2026 14:07:09 -0700

Learns dexterous manipulation policies that explicitly ground actions in generative contact predictions from visuotactile observations, improving robustness on contact-rich tasks.

Freeform Preference Learning (FPL) for Robotic Manipulation

Shaoqing Tan — Mon, 06 Jul 2026 05:09:07 -0700

Introduces multi-axis preference supervision to learn dense, language-conditioned rewards across speed/precision/subtask axes without segmentation; enables compositional generalization and better long-horizon credit assignment than single-reward baselines.

Orca: The World is in Your Mind

Shaoqing Tan — Sun, 05 Jul 2026 14:09:11 -0700

Proposes a general world foundation model leveraging Next-State-Prediction to jointly generate text, images, and embodied actions within a unified framework.

Qwen-RobotNav: A Scalable Unified Navigation Model for Agentic Robotics

Shaoqing Tan — Sun, 05 Jul 2026 14:08:08 -0700

A unified 2B–8B parameter model for robot navigation tasks (VLN, ObjectNav, tracking, autonomous driving) via a configurable observation protocol, with demonstrated zero-shot deployment on real quadruped robots using agentic planners.

Scaling Robot Skills from Cheap Human Videos

Shaoqing Tan — Wed, 01 Jul 2026 14:08:01 -0700

Replaces noisy 6-DoF hand poses with relative wrist translation as a shared action space between humans and bimanual robots, enabling scalable skill acquisition from inexpensive video data. This approach outperforms full-pose baselines for robot skill learning.

ABC: An Open Behavior Cloning Stack for Bimanual Manipulation

Shaoqing Tan — Wed, 01 Jul 2026 05:16:54 -0700

Large-scale open-source framework for real-world robotic manipulation using behavior cloning, including the ABC-130K dataset with 3,500 hours and 130K+ episodes across 195 tasks. Provides hardware setups, simulators, and training recipes for Diffusion Transformers and Vision-Language-Action models.

Qwen-RobotManip: Alignment Unlocks Scale for Robotic Manipulation Foundation Models

Shaoqing Tan — Wed, 01 Jul 2026 05:12:07 -0700

Presents alignment techniques to scale robotic manipulation foundation models, building on the Qwen model family for dexterous robot control.

ASPIRE: Automated Skill Discovery for Robotics

Shaoqing Tan — Tue, 30 Jun 2026 14:11:15 -0700

Introduces the first automated system that continuously discovers, evolves, and accumulates reusable sensorimotor skills via evolutionary search over control programs, enabling compounding multi-task, sim-to-real, and cross-embodiment transfer without retraining end-to-end policies.

SERF: 4D Latent Mapping for Long-Horizon Mobile Manipulation

Shaoqing Tan — Tue, 30 Jun 2026 05:48:09 -0700

Embeds both the robot and environment into a shared 4D latent space augmented with forward-kinematics robot points, enabling a vision-language-action model to handle dynamic scenes and long-horizon memory. Outperforms image-only VLA baselines on the BEHAVIOR-1K benchmark for mobile manipulation.

ViserDex: Visual Sim-to-Real for Robust Dexterous In-Hand Reorientation

Shaoqing Tan — Tue, 30 Jun 2026 05:13:30 -0700

A single-camera sim-to-real framework that uses physically consistent 3D Gaussian Splatting augmentations to achieve zero-shot transfer of dexterous in-hand reorientation policies to an Allegro hand. The approach trains entirely on consumer hardware while maintaining high fidelity to real-world dynamics.

DexSkin: A High-Coverage, Conformable "Electronic Skin" for Robot Fingers

Shaoqing Tan — Tue, 30 Jun 2026 03:12:49 -0700

Introduces a high-coverage, conformable robotic skin hardware system designed to improve data collection and policy learning for contact-rich, dexterous manipulation tasks. The system provides rich tactile sensing coverage to enable more capable robot manipulation policies.

EBench: A Diagnostic Benchmark for Generalist Manipulation Policies

Shaoqing Tan — Tue, 30 Jun 2026 03:11:09 -0700

A CAT-scan style diagnostic benchmark for robot foundation models that evaluates policies such as π0, π0.5, and Qwen-RobotManip beyond single success rates. The benchmark is designed to distinguish genuine generalization from overfitting to demonstrations in generalist manipulation policies.

VITRA: A Foundation for Dexterous VLA via Human Video Pretraining

Shaoqing Tan — Mon, 29 Jun 2026 14:13:41 -0700

A scalable VLA pretraining pipeline that converts unstructured egocentric human videos into robot training data, trains a dexterous hand VLA, and fine-tunes on robot data, achieving strong zero-shot generalization and real-robot dexterous manipulation.

DexWM: A Dexterous Manipulation World Model from Human Videos

Shaoqing Tan — Mon, 29 Jun 2026 14:08:49 -0700

A dexterous manipulation world model pretrained on 829 hours of EgoDex human data and DROID robot data using conditioned diffusion transformers, enabling open-loop rollouts and sim-to-real transfer with minimal robot fine-tuning.

PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning

Shaoqing Tan — Mon, 29 Jun 2026 05:14:05 -0700

Introduces PoLAR, a method that factorizes latent action representations into extent and mode components to improve robot policy learning efficiency and generalization.

Continual Robot Policy Learning via Variational Neural Dynamics

Shaoqing Tan — Mon, 29 Jun 2026 05:13:48 -0700

Proposes a variational neural dynamics framework for continual robot policy learning, enabling robots to acquire new skills without forgetting previously learned ones.

PhysisForcing: Physics-Reinforced World Models for Robotic Manipulation

Shaoqing Tan — Mon, 29 Jun 2026 03:24:42 -0700

Plug-and-play training framework that enforces physical plausibility in robotic video generation models, achieving SOTA on R-Bench, PAI-Bench, and EZS-Bench. Lifts WorldArena success rate from 16% to 24% with zero extra inference cost.

Translation as a Bridging Action

Shaoqing Tan — Mon, 29 Jun 2026 03:11:02 -0700

Replaces noisy 6DoF hand poses with relative wrist translation as a shared action space between cheap human videos and bimanual robots. Scales data-efficiently and outperforms full-pose baselines on manipulation tasks.

Play2Perfect: Dexterous Play Pretraining for Precise Assembly

Shaoqing Tan — Sun, 28 Jun 2026 14:13:36 -0700

Pre-trains a dexterous hand via unstructured 'play' interactions with objects, then fine-tunes for precise assembly tasks including 0.5 mm clearance insertions and furniture screwing, achieving 33x better sample efficiency than RL from scratch.

Dexora: Open-Source VLA for High-DoF Bimanual Dexterity

Shaoqing Tan — Sun, 28 Jun 2026 14:11:01 -0700

First open-source Vision-Language-Action (VLA) model for dual-arm, dual-hand 36-DoF dexterous manipulation, trained on 100K simulated and 10K real trajectories with strong cross-embodiment transfer capabilities.

WorldVLA: Towards Autoregressive Action World Model

Shaoqing Tan — Sun, 28 Jun 2026 05:09:08 -0700

Unifies VLA and world-modeling in a single autoregressive transformer that predicts both future images and actions. Outperforms separate VLA or world models on LIBERO simulation benchmarks.

HumDex: Humanoid Dexterous Manipulation Made Easy

Shaoqing Tan — Sun, 28 Jun 2026 05:08:49 -0700

HumDex targets humanoid dexterous manipulation, aiming to simplify the development of dexterous manipulation capabilities for humanoid robots.

ForceBand: Learning Forceful Manipulation with sEMG

Shaoqing Tan — Sat, 27 Jun 2026 14:11:53 -0700

Presents an open-source, low-cost sEMG wristband framework that extracts force signals from human muscle activity in videos, enabling zero-shot human-to-robot transfer of forceful manipulation policies across any robot, camera, or environment.

In-Context World Modeling for Robotic Control

Shaoqing Tan — Sat, 27 Jun 2026 14:08:16 -0700

Introduces ICWM, a method that learns world dynamics from just seconds of a robot's self-generated interaction data, enabling zero-shot adaptation to unseen cameras and new robot morphologies without any fine-tuning.

WOLF-VLA: Vision-Language-Action for Humanoid Walking

Shaoqing Tan — Sat, 27 Jun 2026 05:15:30 -0700

Introduces a framework integrating vision-language-action models for whole-body humanoid locomotion, addressing optimal control and learning for complex bipedal behaviors. Combines VLA learning with locomotion-specific control for humanoid robots.

Motion-Focused Latent Action for Cross-Embodiment VLA from Human Videos

Shaoqing Tan — Sat, 27 Jun 2026 05:14:57 -0700

Proposes a motion-focused latent action representation for cross-embodiment vision-language-action policies learned from human videos, accepted to IROS 2026.

ManiFlow: Manipulation via Rectified Flow

Shaoqing Tan — Sat, 27 Jun 2026 03:11:30 -0700

ManiFlow is a visuomotor imitation learning policy using consistency flow matching with a DiT-X architecture that generates high-quality actions in 1–2 steps. It works across single-arm, bimanual, and humanoid platforms using RGB or point cloud inputs.

RL-100: Toward Highly Reliable Real-World Robot Reinforcement Learning

Shaoqing Tan — Sat, 27 Jun 2026 03:10:48 -0700

RL-100 demonstrates highly reliable real-world RL manipulation achieving 900/900 success rates across 7 tasks with up to 250 consecutive trials without failure. It also shows strong robustness to disturbances and zero/few-shot adaptation capabilities.

Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation

Shaoqing Tan — Fri, 26 Jun 2026 14:10:00 -0700

A video-diffusion world model trained on over 1 million manipulation episodes (3,000 hours) that includes an action model and neural simulator for closed-loop robotic manipulation control, with all code and models open-sourced.

Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning for Long-Horizon Contact-Rich Manipulation

Shaoqing Tan — Fri, 26 Jun 2026 05:12:44 -0700

Proposes a hierarchical imitation learning framework using bilateral control, subtask-level progress tracking, and keyframe memory to handle long-horizon, contact-rich manipulation tasks.

From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation

Shaoqing Tan — Fri, 26 Jun 2026 05:08:58 -0700

Uses reinforcement learning to improve process reasoning capabilities in robotic manipulation policies, shifting the model from passive observation to active critique.

ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning

Shaoqing Tan — Fri, 26 Jun 2026 03:12:15 -0700

ROVE leverages reinforcement learning to enable humanoid robots to benefit from human interventions during manipulation tasks.

ConstrainedMimic: Safe Humanoid Robot Motion Tracking

Shaoqing Tan — Fri, 26 Jun 2026 03:11:55 -0700

A control framework for safe humanoid robot motion tracking using RL policies with real-time constraint enforcement via kinematics, dynamics, and control barrier functions.

REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering

Shaoqing Tan — Thu, 25 Jun 2026 14:13:34 -0700

Introduces spatio-temporal policy learning combined with physics-guided filtering to achieve robust and extremely agile robot control.

HiFlow: Tokenization-Free Scale-Wise Autoregressive Policy Learning via Flow Matching

Shaoqing Tan — Thu, 25 Jun 2026 14:08:48 -0700

Introduces a tokenization-free autoregressive policy learning framework using flow matching across scales for robotic control.

Reactive Diffusion Policy: Slow-Fast Visual-Tactile Learning for Contact-Rich Manipulation

Shaoqing Tan — Thu, 25 Jun 2026 05:15:32 -0700

Introduces a slow-fast imitation learning framework combining diffusion-based planning with reactive tactile/force feedback for contact-rich manipulation tasks. Also includes TactAR, an AR-based teleoperation system with tactile sensing.

SARM2 + SPIRAL: Multi-Task Reward Models and RL Refinement for Long-Horizon Dexterous Manipulation

Shaoqing Tan — Thu, 25 Jun 2026 05:12:28 -0700

Combines scalable autonomous reward modeling with RL-based refinement to improve vision-language-action policies on long-horizon dexterous manipulation tasks via autonomous rollouts. Demonstrates significant gains over imitation learning baselines.

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm VLA Systems

Shaoqing Tan — Thu, 25 Jun 2026 03:34:15 -0700

Introduces coordination-aware structured action modeling for dual-arm robotic systems within a VLA framework. Addresses the unique challenges of bimanual manipulation through specialized action representations.

ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation

Shaoqing Tan — Thu, 25 Jun 2026 03:31:11 -0700

Proposes interleaved vision and language reasoning for robotic manipulation within a VLA framework. Aims to improve instruction following and task performance through integrated multimodal reasoning.

Playful Agentic Robot Learning

Shaoqing Tan — Thu, 25 Jun 2026 03:22:48 -0700

Self-directed play combined with Code-as-Policy for reusable skill acquisition and downstream manipulation tasks.

Learning Unified Force and Position Control for Legged Loco-Manipulation

Shaoqing Tan — Wed, 24 Jun 2026 14:14:01 -0700

A unified RL policy for quadrupeds and humanoids that jointly handles force and position control without force sensors, enabling compliant behaviors, force-aware imitation learning, and contact-rich tasks.

Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies

Shaoqing Tan — Wed, 24 Jun 2026 14:11:02 -0700

Explores imitation learning approaches for multi-robot systems, focusing on policy coupling through sequential asymmetric imitation to enable collaborative robot behaviors.

AstraBrain-WBC 0.5: A Humanoid Robot Cerebellum Foundation Model

Shaoqing Tan — Wed, 24 Jun 2026 05:17:07 -0700

A humanoid robot 'cerebellum' foundation model trained on 20,000 hours of human motion data that demonstrates scaling laws for robot motion control and enables zero-shot execution of unseen motions on real humanoids.

SRL: Combining SLIP Model and Reinforcement Learning for Agile Robotic Jumping

Shaoqing Tan — Wed, 24 Jun 2026 05:14:53 -0700

Combines the Spring-Loaded Inverted Pendulum (SLIP) model with reinforcement learning to achieve agile jumping behaviors in robotic systems.

DataClaw0: Agentic Tailoring for Raw Multimodal Streams

Shaoqing Tan — Wed, 24 Jun 2026 03:20:11 -0700

A 9B model that filters noise from videos, GUI, and embodied data streams, reorganizing them into dense supervision via factual anchors and semantic synthesis; trained with SFT + GRPO across five domains with benchmarks.

ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining

Shaoqing Tan — Wed, 24 Jun 2026 03:16:42 -0700

Converts 6K+ hours of mixed human/robot egocentric video into robot pseudo-actions via camera-space alignment and reliability-aware loss, achieving 72.8% on RoboCasa and 91.1% on RoboTwin.

VERA: Video-to-Action World Model Policy

Shaoqing Tan — Wed, 24 Jun 2026 00:03:04 -0700

A 14B-parameter video world model that converts predicted visual futures into embodiment-agnostic actions via Jacobian inverse-dynamics, enabling zero-shot cross-robot transfer across a Panda arm and 16-DoF hand with open-sourced weights and training code.

GEN-1: Scaled Dexterous Manipulation Foundation Model

Shaoqing Tan — Wed, 24 Jun 2026 00:01:21 -0700

A dexterous manipulation foundation model trained on 500k hours of real-world bimanual data that handles deformable objects such as cardboard folding and screw packing, featuring online retry and adaptation capabilities.

Efficient Hybrid SE(3)-Equivariant Visuomotor Flow Policy via Spherical Harmonics for Robot Manipulation

Shaoqing Tan — Mon, 22 Jun 2026 05:16:39 -0700

Develops an SE(3)-equivariant flow-based visuomotor policy leveraging spherical harmonics for efficient and geometrically consistent robot manipulation.

Cortical Policy: A Dual-Stream View Transformer for Robotic Manipulation

Shaoqing Tan — Mon, 22 Jun 2026 05:10:43 -0700

Introduces a dual-stream transformer architecture inspired by cortical visual processing for learning robotic manipulation policies.

VisualClaw: A Self-Evolving Wearable Vision Agent

Shaoqing Tan — Mon, 22 Jun 2026 03:05:59 -0700

An edge-filtered video streaming agent that evolves skills from memory and runs on smart glasses, reducing API costs by 98%, accompanied by the VisualClawArena benchmark dataset.

Kairos: A Native World Model Stack for Physical AI

Shaoqing Tan — Sun, 21 Jun 2026 14:23:15 -0700

A 4B unified architecture for world understanding, generation, and action with hybrid linear attention enabling real-time edge inference across embodiments, outperforming 14B models on embodied benchmarks.

DragMesh-2: A Contact-Driven Framework for Dexterous Hand–Object Interaction

Shaoqing Tan — Sun, 21 Jun 2026 14:12:42 -0700

A framework that trains a 51-DoF dexterous hand to open drawers and doors using only physical contact without requiring tactile sensors.

Guava: A Universal Harness for Robot Manipulation

Shaoqing Tan — Sun, 21 Jun 2026 05:10:01 -0700

A 4B open-source VLA-style model trained on fewer than 2K simulation trajectories that matches closed frontier systems on real-world manipulation tasks with zero-shot generalization to novel objects and long-horizon behaviors, including failure-recovery demonstrations.

Geometric Action Model for Robot Policies

Shaoqing Tan — Sun, 21 Jun 2026 05:09:28 -0700

A new geometric action model for robot manipulation policies that focuses on structured action representations to improve policy learning and generalization.

ENPIRE: Physical AutoResearch with a Fleet of 8 Robots

Shaoqing Tan — Sun, 21 Jun 2026 03:14:25 -0700

ENPIRE demonstrates fully autonomous physical AutoResearch where Codex agents control a fleet of 8 robots overnight, self-improving through real hardware rollouts on tasks like zip-tie tying and GPU installation, while discovering physical scaling laws with built-in safety harnesses and frozen reward classifiers derived from demonstrations.

MolmoAct2: An Open Foundation Model for Real-World Robotics

Shaoqing Tan — Sun, 21 Jun 2026 00:43:08 -0700

An open VLA-style robotics foundation model featuring open weights, open dataset, open action tokenizer, and a depth-reasoning variant; designed to enable community experiments on real robots for manipulation and generalist policies.

Hy-Embodied-0.5-VLA: A Massive Bimanual Teleoperation Dataset for Vision-Language-Action

Shaoqing Tan — Mon, 15 Jun 2026 05:27:45 -0700

Released a massive bimanual robot manipulation dataset with 2,163 hours and 250K+ episodes across 70+ tasks, along with a compatible VLA model for multi-view egocentric teleop. The dataset and model are fully compatible with LeRobot v3.0.

Q-Guided Flow: Test-Time Gradient Guidance of Flow Policies

Shaoqing Tan — Sun, 14 Jun 2026 14:23:14 -0700

New framework for guided flow-matching policies that improves long-horizon robotic control and sample efficiency.

Flow Reversal Steering: Guiding Diffusion-Based Robot Policies with High-Level Reasoning

Shaoqing Tan — Sun, 14 Jun 2026 14:13:15 -0700

Introduces flow reversal steering to guide diffusion-based vision-language-action models with high-level VLM reasoning and enables RL directly in the diffusion noise space.

Test-Time Compute Scaling for Robot Policies (DIRECT)

Shaoqing Tan — Sun, 14 Jun 2026 05:26:08 -0700

Larger models + more thinking + more context improve performance on some prompts but not others; a learned router enables better performance/latency trade-offs.

LabVLA: Bringing Vision-Language-Action to the Chemistry Lab

Shaoqing Tan — Sun, 14 Jun 2026 05:16:11 -0700

RoboGenesis generates 10K+ lab scenes across 16 robot embodiments; LabVLA (Qwen3-VL + DiT flow-matching) achieves 71.1% success on LabUtopia and transfers to real Franka arms.

Humanoid-GPT: A Foundation Model for Zero-Shot Humanoid Control

Shaoqing Tan — Sat, 13 Jun 2026 14:17:57 -0700

GPT-style Transformer pretrained on 2 billion motion frames that achieves agile, generalist zero-shot control on a real Unitree G1 humanoid for tasks like soccer, dancing, and digging. Requires no fine-tuning or task-specific adaptation.

CHORUS: Decentralized Multi-Robot Collaboration with a Single Shared VLA Model

Shaoqing Tan — Sat, 13 Jun 2026 14:07:47 -0700

Finetunes a single Vision-Language-Action (VLA) foundation model so that any robot in a team can control any other. Outperforms both per-robot specialists and a monolithic centralized policy while scaling to large teams.

RISE: Self-Improving Robot Policy with Compositional World Model

Shaoqing Tan — Sat, 13 Jun 2026 05:18:47 -0700

Trains a compositional world model on real robot data to enable closed-loop policy improvement via future prediction and progress evaluation, bypassing both risky real-world RL and traditional sim-to-real gaps.

EmbodiedOneVision: Interleaved Vision-Text-Action Pretraining for General Robot Control

Shaoqing Tan — Fri, 12 Jun 2026 14:34:55 -0700

Open-source 3B unified embodied foundation model trained on 1.5M interleaved vision-text-action samples for perception, planning, and acting.

Robix: A Unified Model for Robot Interaction, Reasoning and Planning

Shaoqing Tan — Fri, 12 Jun 2026 14:15:36 -0700

A single vision-language model that unifies reasoning, task planning, and human-robot interaction for complex instructions and long-horizon tasks.

Robotic World Model: Learning to Simulate for Robust Robot Control

Shaoqing Tan — Fri, 12 Jun 2026 05:07:03 -0700

Presents a neural network-based world model for model-based reinforcement learning in robotics, focusing on sim-to-real transfer for quadrupedal and humanoid robots. Enables robust policy optimization through learned environment simulation.

AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization

Shaoqing Tan — Wed, 10 Jun 2026 14:21:45 -0700

Embodied egocentric simulation framework that controls first-person worlds with 3D human motion and customizes evolving scenes via pose-anchored views.

ArtiFixer: Few-Step Diffusion for 3D Scene Reconstruction

Shaoqing Tan — Wed, 10 Jun 2026 14:10:57 -0700

Few-step auto-regressive diffusion model that converts broken 3D reconstructions into fully realized scenes, outperforming prior methods by 1-3 dB PSNR.

Deployment-Time Memorization in Foundation-Model Agents

Shaoqing Tan — Wed, 10 Jun 2026 05:17:10 -0700

Examines memorization phenomena that occur during deployment of foundation model agents in practical applications.

Adversarial Machine Learning: Taxonomy, Threat Models, and Mitigation Strategies in Deep Neural Networks

Shaoqing Tan — Tue, 09 Jun 2026 14:08:24 -0700

Focuses on security aspects of deep neural networks, providing taxonomy and mitigation strategies for adversarial attacks.

SoCRATES: Evaluating LLM Mediators in Conflict Scenarios

Shaoqing Tan — Tue, 09 Jun 2026 05:38:49 -0700

First comprehensive framework for evaluating LLM mediators in real-time, emotional, socio-cognitive scenarios.

Unembedding Matrix as a Feature Lens: Unlocking Better Text Embeddings

Shaoqing Tan — Tue, 09 Jun 2026 05:32:33 -0700

Improves embedding quality without extra training by using the unembedding matrix as a feature lens for text embeddings.

LeanMarathon: Autonomous Formalization of Math Proofs on Erdős Problems

Shaoqing Tan — Mon, 08 Jun 2026 14:38:40 -0700

Presents an autonomous system for formalizing mathematical proofs, specifically targeting Erdős problems. Demonstrates automated proof formalization capabilities in the Lean theorem prover.

Deep Research Agents: Survey and Roadmap for Autonomous AI Research

Shaoqing Tan — Mon, 08 Jun 2026 14:26:32 -0700

Provides a comprehensive taxonomy, benchmarks, and future directions for autonomous AI research agents. Examines systematic approaches to developing agents capable of conducting independent research.

Cosmos 3: Omnimodal World Models for Physical AI

Shaoqing Tan — Sun, 07 Jun 2026 14:08:40 -0700

Omnimodal world models explicitly designed for Physical AI and robotics applications. Enables improved simulation and control for robotic systems through multimodal understanding.

Humanoid-GPT: GPT-Style Transformer for Zero-Shot Dynamic Humanoid Control

Shaoqing Tan — Sun, 07 Jun 2026 05:11:47 -0700

GPT-style Transformer trained on 2 billion motion frames enabling zero-shot dynamic humanoid control for tasks like soccer, dancing, and digging on real Unitree G1 robots without fine-tuning. Breaks the agility-generalization trade-off in humanoid robotics.

Bending Paper, Shaping Dexterity: The Robotic Origami Challenge

Shaoqing Tan — Fri, 05 Jun 2026 14:20:26 -0700

New IROS benchmark providing 500+ teleoperation episodes and physically accurate simulation assets for training policies that outperform human origami experts.

GraspGen-X: A Foundation Model for Zero-Shot 6-DoF Grasping

Shaoqing Tan — Fri, 05 Jun 2026 14:10:23 -0700

First foundation model for zero-shot grasping trained on billions of simulated grasps, enabling generalized manipulation without task-specific training.

When Does Deep RL Beat Calibrated Baselines?

Shaoqing Tan — Thu, 04 Jun 2026 05:24:54 -0700

Benchmark study examining when deep reinforcement learning outperforms calibrated baseline methods in adaptive resource control tasks.

Training Deep Networks as Random Effects: An Optimization–Inference Duality

Shaoqing Tan — Thu, 04 Jun 2026 05:10:17 -0700

Explores training dynamics of deep neural networks through a statistical lens, examining the duality between optimization and inference perspectives.

Generative Depth Supervision for Embodied Vision-Language Models

Shaoqing Tan — Tue, 02 Jun 2026 05:07:28 -0700

Vision-language model that adds generative depth prediction during pre-training for physical grounding; achieves SOTA on embodied benchiments and transfers directly to real-robot tasks.

PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation

Shaoqing Tan — Mon, 01 Jun 2026 05:12:12 -0700

Presents a 3D point-cloud-based world model trained on mixed real/sim data that enables zero-shot grasping and articulated object handling on real robots by explicitly modeling spatial structure.

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

Shaoqing Tan — Sun, 31 May 2026 16:41:21 -0700

NVIDIA Research presents LocateAnything, a unified generative grounding and detection framework based on Parallel Box Decoding (PBD). Unlike prior VLMs that serialize bounding boxes into sequential coordinate tokens, PBD treats each box as an atomic unit and predicts all coordinates in a single forward pass. This preserves intra-box geometric coherence while achieving 2.5x faster decoding throughput. The model supports diverse localization tasks including document understanding, GUI grounding, dense object detection, and OCR localization. Built on Moon-ViT vision encoder and Qwen2.5 language decoder. Trained on LocateAnything-Data with 138M language queries and 785M bounding boxes. Achieves state-of-the-art on LVIS, M6Doc, and ScreenSpot-Pro benchmarks. Models and demo available on HuggingFace.

LT2: Linear-Time Looped Transformers

Shaoqing Tan — Sun, 31 May 2026 05:21:02 -0700

Replaces quadratic softmax attention in looped architectures with linear/sparse mechanisms for iterative memory refinement, achieving parity with standard looped transformers at much lower cost.

One Learning Rate Doesn't Fit All: Layerwise Spectral Scheduling for Transformers

Shaoqing Tan — Sun, 31 May 2026 05:07:55 -0700

Shows that modern transformers are highly heterogeneous across layers and proposes layerwise learning rates based on weight spectrum shape, yielding up to 1.5× training speedup on LLaMA/GPT-style models.

SimToolReal: Procedural Tool Generation and a Universal Objective for Zero-Shot Tool Manipulation

Shaoqing Tan — Sat, 30 May 2026 14:19:08 -0700

Trains generalist policies in simulation on procedurally generated tools to move objects, enabling real-world tool use across varied shapes/sizes.

Robometer and the Future of Robotic Reward Modeling

Shaoqing Tan — Sat, 30 May 2026 14:07:00 -0700

New framework for scalable robotic reward modeling using trajectory comparisons to train general-purpose reward models.

Qwen-VLA: A Generalist Vision–Language–Action Robot Model

Shaoqing Tan — Fri, 29 May 2026 14:15:49 -0700

A single generalist VLA built on Qwen3.5-4B + 1.15B DiT flow-matching action decoder that unifies manipulation, navigation, and trajectory prediction across 11 embodiments via text-described embodiment prompts. Trained in four stages and outperforms task-specific specialists on real ALOHA and sim benchmarks without per-task fine-tuning.

EXPO-FT: Sample-Efficient Reinforcement Learning Fine-Tuning for Vision-Language-Action Models

Shaoqing Tan — Fri, 29 May 2026 05:13:54 -0700

Extends the EXPO method with real-world RL post-training for VLAs using image observations, action chunking, DAgger, and on-the-fly Q-value maximization. Achieves 30/30 success on 8 challenging manipulation tasks with only ~19 min of RL data on average.

RoboMeter: Learning Dense Rewards from Successes and Failures

Shaoqing Tan — Thu, 28 May 2026 22:00:45 -0700

RoboMeter trains dense reward models from both successful and failed robot trajectories, solving a key gap in prior methods that only learn from expert demos.

MobileGym: A Controllable, Parallel Sandbox for Mobile GUI Agents

Shaoqing Tan — Wed, 27 May 2026 05:34:07 -0700

Browser-hosted mobile environment with JSON state, deterministic judges, and 256 parallel rollouts. Reports +40.7 real-device points after GRPO training on 416 tasks for GUI agent development.

ANY2ANY: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking

Shaoqing Tan — Wed, 27 May 2026 05:19:30 -0700

Introduces a method to transfer a Unitree G1 foundation policy (Gear-Sonic) to LimX Oli/Luna humanoids using only 1% of the original compute/data. Achieves fast convergence and strong tracking performance for humanoid whole-body control.

TriSplat: Feed-Forward 3D Reconstruction with Triangulated Meshes

Shaoqing Tan — Tue, 26 May 2026 14:31:44 -0700

Outputs physics-engine-compatible triangle meshes directly from sparse, unposed images without Gaussian splatting or post-processing.

MIKASA-Robo-VLA: A Memory-Intensive Benchmark for Vision-Language-Action Robotics

Shaoqing Tan — Tue, 26 May 2026 14:11:34 -0700

Releases a benchmark suite for systematically evaluating memory in Vision-Language-Action policies on tabletop manipulation tasks.

PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation

Shaoqing Tan — Mon, 25 May 2026 14:12:29 -0700

Introduces large-scale 3D world models pretrained on diverse real-world video to enable robust robotic manipulation policies that generalize beyond simulation.

Bimanual Pegboard Manipulation: A Benchmark for Vision-Language-Action Models

Shaoqing Tan — Sun, 24 May 2026 14:14:31 -0700

New LeRobot-based bimanual pegboard manipulation dataset with 52 episodes, 30k frames, 3 camera views, and 14-DOF arms for VLA evaluation. Provides standardized benchmark for vision-language-action model assessment.

FutureSim: Replaying Real-World Events to Evaluate AI Forecasting Agents

Shaoqing Tan — Sun, 24 May 2026 05:31:47 -0700

A benchmark designed to test AI models' capabilities in making accurate 3-month future predictions.

AgentFloor: A Benchmark for Long-Horizon Agent Planning

Shaoqing Tan — Sun, 24 May 2026 05:18:22 -0700

A 30-task benchmark for evaluating long-horizon planning capabilities across 16 different AI models.

AlexNet: The Deep Convolutional Network That Transformed Vision

Shaoqing Tan — Sat, 23 May 2026 14:21:12 -0700

AlexNet paper that sparked the modern deep learning revolution through convolutional neural networks.

A Few Useful Things to Know About Machine Learning

Shaoqing Tan — Sat, 23 May 2026 14:13:24 -0700

Practical insights into ML pitfalls and best practices for machine learning practitioners.

SimToolReal: A Universal Dexterous Tool-Use Policy

Shaoqing Tan — Sat, 23 May 2026 05:26:37 -0700

Introduces an object-centric sim-to-real policy that enables zero-shot dexterous tool use on physical robots without task-specific fine-tuning. Leverages simulation data for robust real-world transfer.

Mimic-Video: Learning Physics Priors from Web-Scale Video for Robot Dexterity

Shaoqing Tan — Sat, 23 May 2026 05:13:14 -0700

Pretrains robot policies on large-scale web video to acquire dynamics and physics understanding instead of static images or VLMs. Yields faster training, better generalization, and superior dexterous manipulation results in real-world tasks.

Deep Residual Learning for Image Recognition (ResNet)

Shaoqing Tan — Sat, 23 May 2026 02:03:20 -0700

Introduced residual connections (ResNet) enabling training of very deep networks, still widely used in modern architectures.

Attention Is All You Need – The Transformer Revolution

Shaoqing Tan — Sat, 23 May 2026 01:49:20 -0700

Introduced the Transformer architecture based purely on attention mechanisms, becoming the foundation of nearly all modern large language models.

NVIDIA Cosmos: World Foundation Models for Physical AI

Shaoqing Tan — Wed, 20 May 2026 05:19:16 -0700

World foundation models for video and physics prediction with SynthID watermarking for responsible AI practices. Developed in collaboration with Google DeepMind.

LATENT: Teaching a Humanoid to Play Tennis from Imperfect Data

Shaoqing Tan — Tue, 19 May 2026 14:11:09 -0700

Introduces a three-stage pipeline that extracts a latent action space from noisy, low-quality human motion capture, then trains a high-level RL policy in simulation to compose and execute dynamic whole-body tennis skills. Achieves volleys at human-level performance on a humanoid robot.

CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models

Shaoqing Tan — Tue, 19 May 2026 05:26:22 -0700

Closed-loop framework coupling Vision-Language Models with Video Generation Models at step-level granularity. Mitigates long-horizon drift and mid-clip errors in goal-directed video reasoning for robotic planning.

World Action Models: The Next Frontier in Embodied AI

Shaoqing Tan — Tue, 19 May 2026 05:10:48 -0700

First systematic survey defining World Action Models (WAMs) as embodied foundation models that jointly predict future states and generate actions. Covers architectures, data ecosystems, and evaluation protocols.

Training a Whole-Body Control Foundation Model

Shaoqing Tan — Mon, 18 May 2026 14:26:59 -0700

Describes end-to-end learning of a foundation model for adaptive whole-body humanoid control via massive simulation variation. Combines proprioceptive perception and policy adaptation across embodiments.

DexJoCo: A Unified Benchmark for Task-Oriented Dexterous Manipulation

Shaoqing Tan — Mon, 18 May 2026 14:11:24 -0700

Releases an open-source MuJoCo-based benchmark with 11 dexterous tasks, low-cost teleoperation hardware, and 1.1K human demonstrations. Designed to evaluate and train modern VLA/robotic policies.

MMSkills: Building Multimodal Skill Libraries for Visual Agents

Shaoqing Tan — Mon, 18 May 2026 05:29:29 -0700

Skill library, demonstrations, and dataset for multi-modal robotic skill learning and manipulation tasks.

PhysBrain 1.0 VLA (TwinBrainVLA): Dual-Brain Vision-Language-Action with Physics-Grounded Learning

Shaoqing Tan — Mon, 18 May 2026 05:16:17 -0700

Introduces dual-brain fusion Vision-Language-Action model with LangForce physics-grounded training methodology.

MolmoAct2-LIBERO: An Open Vision-Language-Action Model for Robotics

Shaoqing Tan — Sun, 17 May 2026 14:24:27 -0700

Vision-Language-Action (VLA) model fine-tuned on the merged LIBERO robotics dataset (1,693 episodes, 273k+ frames) achieving 98.25% success rate on manipulation tasks. Released with both checkpoint and dataset for VLA finetuning.

SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Diffusion Transformers

Shaoqing Tan — Sun, 17 May 2026 14:12:10 -0700

A 2.6B-parameter open-source world model that generates coherent 720p, minute-long videos with precise 6-DoF camera control on a single GPU using a Hybrid Linear Diffusion Transformer + Gated DeltaNet for long-context efficiency. Targets controllable physics simulation.

WildClawBench: A Real-World, Long-Horizon Benchmark for AI Agents

Shaoqing Tan — Sun, 17 May 2026 05:24:48 -0700

New benchmark and dataset for robotic manipulation in unconstrained 'wild' environments. Includes standardized containers, leaderboards, and evaluation protocols for cross-embodiment policies.

MCP-Cosmos: Bring Your Own World Model

Shaoqing Tan — Sun, 17 May 2026 05:15:11 -0700

Introduces a latent-space world model framework that lets agents simulate state transitions and iteratively refine plans before real-world execution. Evaluated on 20+ MCP-Bench tasks with measurable gains in tool-use success.

OpenAI o1: Teaching LLMs to Think Slow and Deep

Shaoqing Tan — Sat, 16 May 2026 18:41:07 -0700

Details OpenAI's reasoning-focused o1 model and its 'long thought' approach using test-time compute scaling. Explores how extended reasoning during inference can improve model performance on complex tasks.

The Llama 3 Herd of Models

Shaoqing Tan — Sat, 16 May 2026 18:32:22 -0700

Comprehensive technical report on the Llama 3 family, covering architecture, training at scale, multimodal extensions, and real-world impact. Details the development of Meta's flagship open-source language model series.

LATENT: Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

Shaoqing Tan — Sat, 16 May 2026 18:03:21 -0700

Introduces a three-stage pipeline that extracts a latent action space from low-quality human tennis demonstrations, then trains a high-level policy in simulation via reinforcement learning. Enables dynamic whole-body humanoid tennis play with back-and-forth volleys at human level.

AnyFlow: Any-Step Video Diffusion for Predictive World Modeling

Shaoqing Tan — Thu, 14 May 2026 16:13:25 -0700

First any-step video diffusion framework using flow maps, allowing a single model to adapt to arbitrary inference budgets for scalable high-quality video generation relevant to predictive world modeling.

# Robotics: The Endgame

Shaoqing Tan — Thu, 14 May 2026 16:02:02 -0700

Technical roadmap mirroring LLM scaling: critiques VLAs, advocates video world models as second pretraining phase, introduces World Action Models (WAM), manipulation data flywheels, EgoScale with new Dexterity Scaling Law, and DreamDojo end-to-end neural physics engine for sim RL.

Claw-Eval: Toward Trustworthy and Transparent Evaluation of Autonomous Agents

Shaoqing Tan — Wed, 08 Apr 2026 07:19:18 -0700

Benchmark with 2,159 rubric items across 300 tasks using trajectory-aware grading and 3-trial Pass^3 scoring to mitigate luck. Evaluates agent reliability in real-world robotics settings.

LIBERO-Para: Paraphrase Robustness in Robotic Manipulation

Shaoqing Tan — Wed, 08 Apr 2026 07:18:01 -0700

Reveals paraphrase fragility in VLAs causing 22-52% success drops due to task misidentification. Introduces PRIDE metric weighting success by paraphrase difficulty on LIBERO benchmark manipulation tasks.

YOR: Your Own Mobile Manipulator for Generalizable Robotics

Shaoqing Tan — Tue, 07 Apr 2026 07:41:37 -0700

Low-cost mobile manipulator design and training strategies for broad generalization in real-world tasks.

EgoSim: Egocentric World Simulator for Embodied Interaction Generation

Shaoqing Tan — Tue, 07 Apr 2026 07:29:11 -0700

Closed-loop egocentric video simulator maintaining persistent 3D scene state for consistent interactions, enabling cross-embodiment transfer from human videos to robotic manipulation.

Accelerating Video World Models: From Generative Videos to Real-Time Simulators

Shaoqing Tan — Mon, 06 Apr 2026 22:17:58 -0700

Comprehensive survey taxonomizing efficient architectures/algorithms for video world models as simulators, targeting compute bottlenecks in embodied AI, autonomous driving, and games with techniques like short-window attention for real-time long-horizon prediction.

From Tokens to Thoughts: Continuous Latent Reasoning in Large Models and Robot Control

Shaoqing Tan — Mon, 06 Apr 2026 22:14:05 -0700

Curated collection of 100+ works surveying shift to continuous latent spaces in LLMs/VLMs/VLAs for improved reasoning over discrete tokens, with relevance to robotics action modeling.

CaP-X: Coding Agents for Physical eXecution

Shaoqing Tan — Mon, 06 Apr 2026 07:11:45 -0700

CaP-X is an open-source agentic robotics framework where LLMs/VLMs generate code to call perception and control APIs for execution across diverse simulated and real robots in CaP-Gym's 187 manipulation tasks. The framework includes CaP-Bench for evaluating frontier models and CaP-RL, which boosts a 7B model's success from 20% to 72% with minimal sim-to-real gap.

DoRA: Weight-Decomposed Low-Rank Adaptation

Shaoqing Tan — Sun, 05 Apr 2026 22:30:22 -0700

An upgrade over LoRA for parameter-efficient fine-tuning, enabling better performance in LLMs by decomposing weights into magnitude and direction components.

AI Model Collapse: What Happens When AI Trains on Its Own Outputs

Shaoqing Tan — Sun, 05 Apr 2026 22:15:47 -0700

Seminal work showing how training on AI-generated data leads to 'model collapse' in neural networks, with urgent implications for future scaling.

PhAIL: Benchmarking Vision-Language-Action Models on Real-World Bin-Picking

Shaoqing Tan — Sun, 05 Apr 2026 07:19:40 -0700

Real-world hardware evaluation of VLAs on blind bin-to-bin picking, achieving max 64 picks/hour across hundreds of runs, with full videos/data exposing gaps in production-scale robotic manipulation reliability.

Co-training Large Behavior Models: Data Modalities and Training Strategies for Robot Manipulation

Shaoqing Tan — Sat, 04 Apr 2026 22:42:13 -0700

Comprehensive evaluation of 89 policies showing optimal co-training practices mixing real robot data with sim/egocentric human videos to boost diversity and performance in large robotics foundation models.

HyDRA: Hybrid Memory for Dynamic Video World Models

Shaoqing Tan — Sat, 04 Apr 2026 22:31:30 -0700

Novel memory system preserving dynamic object identity and motion continuity across occlusions in video world models, addressing frozen/vanishing issues for improved predictive physics in embodied AI.

# WildWorld: Dynamic World Modeling with Actions and Explicit State

Shaoqing Tan — Sat, 04 Apr 2026 07:29:53 -0700

Massive dataset enabling dynamic world models with explicit states and actions, supporting predictive modeling for cross-embodiment robotic control.

Omni-WorldBench: Evaluating Interactive 4D World Models

Shaoqing Tan — Sat, 04 Apr 2026 07:18:25 -0700

New benchmark assessing world models on interaction tasks, pushing predictive physics and video modeling towards robotics applications with action-conditioned evaluation.

SIMART: From Static Meshes to Sim-Ready Articulated Models

Shaoqing Tan — Fri, 03 Apr 2026 22:37:59 -0700

Unified MLLM framework with Sparse 3D VQ-VAE (70% token reduction) for part-level mesh decomposition and kinematic chain prediction, enabling physics-based robotic simulation from monolithic assets.

EgoSim: An Egocentric World Simulator for Embodied Interaction

Shaoqing Tan — Fri, 03 Apr 2026 22:23:00 -0700

Closed-loop egocentric simulator persistently updating 3D scene state to generate spatially consistent interaction videos for continuous simulation, enabling cross-embodiment transfer from human videos to robotic manipulation tasks.

Digit's New Motor Cortex: Sim-to-Real RL for Whole-Body Control

Shaoqing Tan — Fri, 03 Apr 2026 07:13:57 -0700

AI-trained capabilities for new whole-body motions using mocap/teleop data and sim-to-real reinforcement learning, deployable overnight on hardware.

EgoNav: Diffusion-Based Humanoid Navigation from Human Egocentric Video

Shaoqing Tan — Thu, 02 Apr 2026 22:32:11 -0700

Diffusion-based humanoid navigation trained solely on 5 hours of human egocentric video data, enabling zero-shot deployment on Unitree G1 for complex behaviors like handling glass walls, crowds, and dynamic obstacles via 360° visual memory and hybrid trajectory sampling; upcoming release of dataset, models, and code.

CaP-X: A Code-as-Policy Framework for Robot Manipulation

Shaoqing Tan — Thu, 02 Apr 2026 22:19:27 -0700

Comprehensive open-source agentic robotics framework treating VLMs/LLMs as code-generating APIs for perception (SAM3, Molmo) and control (IK, grasping), with CaP-Gym benchmark of 187 diverse manipulation tasks (tabletop, bimanual, mobile; sim/real) and CaP-Bench evaluating 12 frontier models; demonstrates rapid RL gains (7B model from 20% to 72% success) with strong sim-to-real transfer.

Embodied Intelligence Breakthrough: Generalist AI’s GEN-1 Robots

Shaoqing Tan — Thu, 02 Apr 2026 12:58:30 -0700

We've created GEN-1, our latest milestone in scaling robot learning. We believe it to be the first general-purpose AI model that crosses a new performance threshold: mastery of simple physical tasks. It improves average success rates to 99% on tasks where previous models achieve 64%, completes tasks roughly 3x faster than state of the art, and requires only 1 hour of robot data for each of these results. GEN-1 unlocks commercial viability across a broad range of applications—and while it cannot solve all tasks today, it is a significant step towards our mission of creating generalist intelligence for the physical world.

CaP-X: LMs' First Physical Exam

Shaoqing Tan — Thu, 02 Apr 2026 12:43:57 -0700

A novel benchmark that evaluates language models on physical examination tasks, testing their ability to understand and perform clinical physical exam procedures in simulated environments. This work introduces a comprehensive evaluation framework for AI systems in medical/clinical settings.

AI Model Collapse: The Danger of Training on AI-Generated Data

Shaoqing Tan — Tue, 31 Mar 2026 07:36:21 -0700

Demonstrated that LLMs trained recursively on AI-generated data suffer model collapse, a degenerative process where they lose grasp of true data distributions. Sparked critical debates on data provenance and the importance of preserving human-generated training data.

High-Level Automated Reasoning with Qwen2.5-7B

Shaoqing Tan — Tue, 31 Mar 2026 07:35:15 -0700

Qwen2.5-7B achieved 79.6% on MATH benchmark, surpassing GPT-4o, by employing atomic reasoning actions combined with Monte Carlo Tree Search. Demonstrated that strategic reasoning architectures can enable smaller models to outperform much larger ones.

Co-Training Large Behavior Models: Multimodal Data for Robot Manipulation

Shaoqing Tan — Mon, 30 Mar 2026 22:19:22 -0700

Explores data modalities and co-training strategies to enhance large behavior models (foundation models) for improved performance in robot manipulation tasks, supporting end-to-end learning and cross-embodiment generalization.

HyDRA: Hybrid Memory for Dynamic Video World Models

Shaoqing Tan — Sun, 29 Mar 2026 22:20:50 -0700

Memory architecture preserving identity and motion continuity for out-of-view dynamic subjects, addressing frozen/vanishing issues in video world models.

DexWM: Leveraging Human Videos for Dexterous Robot World Models

Shaoqing Tan — Sun, 29 Mar 2026 22:18:43 -0700

Dataset of robot trajectories designed for training world models to learn dexterous hand-object interactions directly from human videos.

World Models in Robotics

Shaoqing Tan — Sun, 29 Mar 2026 07:14:29 -0700

Technical survey categorizing world models into action-conditioned, video-inverse dynamics, and joint world-action models (WAMs), discussing their generalization, video data leverage, and trends for closing the robotics data gap.

SIMART: Decomposing Monolithic Meshes into Sim-Ready Articulated Assets

Shaoqing Tan — Sat, 28 Mar 2026 07:21:21 -0700

Unified MLLM framework with Sparse 3D VQ-VAE that reduces tokens by 70% for efficient part-level decomposition and kinematic prediction in physics-based robotic simulations.

LeWorldModel: A Stable JEPA World Model from Pixels

Shaoqing Tan — Fri, 27 Mar 2026 22:16:24 -0700

Stable end-to-end JEPA world model trained directly from pixels using simple MSE prediction loss and SIGReg anti-collapse regularization, enabling efficient latent planning under 1 second on 15M params with emergent spatial structure outperforming prior methods.

World Models for Robots: The Next Big Leap?

Shaoqing Tan — Fri, 27 Mar 2026 07:39:49 -0700

Technical overview defining world models in robotics, their potential to solve diverse problems via video prediction, and key enablers like scale.

Harnessing Long-Running AI in Embodied Systems

Shaoqing Tan — Fri, 27 Mar 2026 00:11:43 -0700

As AI moves from quick Q&A to marathon tasks, designers grapple with continuity. This episode explores how Anthropics harness design principles translate to embodied AI - robots that need to maintain context across long-running missions.

HoMMI: Learning Whole-Body Mobile Manipulation from Human Demonstrations

Shaoqing Tan — Wed, 25 Mar 2026 22:29:11 -0700

Whole-Body Mobile Manipulation Interface (HoMMI) that learns bimanual and whole-body manipulation, long-horizon navigation, and active perception directly from egocentric human demonstrations without teleoperation.

TurboQuant: Redefining AI Efficiency with Extreme Compression

Shaoqing Tan — Wed, 25 Mar 2026 17:52:48 -0700

This episode explores TurboQuant, a revolutionary set of quantization algorithms from Google Research that redefines AI efficiency through extreme compression.

We dive deep into how TurboQuant addresses one of AI's most pressing challenges: the memory bottleneck created by high-dimensional vectors in key-value caches. The research introduces theoretically grounded quantization methods that enable massive compression for large language models and vector search engines without sacrificing performance.

Key topics covered:

The theoretical foundations of TurboQuant's quantization algorithms
How extreme compression works for LLMs and vector search engines
Impact on high-dimensional vectors and key-value cache memory bottlenecks
Performance metrics and comparisons with existing methods
Practical implications for AI deployment and efficiency

Links:
Paper: https://arxiv.org/pdf/2504.19874
Blog: https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/

DexWM: Learning Dexterous Object Manipulation from Human Videos

Shaoqing Tan — Wed, 25 Mar 2026 07:19:46 -0700

Dataset of robot trajectories designed for training world models that learn dexterous hand-object interactions from human videos, released on Hugging Face.

FlashAttention-3: Fast & Accurate Attention with Asynchrony & Low-Precision

Shaoqing Tan — Tue, 24 Mar 2026 22:54:40 -0700

Major efficiency leap for Transformer attention mechanisms, enabling faster training/inference on long sequences with low-precision compute.

When AI Trains on Its Own Output: The Model Collapse Problem

Shaoqing Tan — Tue, 24 Mar 2026 22:39:06 -0700

Warns of "model collapse" in LLMs trained on synthetic data from prior models, urging preservation of human-generated data. One of 2024's most influential papers.

MolmoBot: A Vision-Language Model for Zero-Shot Robot Manipulation

Shaoqing Tan — Tue, 24 Mar 2026 07:22:35 -0700

Vision-language model (VLM) for zero-shot robot manipulation, trained entirely in simulation without real-world data; achieves 79.2% success rate on real-world tabletop tasks, outperforming π₀.₅ baseline at 39.2%.

LeWorldModel: Stable End-to-End JEPA from Pixels

Shaoqing Tan — Tue, 24 Mar 2026 01:12:52 -0700

A stable end-to-end Joint Embedding Predictive Architecture (JEPA) trained directly from pixels that enables robust world modeling for embodied AI systems.

EgoVerse: An Egocentric Data Ecosystem for Scaling Robot Learning

Shaoqing Tan — Mon, 23 Mar 2026 22:18:26 -0700

Ecosystem with over 1300 hours of egocentric human video data spanning 240 scenes and 2000+ tasks, designed for scalable robot policy training via behavior cloning; includes cloud infrastructure, data viewer, and human-to-robot transfer algorithms to enable cross-embodiment learning without teleoperation.

HSImul3R: Physics-Driven Reconstruction of Human–Scene Interactions

Shaoqing Tan — Mon, 23 Mar 2026 22:15:11 -0700

Physics-in-the-loop bi-directional optimization pipeline reconstructing stable, simulation-ready 3D human-scene interactions from casual videos, deployable directly to humanoid robots for world modeling and manipulation.

MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation

Shaoqing Tan — Mon, 23 Mar 2026 11:48:08 -0700

Open-source suite of large-scale simulation environments and benchmarks designed for advancing end-to-end learning in robot navigation and manipulation across multiple embodiments.

DreamZero: World Action Models Are Zero-Shot Policies

Shaoqing Tan — Mon, 23 Mar 2026 11:36:32 -0700

Introduces World Action Models (WAMs), a family of 14B-parameter autoregressive diffusion models that jointly predict video and robotic actions to enable zero-shot generalization across manipulation tasks, outperforming fine-tuned Vision-Language-Action models on benchmarks like MolmoSpaces and RoboArena.

Kinema4D: A 4D Generative Simulator for Embodied AI

Shaoqing Tan — Sun, 22 Mar 2026 19:16:00 -0700

An action-conditioned 4D generative robotic simulator that disentangles precise kinematic control from environmental dynamics, facilitating physically-plausible simulations of complex robot-world interactions for training and world modeling.

VEGA-3D: Teaching multimodal LLMs spatial reasoning through video generation

Shaoqing Tan — Sun, 22 Mar 2026 19:02:22 -0700

A plug-and-play framework extracts implicit 3D priors from video diffusion models to enhance multimodal LLMs with spatial reasoning capabilities, enabling improved geometric scene understanding and embodied decision-making without explicit 3D supervision.