Shaoxiong Yao

I am a research scientist at Dyna, working on dexterous manipulation. I recently received my PhD in Computer Science from University of Illinois at Urbana-Champaign, where I was advised by Prof. Kris Hauser. I have dual undergraduate degrees in Computer Science from University of Michigan and Electrical and Computer Engineering from Shanghai Jiao Tong University.

CV / Biography

Research Interests

My research focuses on developing physically grounded models for robotic manipulation through multimodal real-world observations, including vision, touch, and language, with works in Robotic Manipulation [CVPR 2026, ICRA 2026 Workshop, ICRA 2025], Multimodal Perception [ICRA 2023, CoRL 2024, ICRA 2025] and Physical Modeling [ICRA 2024].

Publications

	Guava: Distilling Frontier VLMs into a Compact Agent through a Robotic Manipulation Harness Haowen Liu, Xirui Li, Shaoxiong Yao, Peng Shi, Tianyi Zhou, Jia-Bin Huang, Furong Huang and Jiayuan Mao arXiv preprint*, 2026 arXiv / Project Webpage / Code vision-language modelstool-using agentsdistillationsim-to-real
	Dexterous Non-prehensile Manipulation using Environment Guided Diffusion Models Shaoxiong Yao, Chengyi Xing, Yuki Shirari, Devesh Jha and Diego Romeres ICRA 2026 Workshop on Dexterity with Multifingered Hands: Hardware, Sensing, and Skills, 2026 Paper dexterous manipulationnon-prehensile manipulationguided diffusion
	SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models Haowen Liu, Shaoxiong Yao*, Haonan Chen, Jiawei Gao, Jiayuan Mao, Jia-Bin Huang and Yilun Du IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2026 arXiv / Project Webpage / Code vision-language modelsreal-to-simphysical reasoningmanipulation planning
	Estimating High-Resolution Neural Stiffness Fields using Visuotactile Sensors Jiaheng Han, Shaoxiong Yao** and Kris Hauser International Conference on Robotics and Automation (ICRA), 2025 Paper / PDF / Project Webpage / Code tactile perceptionneural fieldsdeformable object modeling
	Safe Leaf Manipulation for Accurate Shape and Pose Estimation of Occluded Fruits Shaoxiong Yao, Sicong Pan, Maren Bennewitz and Kris Hauser International Conference on Robotics and Automation (ICRA), 2025 arXiv / Project Webpage / Code / Video agricultural roboticsdeformable object manipulationshape completion
	Generating and Estimating Linear FEM Plant Simulation Models Shaoxiong Yao and Kris Hauser IROS2024 Workshop on Agricultural Robotics for a Sustainable Future, 2024 PDF / Code (WIP) physical modelingFEM simulationagricultural robotics
	Structured Bayesian Meta-Learning for Data-Efficient Visual-Tactile Model Estimation Shaoxiong Yao, Yifan Zhu and Kris Hauser Conference on Robot Learning (CoRL), 2024 Paper / PDF / Project Webpage / Code meta-learningvisual-tactile perceptiondeformable object modeling
	3D Force and Contact Estimation for a Soft-Bubble Visuotactile Sensor Using FEM Jing-Chen Peng, Shaoxiong Yao and Kris Hauser International Conference on Robotics and Automation (ICRA), 2024 PDF / Code tactile sensingFEMcontact force estimation
	Estimating Tactile Models of Heterogeneous Deformable Objects in Real Time Shaoxiong Yao and Kris Hauser International Conference on Robotics and Automation (ICRA), 2023 PDF / Code / Earlier workshop version (ICRA 2022) tactile perceptiondeformable object modelingreal-time estimation