Xuhua Huang

I am a Research Scientist at Luma AI, focusing on Physical AI. Currently cooking up some exciting work on agentic robot policies and world action models (WAMs).

Before Luma, I was a Senior Applied Research Scientist at Meta for four years, building large-scale multimodal generative models for Codec Avatars. I was a graduate student at Robotics Institute of Carnegie Mellon University, advised by Prof. Kris Kitani and Prof. Deva Ramanan. I received my Bachelor's degree from Hong Kong University of Science and Technology (First Class Honors), where I worked with Prof. Chi Keung Tang, Prof. Qifeng Chen and Prof. Yu-Wing Tai.

My interests include physical AI, robot learning, world models, multimodal generative models and 3D vision.

Email  /  Google Scholar  /  LinkedIn

profile photo
Publications
(* indicates equal contribution)
Pretraining Knows What, Simulation Teaches How: Compositional Generalization for Agentic Robotics
Benjamin Pikus, Daniel Saunders, Mircea Mironenco, Salman Mohammadi, Mathew Mathew, Sirou Zhu, Divya Makkar, Xuhua Huang, Qinbo Li, Supreeth Narasimhaswamy, Siyuan Cen, Dejia Xu, Shyamal Buch, Thomas Neff
CoRL, 2026  (Late Breaking Results)  NEW
project page / arXiv

TL;DR Train a VLM robot agent entirely in simulation that transfers zero-shot to a real bimanual robot (97% success), beating SOTA VLAs and matching GPT-6 Astra at 3× the speed and 1/17 the cost.

Area: Physical AI, Agentic Robotics, Sim-to-Real, Reinforcement Learning

Large-scale Codec Avatars: a gallery of generated full-body avatars Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining
Junxuan Li*, Rawal Khirodkar*, Chengan He, Zhongshi Jiang, Giljoo Nam, ..., Peihong Guo, Xuhua Huang, Ariyan Zarei, ..., Jason Saragih, Yaser Sheikh, Shunsuke Saito (authors list here)
CVPR, 2026  NEW
project page / arXiv

TL;DR Pretrain a 3D avatar model on 1M everyday videos, then fine-tune it on studio captures, so a casual phone capture of anyone becomes a realistic, fully animatable full-body avatar.

Area: Generative Models, 3D Avatars, Large-scale Pretraining

Lidar Panoptic Segmentation and Tracking without Bells and Whistles
Abhinav Agarwalla, Xuhua Huang, Jason Ziglar, Francesco Ferroni, Laura Leal-Taixe, James Hays, Aljosa Osep, Deva Ramanan
IROS, 2023
project page / arXiv / code

TL;DR A simple object-centric approach that labels every LiDAR point and tracks each object over time for self-driving, performing strongly without extra tricks.

Area: Panoptic Segmentation, Lidar Segmentation and Tracking


Multiface: A Dataset for Neural Face Rendering
Cheng-hsin Wuu*, Ningyuan Zheng*, Scott Ardisson, Rohan Bali, Danielle Belko, Eric Brockmeyer, Lucas Evans, Timothy Godisart, Hyowon Ha, Xuhua Huang, Alexander Hypes, Taylor Koska, Steven Krenn, Stephen Lombardi, Xiaomin Luo, Kevyn McPhail, Laura Millerschoen, Michal Perdoch, Mark Pitts, Alexander Richard, Jason Saragih, Junko Saragih, Takaaki Shiratori, Tomas Simon, Matt Stewart, Autumn Trimble, Xinshuo Weng, David Whitewolf, Chenglei Wu, Shoou-I Yu, Yaser Sheikh
CVPR Workshop, 2023
project page / arXiv / code (600+ stars in Github)

TL;DR A large multi-view, high-resolution dataset of human faces captured in a Codec Avatar studio, with baselines for training photorealistic face renderers.

Area: Neural Face Rendering


Ego4D: Around the World in 3,000 Hours of Egocentric Video
Ego4D Team   (authors list here)
CVPR, 2022   (Oral, Best Paper Finalist)
project page / arXiv / code

TL;DR The largest first-person video dataset: 3,000+ hours from 74 locations in 9 countries, with benchmarks for understanding daily activities from the wearer's view.

Area: Egocentric Video Analysis


A Categorized Reflection Removal Dataset with Diverse Real-world Scenes
Chenyang Lei*, Xuhua Huang*, Chenyang Qi, Yankun Zhao, Wenxiu Sun, Qiong Yan, Qifeng Chen
CVPR Workshop, 2022
project page / arXiv / code

TL;DR A real-world reflection-removal dataset grouped by scene type and difficulty, showing where existing methods still fail.

Area: Computational Photography, Physics-based Vision

EgoAugment: CMU-KLAB Submission to EPIC-Kitchens 2021 Challenge
Xuhua Huang, Ye Yuan, Xingyu Liu, Qichen Fu, Kris M. Kitani
CVPR Workshop, 2021
report / challenge report / challenge page

TL;DR Adds an efficient transformer branch to a SlowFast network for first-person action recognition, outperforming SlowFast, TSM and other baselines on EPIC-Kitchens-100.

Area: Egocentric Video Analysis

Fast Video Object Segmentation with Temporal Aggregation Network and Dynamic Template Matching
Xuhua Huang*, Jiarui Xu*, Yu-Wing Tai, Chi Keung Tang
CVPR, 2020
project page / arXiv

TL;DR Brings tracking-by-detection into video object segmentation with a temporal aggregation network and dynamic template matching, achieving SOTA speed and accuracy.

Area: Segmentation, Motion and Tracking

Polarized Reflection Removal with Perfect Alignment in the Wild
Chenyang Lei, Xuhua Huang, Mengdi Zhang, Qiong Yan, Wenxiu Sun, Qifeng Chen
CVPR, 2020
project page / arXiv / code

TL;DR Uses polarization cues and a physics-based network, plus real image pairs with perfect alignment, to remove glass reflections from real-world photos.

Area: Computational Photography, Physics-based Vision, Deep Learning

Research & Work Experiences
Luma AI Luma AI, Inc.
Research Scientist in Redwood City, CA

Aug 2026 - Present | Team: Research

Topic: Physical AI, Agentic Robotics, World Action Models, Reinforcement Learning

Meta Platforms, Inc.
Senior Applied Research Scientist in Menlo Park, CA

Aug 2022 - Aug 2026 | Team: Meta Reality Labs, Codec Avatars
May 2021 - Aug 2021 | Team: Meta Reality Labs (R&D intern)

Topic: Large-scale Multimodal Generative Models, 3D Avatars, AR/VR

Argo AI
Research Collaborator in Pittsburgh, PA

Aug 2021 - May 2022 | Advisor: Prof. Deva Ramanan

Topic: Lidar Panoptic Segmentation, Autonomous Driving

SenseTime Group Limited
Research Intern in Hong Kong

Feb 2020 - Dec 2020 | Advisor: Dr. Qiong Yan
Mar 2019 - Nov 2019 | Advisor: Prof. Qifeng Chen

Topic: Image Auto Cropping, Reflection Removal, Low-level Vision

Palo Alto Networks Palo Alto Networks
SWE Intern in Mountain View, CA

Sep 2018 - Jun 2019 | Advisor: Dr. May Wang and Ran Xia

Topic: Big Data Analytics, Cloud Development

Tencent Tencent YouTu Lab
Research Intern in Shenzhen

May 2018 - Aug 2018 | Advisor: Prof. Yu-Wing Tai

Topic: Object Detection, Network Pruning

Microsoft Corporation
SWE Intern in Hong Kong

May 2017 - Aug 2017

Topic: Artificial Intelligence, SQL Database, Azure Development

HKUST Robotics Institute
Research Assistant in Hong Kong

Dec 2016 - Sep 2017 | Advisor: Prof. Kam Tim Woo

Topic: Intelligent Car Design and Development

Professional Activities

Conference Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICLR, ICML, AAAI, IROS, ACCV
Journal Reviewer: TPAMI, IJCV, TNNLS, TIP, TMM, TCSVT, IEEE Signal Processing Letters, IEEE Transactions on Big Data, Signal Processing (Elsevier), Neural Networks (Elsevier), Science China Information Sciences, Frontiers of Computer Science

Education
Carnegie Mellon University

2020-2022 M.S. in Computer Vision (GPA: 4.0/4.0)

Hong Kong University of Science and Technology

2015-2020 B.Eng. in Computer Science (First Class Honors, Dean's List)

Washington University

2018 Exchange in Computer Science (Spring)

Tsinghua University

2017 Exchange in Computer Science (Summer)

Other Projects
Neural Architecture Search with Bayesian Optimization
Tech Report, 2018
project page / report / code

Advisor: Prof. Roman Garnett

Area: Bayesian Optimization, Machine Learning

Neural Network Pruning and Object Detection Model Optimization
Tech Report, 2018
project page / code

Advisor: Prof. Yu-Wing Tai

Area: Object Detection, Network Optimization

Sokoban Game Solver with Search Algorithms and Reinforcement Learning (Q-Learning)
Tech Report, 2017
project page / report / code

Advisor: Prof. Yangqiu Song

Area: Reinforcement Learning


© 2026 Xuhua Huang  ·  Last updated: Nov 2026
He gives a great template.