Publications
(* indicates equal contribution)
|
|
Pretraining Knows What, Simulation Teaches How: Compositional Generalization for Agentic Robotics
Benjamin Pikus, Daniel Saunders, Mircea Mironenco, Salman Mohammadi, Mathew Mathew, Sirou Zhu, Divya Makkar, Xuhua Huang, Qinbo Li, Supreeth Narasimhaswamy, Siyuan Cen, Dejia Xu, Shyamal Buch, Thomas Neff
CoRL, 2026 (Late Breaking Results) NEW
project page / arXiv
TL;DR Train a VLM robot agent entirely in simulation that transfers zero-shot to a real bimanual robot (97% success), beating SOTA VLAs and matching GPT-6 Astra at 3× the speed and 1/17 the cost. Area: Physical AI, Agentic Robotics, Sim-to-Real, Reinforcement Learning
|
|
Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining
Junxuan Li*, Rawal Khirodkar*, Chengan He, Zhongshi Jiang, Giljoo Nam, ..., Peihong Guo, Xuhua Huang, Ariyan Zarei, ..., Jason Saragih, Yaser Sheikh, Shunsuke Saito
(authors list here)
CVPR, 2026 NEW
project page / arXiv
TL;DR Pretrain a 3D avatar model on 1M everyday videos, then fine-tune it on studio captures, so a casual phone capture of anyone becomes a realistic, fully animatable full-body avatar. Area: Generative Models, 3D Avatars, Large-scale Pretraining
|
|
Lidar Panoptic Segmentation and Tracking without Bells and Whistles
Abhinav Agarwalla, Xuhua Huang, Jason Ziglar, Francesco Ferroni, Laura Leal-Taixe, James Hays, Aljosa Osep, Deva Ramanan
IROS, 2023
project page / arXiv / code
TL;DR A simple object-centric approach that labels every LiDAR point and tracks each object over time for self-driving, performing strongly without extra tricks.
Area: Panoptic Segmentation, Lidar Segmentation and Tracking
|
|
Multiface: A Dataset for Neural Face Rendering
Cheng-hsin Wuu*, Ningyuan Zheng*, Scott Ardisson, Rohan Bali, Danielle Belko, Eric Brockmeyer, Lucas Evans, Timothy Godisart, Hyowon Ha, Xuhua Huang, Alexander Hypes, Taylor Koska, Steven Krenn, Stephen Lombardi, Xiaomin Luo, Kevyn McPhail, Laura Millerschoen, Michal Perdoch, Mark Pitts, Alexander Richard, Jason Saragih, Junko Saragih, Takaaki Shiratori, Tomas Simon, Matt Stewart, Autumn Trimble, Xinshuo Weng, David Whitewolf, Chenglei Wu, Shoou-I Yu, Yaser Sheikh
CVPR Workshop, 2023
project page / arXiv / code (600+ stars in Github)
TL;DR A large multi-view, high-resolution dataset of human faces captured in a Codec Avatar studio, with baselines for training photorealistic face renderers.
Area: Neural Face Rendering
|
|
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Ego4D Team (authors list here)
CVPR, 2022 (Oral, Best Paper Finalist)
project page / arXiv / code
TL;DR The largest first-person video dataset: 3,000+ hours from 74 locations in 9 countries, with benchmarks for understanding daily activities from the wearer's view.
Area: Egocentric Video Analysis
|
|
A Categorized Reflection Removal Dataset with Diverse Real-world Scenes
Chenyang Lei*, Xuhua Huang*, Chenyang Qi, Yankun Zhao, Wenxiu Sun, Qiong Yan, Qifeng Chen
CVPR Workshop, 2022
project page / arXiv / code
TL;DR A real-world reflection-removal dataset grouped by scene type and difficulty, showing where existing methods still fail.
Area: Computational Photography, Physics-based Vision
|
|
EgoAugment: CMU-KLAB Submission to EPIC-Kitchens 2021 Challenge
Xuhua Huang, Ye Yuan, Xingyu Liu, Qichen Fu, Kris M. Kitani
CVPR Workshop, 2021
report / challenge report / challenge page
TL;DR Adds an efficient transformer branch to a SlowFast network for first-person action recognition, outperforming SlowFast, TSM and other baselines on EPIC-Kitchens-100.
Area: Egocentric Video Analysis
|
|
Fast Video Object Segmentation with Temporal Aggregation Network and Dynamic Template Matching
Xuhua Huang*, Jiarui Xu*, Yu-Wing Tai, Chi Keung Tang
CVPR, 2020
project page /
arXiv
TL;DR Brings tracking-by-detection into video object segmentation with a temporal aggregation network and dynamic template matching, achieving SOTA speed and accuracy.
Area: Segmentation, Motion and Tracking
|
|
Polarized Reflection Removal with Perfect Alignment in the Wild
Chenyang Lei, Xuhua Huang, Mengdi Zhang, Qiong Yan, Wenxiu Sun, Qifeng Chen
CVPR, 2020
project page /
arXiv /
code
TL;DR Uses polarization cues and a physics-based network, plus real image pairs with perfect alignment, to remove glass reflections from real-world photos.
Area: Computational Photography, Physics-based Vision, Deep Learning
|
|
Research & Work Experiences
|
|
Luma AI, Inc.
Research Scientist in Redwood City, CA
Aug 2026 - Present | Team: Research
Topic: Physical AI, Agentic Robotics, World Action Models, Reinforcement Learning
|
|
Meta Platforms, Inc.
Senior Applied Research Scientist in Menlo Park, CA
Aug 2022 - Aug 2026 | Team: Meta Reality Labs, Codec Avatars
May 2021 - Aug 2021 | Team: Meta Reality Labs (R&D intern)
Topic: Large-scale Multimodal Generative Models, 3D Avatars, AR/VR
|
|
Argo AI
Research Collaborator in Pittsburgh, PA
Aug 2021 - May 2022 | Advisor: Prof. Deva Ramanan
Topic: Lidar Panoptic Segmentation, Autonomous Driving
|
|
SenseTime Group Limited
Research Intern in Hong Kong
Feb 2020 - Dec 2020 | Advisor: Dr. Qiong Yan
Mar 2019 - Nov 2019 | Advisor: Prof. Qifeng Chen
Topic: Image Auto Cropping, Reflection Removal, Low-level Vision
|
|
Palo Alto Networks
SWE Intern in Mountain View, CA
Sep 2018 - Jun 2019 | Advisor: Dr. May Wang and Ran Xia
Topic: Big Data Analytics, Cloud Development
|
|
Tencent YouTu Lab
Research Intern in Shenzhen
May 2018 - Aug 2018 | Advisor: Prof. Yu-Wing Tai
Topic: Object Detection, Network Pruning
|
|
Microsoft Corporation
SWE Intern in Hong Kong
May 2017 - Aug 2017
Topic: Artificial Intelligence, SQL Database, Azure Development
|
|
HKUST Robotics Institute
Research Assistant in Hong Kong
Dec 2016 - Sep 2017 | Advisor: Prof. Kam Tim Woo
Topic: Intelligent Car Design and Development
|
|
Professional Activities
Conference Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICLR, ICML, AAAI, IROS, ACCV
Journal Reviewer: TPAMI, IJCV, TNNLS, TIP, TMM, TCSVT, IEEE Signal Processing Letters, IEEE Transactions on Big Data,
Signal Processing (Elsevier), Neural Networks (Elsevier), Science China Information Sciences, Frontiers of Computer Science
|
© 2026 Xuhua Huang · Last updated: Nov 2026
He gives a great template.
|
|