HUM4D provides synchronized multi-view RGB-D sequences aligned with professional Vicon motion capture ground truth, designed to benchmark markerless human motion capture under severe occlusion and multi-person interactions.
The dataset includes challenging scenarios such as
Jittering,
Identity Switching,
Occlusion, and
Near-Far Interaction.
Pipeline. Multi-view RGB-D capture is synchronized with Vicon motion capture. Marker trajectories are reconstructed and retargeted to SMPL to produce pose (θ), shape (β), and translation (t), along with evaluation-ready annotations.
Capture Environment. Professional motion capture studio with 44 synchronized infrared Vicon cameras and a multi-view RGB-D setup.
Hardware Setup. From left to right: RGB-D camera perspective layout (1.45 m height), top-view circular arrangement (3 m radius), Intel RealSense D455 sensor, and the Vicon motion capture system.
Dataset structure. HUM4D provides synchronized RGB-D sequences aligned with marker-based MoCap ground truth. We release evaluation-ready annotations and organized data in a hierarchical structure for easy navigation.
Behind the scenes. Footage from the HUM4D recording sessions, illustrating the multi-sensor setup and multi-person interactions.
You can download the dataset from the following links:
https://https://https://https://
For data that are not publicly available but are included in HUM4D, contact us at
cszghp [at] gmail.com.
For questions, please contact cszghp [at] gmail.com.
The authors would like to thank Michael Walsh for his assistance with the human motion capture acquisition at the RELLIS Starlab facility at Texas A&M University (TAMU). We also thank Morgan Jenks for managing and operating the Vicon motion capture system and for overseeing all aspects of data acquisition. We also acknowledge valuable discussions and feedback from John Keyser from the Department of CSCE at TAMU. Additionally, we thank Jyothi Naidu for support in facilitating the IRB approval process.
@inproceedings{park2026hum4d,
title={A Dataset and Evaluation for Complex 4D Markerless Human Motion Capture},
author={Park, Yeeun and Naduthodi, Miqdad and Kumar, Suryansh},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)},
year={2026}
}