Recovering high-quality 3D human motion in complex scenes from monocular videos is important for many appli- cations, ranging from AR/VR to robotics. However, captur- ing realistic human-scene interactions, while dealing with occlusions and partial views, is challenging; current ap- proaches are still far from achieving compelling results. We address this problem by proposing LEMO: LEarning human MOtion priors for 4D human body capture. By leverag- ing the large-scale motion capture dataset AMASS [38], we introduce a novel motion smoothness prior, which strongly reduces the jitters exhibited by poses recovered over a se- quence. Furthermore, to handle contacts and occlusions occurring frequently in body-scene interactions, we design a contact friction term and a contact-aware motion infiller obtained via per-instance self-supervised training. To prove the effectiveness of the proposed motion priors, we com- bine them into a novel pipeline for 4D human body cap- ture in 3D scenes. With our pipeline, we demonstrate high- quality 4D human body capture, reconstructing smooth mo- tions and physically plausible body-scene interactions.
The code and data are available at [ Ссылка ].
