Towards unconstrained joint hand-object reconstruction from RGB videos
Abstract
Our work aims to obtain 3D reconstruction of hands and manipulated objects from monocular videos. Reconstructing hand-object manipulations holds a great potential for robotics and learning from human demonstrations. The supervised learning approach to this problem, however, requires 3D supervision and remains limited to constrained laboratory settings and simulators for which 3D ground truth is available. In this paper we first propose a learning-free fitting approach for hand-object reconstruction which can seamlessly handle two-hand object interactions. Our method relies on cues obtained with common methods for object detection, hand pose estimation and instance segmentation. We quantitatively evaluate our approach and show that it can be applied to datasets with varying levels of difficulty for which training data is unavailable.
Paper
Code
If you find this work interesting, you will be no doubt interested by the following publication: Reconstructing Hand-Object Interactions in the Wild, Cao et al., ICCV 2021BibTeX
@INPROCEEDINGS{hasson20_handobjectconsist,
title = {Towards unconstrained joint hand-object reconstruction from RGB videos},
author = {Hasson, Yana and Varol, G{\"u}l and Laptev, Ivan and Schmid, Cordelia},
booktitle = {ArXiv},
year = {2021}
}
Acknowledgements
This work was funded in part by the MSR-Inria joint lab, the French government under management of Agence Nationale de la Recherche as part of the ”Investissements d’avenir” program, reference ANR19-P3IA-0001 (PRAIRIE 3IA Institute) and by Louis Vuitton ENS Chair on Artificial Intelligence.
Copyright Notice
The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright.