期刊
ACM TRANSACTIONS ON GRAPHICS
卷 38, 期 4, 页码 -出版社
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3306346.3322958
关键词
hand tracking; hand pose estimation; two hands; depth camera; computer vision
资金
- ERC [770784, 772738]
- Marie Curie Individual Fellowship [707326]
- Marie Curie Actions (MSCA) [707326] Funding Source: Marie Curie Actions (MSCA)
- European Research Council (ERC) [772738] Funding Source: European Research Council (ERC)
We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands. Our approach is the first two-hand tracking solution that combines an extensive list of favorable properties, namely it is marker-less, uses a single consumer-level depth camera, runs in real time, handles inter- and intra-hand collisions, and automatically adjusts to the user's hand shape. In order to achieve this, we embed a recent parametric hand pose and shape model and a dense correspondence predictor based on a deep neural network into a suitable energy minimization framework. For training the correspondence prediction network, we synthesize a two-hand dataset based on physical simulations that includes both hand pose and shape annotations while at the same time avoiding inter-hand penetrations. To achieve real-time rates, we phrase the model fitting in terms of a nonlinear least-squares problem so that the energy can be optimized based on a highly efficient GPU-based Gauss-Newton optimizer. We show state-of-the-art results in scenes that exceed the complexity level demonstrated by previous work, including tight two-hand grasps, significant inter-hand occlusions, and gesture interaction.(1)
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据