PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging

IJCAI 2021

Yuwei Li1,2,3, Minye Wu1,2,3, Yuyao Zhang1, Lan Xu1, Jingyi Yu1

1School of Information Science and Technology, ShanghaiTech University
2Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences
3University of Chinese Academy of Sciences

Paper Code Data

Hand modeling is critical for immersive VR/AR, action understanding or human healthcare. Existing parametric models account only for hand shape, pose or texture, without modeling the anatomical attributes like bone, which is essential for realistic hand biomechanics analysis. In this paper, we present PIANO, the first parametric bone model of human hands from MRI data. Our PIANO model is biologically correct, simple to animate and differentiable, achieving more anatomically precise modeling of the inner hand kinematic structure in a data-driven manner than traditional hand model based on outer surface only. Furthermore, our PIANO model can be applied in neural network layers to enable training with a fine-grained semantic loss, which opens up the new task of data-driven fine-grained hand bone anatomic and semantic understanding from MRI or even RGB images.


We present the first statistical internal hand bone model. Our model enables anatomically and physically precise modeling of the hand kinematic structure of different individuals, opening up the study of data-driven hand bone anatomic and semantic fine-grained understanding from MRI or even RGB images.


@inproceedings{li21piano, title = {PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging}, author = {Li, Yuwei and Wu, Minye and Zhang, Yuyao and Xu, Lan and Yu, Jingyi}, booktitle = {Proceedings of the Thirtieth International Joint Conference on
Artificial Intelligence, {IJCAI-21}}, pages = {816--822}, year = {2021}, month = {8}, doi = {10.24963/ijcai.2021/113}, url = {https://doi.org/10.24963/ijcai.2021/113} }