PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging

Yuwei Li1,2,3,  Minye Wu1,2,3,  Yuyao Zhang1,  Lan Xu1,  Jingyi Yu1

1Shanghai Engineering Research Center of Intelligent Vision and Imaging, School of Information Science and Technology, ShanghaiTech University  
2Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences
3University of Chinese Academy of Sciences  

Abstract


Hand modeling is critical for immersive VR/AR, action understanding or human healthcare. Existing parametric models account only for hand shape, pose or texture, without modeling the anatomical attributes like bone, which is essential for realistic hand biomechanics analysis. In this paper, we present PIANO, the first parametric bone model of human hands from MRI data. Our PIANO model is biologically correct, simple to animate and differentiable, achieving more anatomically precise modeling of the inner hand kinematic structure in a data-driven manner than traditional hand model based on outer surface only. Furthermore, our PIANO model can be applied in neural network layers to enable training with a fine-grained semantic loss, which opens up the new task of data-driven fine-grained hand bone anatomic and semantic understanding from MRI or even RGB images.


[Arxiv] [Paper] [Video(Coming soon)]

MRI Dataset


Coming soon

PIANO Model


Coming soon

Bibtex


Please cite this paper in your publications if it helps your research:

@inproceedings{li2021piano,
title={PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging},
author={Yuwei, Li and Minye, Wu and Yuyao, Zhang and Lan, Xu and Jingyi, Yu},
booktitle={Proceedings of the 30th International Joint Conference on Artificial Intelligence, {IJCAI-21}},
year={2021}
}