DorsalNet : 3D Hand Pose Estimation

Researchers at XRTC have made an advancement in wearable technology by developing a wrist-worn camera system capable of interpreting human finger movements with high precision in real-time. This is significant as it eliminates the need for extra devices attached to the fingers, commonly required in previous technologies. These technologies are used in several mainstream systems such as gaming, VR, Motion Capture in Film and Animation, and Sign Language.

The key challenge that the researchers addressed were the limited and often obstructed view of the fingers due to the camera's position. To fix this, they developed 'DorsalNet', an advanced neural network. This "neural network" is a type of artificial intelligence technology inspired by the structure and function of the human brain. This was designed to analyze the back (dorsal side) of the hand, paying close attention to the subtle movements and deformations of the skin, muscles, and tendons. The detailed analysis enables the system to predict the positions and movements of the finger joints with accuracy.

The system is capable of estimating the angle of each finger joint with an average error of less than 10 degrees. This level of accuracy marks a substantial improvement over existing methods, enhancing the ability to recognize dynamic hand gestures by 20% and achieving a 75% success rate in identifying different types of hand grips.

DorsalNet has a wide range of uses. It can work like a remote control, allowing users to make hand gestures enabling different features on mobile devices or computers. Imagine changing settings or navigating menus just by moving your hands. It can replace traditional tools like a mouse or keyboard. Instead of clicking or typing, you can simply use hand movements to give commands to your computer. Lastly, DorsalNet has the capability to recognize different objects based on how you hold them. 

DorsalNet is changing the way we view mobile computing, virtual reality, and assistive technology sectors. It paves the way for more intuitive, natural, and user-friendly interactions with technological devices, aligning them more closely with human behavior and movements.

For those interested in exploring this research in more depth, the comprehensive study can be accessed through its DOI link: https://dl.acm.org/doi/10.1145/3379337.3415897

Researchers: Erwin Wu, Ye Yuan, Hui-Shyong Yeo, Aaron J Quigley, Hideki Koike, Kris Kitani