Back close

Hand Tracking in 3D Space using MediaPipe and PnP Method for Intuitive Control of Virtual Globe

Publication Type : Conference Paper

Publisher : IEEE

Source : 2021 IEEE 9th Region 10 Humanitarian Technology Conference (R10-HTC)

Url :

Keywords : Human-Computer Interaction,Hand - Tracking,MediaPipe,OpenCV,Cesium,Calibration,PnP

Campus : Bengaluru

School : School of Engineering

Department : Mechanical

Year : 2021

Abstract : Thereis a need for new interactive techniques to engage students with academic content. One of the hardest concept students deal with is using the Atlas and maps to understand the Geography, Social Sciences, and History of the world. The current alternative to the Atlas is accessing virtual globes on computers but controlling the globe is done using a mouse or touchscreen. The interaction may turn monotonous with usage. To address this problem, the authors propose a methodology to track a hand using a camera feed of a computer, while further using hand gestures to manipulate a virtual globe. Google's MediaPipe library is employed, along with OpenCV's pose estimation functions that use PnP (Perspective-n-Point) method. Later, an interactive globe environment has been implemented on a web browser using Cesium which can be commanded to perform rotate, pan and zoom on virtual globe and 2D maps as per the user's hand position and gestures. Results regarding the accuracy of the hand tracking system and robustness of control of virtual globe are also reported here.

Cite this Research Publication : Vaishnav Chunduru (2017-21), Mrinalkanti Roy (2017-21), Dasari Romit N. S. (2017-21) and Rajeevlochana G. Chittawadigi, has presented a paper titled, “Hand Tracking in 3D Space using MediaPipe and PnP Method for Intuitive Control of Virtual Globe” 9th IEEE Region 10 Humanitarian Technology Conference (R10 HTC) during Sept. 30 to Oct 2, 2021, held in hybrid mode (online and physical at Bengaluru). Conference proceedings in IEEE Xplore.

Admissions Apply Now