Publication

Abstract : This paper introduces a real-time hand gesture recognition system for manipulating instrumental sounds using the LabVIEW-Python framework. MediaPipe is utilized for recognizing hand landmarks in real-time and classifying the recognized gestures using a Random Forest Classifier(RFC). With a custom data set involving 14 different classes of gestures, an accuracy of 92% is achieved. Various machine learning models are also investigated, and RFC is found to have the best accuracy and inference time relative to other models. The recognized gestures that are mapped to musical notes are observed through LabVIEW to create an interactive audio experience. The observed system latency is around 4 ms, and therefore, the proposed system is potentially suitable for real-time interventions in music education, digital musical performance, and music therapy. LabVIEW opens the opportunity to interface hardware systems for music synthesis and human-centric applications.