Enhancing Teaching and Learning of Vocational Skills through Machine Learning and Cognitive Training (MCT)

Dept/Center/Lab: AMMACHI Labs- Amrita Multi Modal Applications Using Computer & Human Interaction

Thematic Area: SDG 9 Industry, Innovation, and Infrastructure, SDG 5 Gender Equality, SDG 8 Livelihood, SDG 4 Education

Project Incharge:Dr. Bhavani R. Rao - Director, AMMACHI Labs

Co-Project Incharge:Dr. Sidney Strauss, Emeritus Professor, Tel Aviv University; Dr. Praveen Pankajakshan, Chief AL Scientist, Urban Kisaan; Gayathri Manikutty, Senior Researcher, AMMACHI Labs

Funded by:Amrita Vishwa Vidyapeetham, Department of Science and Technology

Enhancing Teaching and Learning of Vocational Skills through Machine Learning and Cognitive Training (MCT)

For the past 15 years, AMMACHI labs has been training women on several vocational trades to enhance their skills, tools, and market reach. This research project complements the initiative by providing neurocognitive data-driven insights on dexterous vocational skill performance specifically on tailoring skills.

We propose to develop a multi-modal system architecture blending the sub-domains in Cognitive Sciences and Artificial Intelligence (AI). We capture multi-modal data from various sources, and analyse the data to better understand what are the key elements that characterize a particular skill, and how these key elements of the skill are effectively transferred. By effective transfer we mean, both aspects, the teaching and the learning of the skills. To achieve this, we have identified three main components that form the structure of the proposed work:

Capture and identify the components that constitute human skills, both the physical and the cognitive aspects, to accurately determine skill level of learners
Develop models to understand what inhibits learning of a new skill based on the cognitive load theory (CLT)
Design cognitive training (CT) strategies based on the above understanding to provide effective instruction to learners for continuous skill improvement.

Two main goals of our R&D program energize our work. The first goal is scientific. We want to better understand skills development with the use of technologies that allow those understandings to have depth, precision and sophistication. And it is our hope that what we will learn in our work can serve as a platform for developing richer models of skill development. The second goal is social. We wish to harness those scientific understandings so as to foster skills development so that people who have been taught skills can become gainfully employed

Project Background & Motivation

The core problem is the lack of accessible, effective hands-on vocational training for rural, low-literate, and marginalized communities, particularly in manual dexterity-based skills. This significantly limits employment opportunities for women and informal sector workers, who make up a large portion of India’s unskilled labor force. Through more than a decade of engagement across 21 Indian states, AMMACHI Labs has observed that existing training methods are often inaccessible, inconsistent, and poorly aligned with learners’ cognitive and physical capabilities.

The broader challenge is to design technology-based interventions that not only teach vocational skills but also enhance the precision and understanding of skill development trajectories. This involves two key issues:

Developing a multi-modal system architecture that captures and classifies skill levels using objectives skill performance data captured through emerging technologies, as well as qualitative data from ethnographic studies.
Designing cognitive training interventions grounded in validated learning models to effectively support cross-domain skill learning for learners with varying skill levels.

Addressing these challenges will enable the creation of a scalable, AI-driven training ecosystem that supports personalized instruction, promotes behavioral change, and facilitates certifiable skill acquisition, ultimately bridging a critical workforce development gap.

Our solution is a multimodal, AI-driven cognitive training system that enhances vocational skill learning through:

Psychometric measurements of skill performance using standardized dexterity tests.
Machine learning-based skill classification and modeling using wearable sensors for fine grained movement classification.
Development of custom datasets and AI models to detect micro-actions relevant to skill performance.
EEG and psychometric-based measurement of cognitive load, attention, working memory capacity, schemas, motivation, metacognition, and emotional states and their interactions with memory systems (sensory, working, long-term) during skill training.
Development of cognitive training (CT) interventions tailored to skill levels and learner profile.

Our intervention targets both the teaching and learning components of vocational education, and involves hardware tools, data acquisition devices, and cloud computation for deep learning models.

AI models are being trained to differentiate skill patterns across proficiency levels from novices to experts which enables targeted feedback and automatic classification of skill levels
A closed-loop system is being designed where data captured from learners informs ML models which generate insights for instructional re-design, thereby creating a continuously improving AI-in-the-loop learning ecosystem.
The analytical framework and models can be extended to other vocational domains requiring complex psychomotor skills (such as surgery, manufacturing, construction), making it a cross domain platform technology for skill assessment and development.
The system is being designed to create a dynamic AI based skill progression map by incorporating measures of cognitive load and learner understanding over time. This is to help instructors and learners identify stagnation points or overload risks allowing for real-time adaptive interventions.

Project Leadership & Collaborators

Lead Institution:

AMMACHI Labs – Amrita Vishwa Vidyapeetham

Collaborating Partners:

Dr. Veena A. Nair – Professor, University of Wisconsin-Madison

Team Members / Roles:

Arjun Venugopal – Research Assistant, AMMACHI Labs
Kancherla Yeswanth Chowdary – Junior Research Associate, Machine learning/ Deep Learning, AMMACHI Labs

Field

Cognitive Science, Skill Development

Funding Amount

₹ 35.41 Lakhs INR

Project Duration

19th March 2024 to 19th February 2028

Project Implementation and Key Activities

Towards Objective 1, we are assessing differences in psychomotor skill on manipulative dexterity tests of experts and novices

The tailoring profession was selected as the vocational context of interest.
We conducted a detailed task analysis using which we identified 31 psychomotor and cognitive abilities relevant to tailoring, based on standard occupational classification (SOC) taxonomy frameworks. Out of these, five key abilities related to dexterity were selected for scientific study.
We are collecting objective movement data during task performance using a motion capture device for fine grained kinematic analysis.

Future Plan and Scalability

In the next phase of this research, we will use EEG based measurements to gain insights into the cognitive demands of skill performance to further clarify how experience influences cognitive processing. Further we will look at the effect of cognitive training on the cognitive load of participants during task performance.