Publication

Abstract : Sign Language Recognition (SLR) plays a crucial role in enabling inclusive communication by translating sign gestures into spoken or written language. Despite notable progress, existing methods often struggle to capture both the fine-grained spatial details and the complex temporal dynamics of sign gestures, especially under rapid motion and subtle inter-class variations. To address these challenges, we propose Hybrid Attention Guided Framewise Temporal Reasoning for Sign Language Recognition (HAFTR-SLR), a tri-modal framework that jointly leverages RGB, skeleton, and optical flow modalities through a hybrid attention guided architecture. The RGB and optical flow streams capture appearance and motion cues, while the skeleton stream models structural hand–body dynamics. A Cross-Transformer Fusion mechanism aligns inter-modal dependencies, and a hybrid attention-based fusion emphasizes complementary features across modalities. Furthermore, an adaptive crucial frame selection strategy minimizes temporal redundancy while preserving salient motion information. Experimental evaluations on Include-50 and SMILE-DSGS-50 datasets demonstrate that HAFTR-SLR consistently outperforms recent state-of-the-art models across multiple metrics, confirming its robustness and efficiency in modeling multimodal gesture semantics.