Back close

Cross-Attention Aggregation: A Novel Approach for Contextual Similarity Using Multi-model Embeddings

Publication Type : Conference Paper

Publisher : Springer Nature Switzerland

Source : Communications in Computer and Information Science

Url : https://doi.org/10.1007/978-3-032-16830-6_3

Campus : Coimbatore

School : School of Computing

Year : 2026

Abstract : Textual similarity measures are an actively open issue in the field of Natural Language Processing (NLP), specifically while comparing varied texts with varied contexts. In classical approaches, embeddings are typically obtained from a single model or a concatenation of multiple models, thereby limiting the ability to represent complex interaction among a large number of contextual representations. This paper proposes a novel method approach, cross-attention aggregation to dynamically combine embedding from multiple models which creates a unified representation that captures both local and global contexts while retaining semantic meaning. The proposed methodology uses a combined text corpus of academic content, such as textbooks and video transcripts, to evaluate and improve context matching. The findings show that a reliable similarity score is calculated by voting mechanism, which aggregates the contributions from multiple models, produces a reliable similarity score. This similarity score captures relevance between video content and educational material, thereby exhibiting potential ensemble-based approaches for more reliable measures of similarity in NLP tasks.

Cite this Research Publication : C. Arunkumar, B. A. Sabarish, S. B. Apoorva, M. Shiva, K. M. Navaneeth, K. Nithya, Cross-Attention Aggregation: A Novel Approach for Contextual Similarity Using Multi-model Embeddings, Communications in Computer and Information Science, Springer Nature Switzerland, 2026, https://doi.org/10.1007/978-3-032-16830-6_3

Admissions Apply Now