Publication

Abstract : Skin cancer is one of the most prevalent and lifethreatening diseases, requiring accurate and early diagnosis for effective treatment. Traditional deep learning approaches, primarily based on Convolutional Neural Networks (CNNs), often struggle with generalization due to data variability, class imbalance, and lack of contextual information. To address these challenges, we propose a novel Hybrid CNN-Transformer Model with Multi-Modal Feature Fusion and Attention-Guided Explainability for skin cancer classification. The proposed approach combines the strengths of CNNs for local feature extraction and Vision Transformers (ViTs) for capturing global dependencies, leading to improved feature representation. Additionally, we incorporate multi-modal feature fusion, integrating dermoscopic images, clinical metadata (e.g., patient history, lesion location, lesion size), and textual descriptions to enhance diagnostic accuracy. An Attention-Guided Explainability Module (AGEM) is introduced to provide interpretability, highlighting critical regions of interest in the lesion images while correlating them with clinical attributes. The model is trained and evaluated on benchmark datasets such as ISIC, achieving superior performance in terms of accuracy, sensitivity, and specificity compared to existing state-of-the-art deep learning methods. The proposed framework not only enhances classification accuracy but also provides a more transparent and interpretable decision-making process, aiding dermatologists in real-world clinical applications.