Publication

Abstract : Medical image captioning is a field that combines medical imaging with natural language processing (NLP). Since the development of deep learning techniques in both NLP and medical image analysis, this area of study has gained significant progress. This research aims to develop a system for providing a brief description when given an image of chest x-ray using a GRU-based Attention Model, which gave better training and validation accuracy and BLEU scores when compared to a simple encoder-decoder, attention mechanism implemented with context flow and global flow and other state-of-the-art models using chexnet. The Chest X-Ray Images dataset, consisting of a large number of images of different chest x-rays suffering from pneumonia with different points of view (anterior-posterior), is utilized for training and validation of the model. The proposed model is trained using a combination of chest x-ray images along with their text features. The findings demonstrate the potential of using the GRU-based attention model for detection and classification of pneumonia, which can be applied for diagnosis in hospitals, clinics, emergency departments and screening programs