Back close

Partially Supervised Image Captioning Model for Urban Road Views

Publication Type : Book

Publisher : Springer, Singapore

Source : Intelligent Data Communication Technologies and Internet of Things, pp 59–73

Url :

Keywords : Instance Segmentation, Partially Supervised Model, MRCNN (Mask region based convolution neural networks) Framework

Campus : Coimbatore

School : School of Engineering

Department : Computer Science

Year : 2022

Abstract : Automatically generating a characteristic language portrayal of an image has pulled in interests in light of its significance in practical applications and on the grounds that it associates two significant artificial intelligence fields: natural language processing and computer vision. This paper proposes a partially supervised model for generating image descriptions based on instance segmentation labels. Instance segmentation, a combined approach of object detection and semantic segmentation is used for generating instance level labels which is then used for generating natural language descriptions for the image. The instance segmentation model uses MRCNN framework with feature pyramid networks and region proposal networks for object detection, and fully convolution layer for semantic segmentation. Information obtained from different local region proposals are used to generate region wise captions. Important aspects of the caption include distance, color and region calculations based on the results obtained from the instance segmentation layers. This paper uses instance segmentation layer information such as ROIs, class labels, probability scores and segmentation values for generating effective captions for the image. The proposed model is evaluated on Cityscape dataset where the primary objective is to provide semantic scene understanding based on the instances available in urban areas.

Cite this Research Publication : Srihari, K., Sikha, O.K. (2022). Partially Supervised Image Captioning Model for Urban Road Views. In: Hemanth, D.J., Pelusi, D., Vuppalapati, C. (eds) Intelligent Data Communication Technologies and Internet of Things. Lecture Notes on Data Engineering and Communications Technologies, vol 101. Springer, Singapore.

Admissions Apply Now