Back close

Convolution-Based Encoding of Depth Images for Transfer Learning in RGB-D Scene Classification

Publication Type : Journal Article

Publisher : Sensors, MDPI

Source : Sensors, MDPI, 21(23):7950, 2021, (IF: 3.576)

Url :

Campus : Bengaluru

School : Department of Computer Science and Engineering

Department : Computer Science

Year : 2021

Abstract : Classification of indoor environments is a challenging problem. The availability of low-cost depth sensors has opened up a new research area of using depth information in addition to color image (RGB) data for scene understanding. Transfer learning of deep convolutional networks with pairs of RGB and depth (RGB-D) images has to deal with integrating these two modalities. Single-channel depth images are often converted to three-channel images by extracting horizontal disparity, height above ground, and the angle of the pixel’s local surface normal (HHA) to apply transfer learning using networks trained on the Places365 dataset. The high computational cost of HHA encoding can be a major disadvantage for the real-time prediction of scenes, although this may be less important during the training phase. We propose a new, computationally efficient encoding method that can be integrated with any convolutional neural network. We show that our encoding approach performs equally well or better in a multimodal transfer learning setup for scene classification. Our encoding is implemented in a customized and pretrained VGG16 Net. We address the class imbalance problem seen in the image dataset using a method based on the synthetic minority oversampling technique (SMOTE) at the feature level. With appropriate image augmentation and fine-tuning, our network achieves scene classification accuracy comparable to that of other state-of-the-art architectures. View Full-Text

Cite this Research Publication : Gopalapillai R, Gupta Deepa, Zakariah M, Alotaibi Y A., (2021), “Convolution-Based Encoding of Depth Images for Transfer Learning in RGB-D Scene Classification”, Sensors, MDPI, 21(23):7950, 2021, (IF: 3.576)

Admissions Apply Now