Gabor Convolutional Neural Network that Integrates Both Rotational and Illumination Invariance to Improving the Performance of Content-Based Image Retrieval

Abstract

Content-Based Image Retrieval (CBIR) is the main stay of current image retrieval systems where a user submits an image based query which is then used by the system to extract visual features like shape, color or texture from images. For the comparison of rotated images to be more effective these images should be insensitive to illumination changes, occlusion changes, pose changes, and less sensitive to noise. When the image rotates, its angle position changes, altering its original consistency and form and limiting it from generating similar outcomes. Furthermore, the way patterns are arranged in an image can distort its appearance, and lighting conditions can alter an image's brightness and yield different outcomes, making it challenging to find similar images. Color has been utilized previously to improve illumination where images were first transformed from RGB to HSV color before they were fed as input in the network model. Contrast texture features have also been combined with other texture descriptors to compute rotation-invariant representations of textures. Gabor Convolutional Neural network methods have also been utilized in rotation invariance. Gabor Convolutional Neural Networks are built to capture texture and spatial frequency information effectively, but they do not inherently account for intensity or lighting shifts. Gabor filters are sensitive to the absolute intensity of input images, meaning brightness changes significantly alter filter responses. This sensitivity can lead to inconsistent feature extraction under varying orientation and lighting conditions, affecting the GCNN's performance in image retrieval. When applied to RI-GCN, where the first layer of convolutional neural networks was convolved with a Gabor filter, and TI-GCN, where the first and last layers were convolved with a Gabor filter, it has shown good results for rotation invariance. Nevertheless, when the image is retrieved throughout the Networks, it continues to encounter challenges with variations in illumination and rotation. This research investigates the effects of integrating these filters into different CNN layers, early, middle, late, combined configurations, and all layers to assess their impact on rotational and illumination invariance across datasets of varying complexity, specifically CIFAR-10 and ImageNet. Experimental results demonstrate that specific layer configurations optimize performance, with early and middle layers providing fundamental color and texture differentiation for simpler datasets, and deeper layers effectively handling complex features in more challenging datasets. The combined configurations enhance both rotational and illumination robustness, contributing to improved retrieval accuracy, precision, and recall. The findings underscore the importance of adaptive, multi-layer filter integration, offering a promising direction for developing robust and efficient CBIR systems. Our results show the model is applicable in various areas such as medical Imaging to retrieve relevant diagnostic images accurately, regardless of how they were captured, facilitating better comparisons, diagnostic accuracy, and treatment planning.