An Optimized Deep Learning Approach for Robust Image Quality Classification

High-quality images are indispensable in both professional and personal spheres of life. Unfortunately, during the processes of image acquisition, transmission, and processing, images can become susceptible to distortion from environmental factors such as poor lighting conditions, device noise, and compression damage. These factors can lead to significant information loss and a consequent reduction in image quality. Images of suboptimal quality not only fail to satisfy human visual requirements but also pose challenges to computer processing and analysis. Consequently, a pressing need exists for the restoration of essential content and details from degraded images [1].

Throughout the stages of image acquisition, processing, compression, transmission, and reproduction, both images and videos can experience a myriad of distortions. Although deteriorations in visual quality are easily discernible to the human eye, objective evaluation of perceived image quality is challenging. For instance, consider a mobile application where customers upload "before" and "after" photos of an insurance event. If the photos are of poor quality, the system should be capable of highlighting these low-quality images and, if necessary, prompt the user to recapture the object [2].

Over the past decade, an increased interest has been observed in objective image quality assessment methodologies. Such methodologies can optimize a wide range of image and video processing techniques, in addition to monitoring image quality deterioration and benchmarking image processing systems.

In recent years, various types of classification networks have emerged, achieving high classification accuracy. More sophisticated models, such as deep convolutional neural networks (DCNNs), have been employed to tackle this challenge. However, real-time image capture under varying conditions such as weather, noise, and motion can result in low-quality images, leading to a significant drop in network accuracy. This is primarily because image degradation obstructs the structural statistical properties of pixels in the neighborhood [3].

In the realm of computer vision, the categorization of low-quality images holds substantial importance. Automated identification of such images can enable numerous practical applications. For instance, search engines can discard low-quality images, digital and phone cameras can alert users of poor-quality shots, and autonomous driving technology can avoid using poorly captured frames, thereby reducing the likelihood of errors. The issues that may arise in an image can stem from a broad range of problems, such as inadequate lighting or blur due to improper photography techniques, or even encoding issues.

Approaches to address this problem vary. One solution involves enhancing the visibility of the image before performing classification. However, such methods often demonstrate low accuracy and poor robustness. Another potential solution involves transforming the problem into a domain adaptation problem. However, most existing domain adaptation approaches require either complex architectures or continuous target domains.

In this study, our focus is on the design of a novel SimpleNet for robust image quality classification. It is demonstrated that a specific classification task can be accomplished with high accuracy simply by calibrating the model for the task using transfer learning.

Based on the SimpleNet and InceptionResNetV2 models, we describe the suggested framework for undesirable picture categorization (bad lighting, fastfading, gblur, jp2k, jpeg, refimgs, and wn). The suggested CNN-based classifiers framework was constructed using "Google-Colab".

3.1 Dataset (Unwanted image)

In this study, a primary dataset containing 1,165 poor quality images as shown in Table 1, we have focused on seven categories (bad_lighting, fastfading, gblur, jp2k, jpeg, refimgs and wn) as shown in Figure 1 (a, b, c, d, e, f and g) one example for each class.

Table 1. Details of photo quality dataset

No. of Class	Name of Class	No. of Image
1	bad_lighting	22
2	fastfading	174
3	gblur	207
4	jp2k	227
5	jpeg	233
6	refimgs	128
7	wn	174

1.png

Figure 1. Example for each class of photo quality

There are several reasons why an image may be regarded to be of low quality [15-19]. In the following, we explain all types of poor images:

• Bad Lighting (bad lighting) Images that were captured without enough light for the camera's timing and aperture. This might make photos appear drab or dark.
• Fast fading (fastfading) is the effect of constructive and destructive interference patterns which are caused due to multipath in channels of the image.
• An out-of-focus camera causes Gaussian Blur (gblur). This form of blur might be caused by a camera's malfunctioning auto-focus system, a poorly designed lens, or a picture focused on the wrong topic.
• JPEG-2K In the year 2000, the JPEG-2K picture format was released. While this picture compression format has numerous advantages, one downside is that it is significantly less content adaptable than the earlier JPEG format, which means that image quality might vary greatly given the same bit rate for different materials.
• Reference Image (refimgs) is used to refer to any visual resource such as a diagram, graph, illustration, design, photograph, or video. They may be found in books, journals, reports, web pages, online videos, DVDs and other kinds of media.
• White Noise (wn) is the appearance of random white grains in a picture. This noise is frequently created by film grain, different sensors and circuits such as CCDs in digital cameras and detectors in scanners, or by the communications channel or signal quantization.

3.2 Deep convolutional neural networks

Deep artificial neural networks (DANNs) or deep convolutional neural networks (CNNs) are used to classify images, group similar images together, and identify objects in scenes [20]. CNN is made up of convolutional and subsampling layers, which are then followed by one or more fully linked layers. CNN's architecture is intended to capitalize on the 2D structures of input pictures. Moreover, CNNs have fewer parameters and are simpler to train than fully connected networks. In order to train and evaluate the CNN model, each input image is routed through a series of convolution layers and pooling for feature learning. Additionally, an activation function like Softmax, Sigmoid, or ReLU is employed to classify an item. In the area of computer vision, CNNs and other cutting-edge machine learning algorithms have become crucial. We create and train a CNN with three convolutional layers to categorize pictures into six predetermined categories in our method. We demonstrate that our technique can achieve reasonably high accuracy when compared to a very sophisticated model (ResNet) with only a small number of parameters.

3.2.1 Our Model (SimpelNet)

The network consists of seven blocks. Each block is made up of a fully connected layer with a Sigmoid activation function, followed by a convolutional layer with a Relu activation function. There are also six node output layers with sigmoid activation functions that output the six required classes. The first convolutional layer takes a 256 256 3 image array and applies 32 filters to create 256 256 3 feature maps. We begin with the bigger filter size since it delivers the best results for picture classification tasks in the higher layers [20]. The feature maps are then normalized, the Relu activation function is used, and a 2 2 Max Pooling is conducted to create separate image domains for each block, beginning with the second Conv2D and ending with the last Conv2D.

In addition, we started from 20% Dropout from the second Conv2D to the third Conv2D and a 30% Dropout from the fourth Conv2D to the fifth Conv2D. Then a 40% Dropout from the sixth Conv2D to the seventh Conv2D.

The goal of this significant dropout is to keep the network from overtraining on the training set and aid in greater generalization when used with other types of images. The output layer, which has six nodes and is connected to every node from the layer before it, has sigmoid activation functions. A total of 1,425,606 parameters makes up the networks. The architecture of our CNN is depicted in Figure 2, and Table 2 lists the layers and parameters for each layer.

2.png

Figure 2. The architecture of our CNN SimpelNet

3.png

Figure 3. Basic block diagram of InceptionResNetV2 model

3.2.2 InceptionResNetV2

Inception ResNetV2 is a hybrid of the Inception and ResNet families, with 164 layers for picture object identification and feature extraction [21-23]. Only the last layer is applied to examine the results. To train the InceptionResNetV2 CNN model, we explicitly replicate resolution inconsistency in affine face wrappings during video editing. Using a trained model reduces the size and difficulty of training. InceptionResNetV2 is one of famous pre-trained networks.

When there are a limited number of training samples available, tuning a pre-trained network is the best option. In this example, the top layers of a pre-trained network's parameters are fine-tuned, but the initial layers, which represent generic characteristics, are frozen. When weights for a layer or group of layers are not changed during the training stage, this is referred to as freezing. Importantly, this method uses the parameters learnt from a network that has already been trained on a certain dataset and then modifies the parameters for the dataset of interest. As a result, fine-tuning modifies the parameters of the repeated model, making it more relevant to the dataset under consideration. The top layers of a pre-trained network can be fine-tuned or all layers; nevertheless, the latter technique is favoured [24].

This is due to the fact that the earliest levels of a structure encode general, reusable properties, and the later layers encode highly specialized traits. As a result, fine-tuning those individual traits is more efficient. Furthermore, because of the enormous number of parameters that must be established during this procedure, fine-tuning all layers produces overfitting. As a result, only the top three levels of pre-existing convents were fine-tuned in this study. These might be the completely linked layers alone, as in InceptionResNetV2. Figure 3 depicts the architecture of the InceptionResNetV2 model's fundamental block diagram.

Machine learning algorithms must be able to recognize when the outcomes of activity are undesirable given the rapid improvement of machine learning and the expanding use of autonomous technologies. The findings of this study demonstrate that our suggested CNN can accurately identify low-quality photos. Finally, the generated model's application was evaluated to find photos that would not be appropriate for the sign categorization task needed by autonomous cars. We demonstrated that it is possible to attain extremely high accuracy for the given classification job by merely calibrating the model for the task using transfer learning. This study was focused on designing a novel SimpleNet and comparing it with the InceptionResNetV2 model which is considered a highly complex model to detect poor-quality images. Moreover, we perform sensitivity and specificity analysis to determine how differences in camera hardware affect the accuracy of the model.