No-Reference Quality Assessment of Blurred Images by Combining Hybrid Metrics

Digital images possess a substantial amount of legitimate data and have found utility in diverse everyday applications, including object recognition, image steganography, facial emotion recognition, and image retrieval. Throughout the various stages of capture, enhancement, and other image processing procedures, it is inevitable that there will ultimately be a decline in quality, leading to the loss of valuable information within the image. Consequently, the assessment of image quality has a crucial function in determining the usability of images from the user's side [1, 2].

Images can experience different types of distortion when they are transmitted and processed. Hence, it is essential to evaluate or address their quality prior to their utilization. Image quality assessment is utilized in numerous applications, such as acquisition, enhancement, compression, and restoration [3].

IQA techniques can be classified as full reference (FR), reduced reference (RR), and non-reference (NR) [4, 5]. NR IQA pertains to the automated evaluation of image quality through an algorithm. This algorithm is designed to predict distorted image quality dependent solely on the information contained within that image [6, 7]. The process of FR IQA requires the inclusion of both a distorted image and a pristine image in order to evaluate the distorted image quality. The RR IQA dataset contains information pertaining to the reference image. Nevertheless, it does not include the actual reference image, irrespective of the presence of a distorted image [8].

Currently, digital images have become an indispensable component of various domains, encompassing scientific endeavors as well as social networking platforms. In the realm of digital imagery, the occurrence of image blurring is a prevalent phenomenon. Blurring is a primary contributor to image degradation, resulting in a reduction in image quality. Therefore, this study focuses on blurry image assessment by introducing a new hybrid NR-IQA metric.

Digital images can exhibit three distinct types of blur effects, namely motion blur, out-of-focus blur, and Gaussian blur [9]. Blur is a phenomenon that arises due to the presence of air noise and improper camera configuration. In addition to its capacity to induce blurring, the presence of noise significantly compromises the quality of the captured image. There are several primary factors that can contribute to the occurrence of a blurry image, including errors in the process of capturing pictures, lens defocusing, atmospheric problems, and low intensity during camera exposure. The human visual system possesses a high level of awareness regarding this phenomenon. Nevertheless, there remains a lack of comprehensive understanding regarding the intricacies of this particular processing mechanism [10, 11]. Hence, the development of metrics for evaluating image blurring is a challenging task.

Over the last years, there has been a focus on the development of blind image quality metrics, including blind image quality metrics (IQMs) comprising spectral kurtosis [12, 13], blind reference-less image spatial quality evaluator (BRISQUE) [14], and the natural image quality evaluator (NIQE) [15]. Some IQMs have employed the statistical characteristics of the deblurred image. In contrast, other researchers have estimated image quality by incorporating the Human Visual System (HVS) [16-19].

It should be noted that each quality metric may produce satisfactory performance when applied to a specific blurring type, such as Gaussian blur, while potentially yielding unsatisfactory results when applied to other types of blurring, such as motion blur and out-of-focus blur. Recent studies [20-23] have demonstrated that the combination of different IQA metrics can help enhance the full-reference and no-reference image quality assessment results. However, the few existing combined NR-IQA metrics require a training phase that may vary from one dataset to another. Most existing combined NR-IQA metrics show limited performance, and there is still significant room for improvement.

In this paper, a novel NR-IQA metric is proposed to enhance the results of existing NR-IQA methods for blurred images. The proposed metric comprises three steps. First, a pseudo-reference image (PRI) is generated from the input blurry image using a re-blur algorithm. Second, quality scores of robust full-reference metrics are computed. Third, the quality scores for the input blurred image are integrated to produce the overall quality score. A non-linear mapping function employed in the studies [21, 23] is used in this study to align the quality scores of the proposed method.

The main contributions of this study are listed below:

• A novel hybrid metric for assessing the quality of blurred images is proposed. The proposed method does not necessitate a training phase.
• Combinations of various individual IQA metrics, including the structural similarity (SSIM) [24], Gradient Magnitude Similarity Deviation (GMSD) [25], and peak signal-to-noise ratio (PSNR) [26], and the use of two fusion techniques are studied and presented.
• Comprehensive evaluations of the proposed method and comparisons with state-of-the-art methods are provided using five IQA databases: SCID [27], SIQAD [28], LIVE [29], TID2013 [30], and KADID-10k [31]. Additionally, three different distortion types-out-of-focus blur, motion blur, and Gaussian blur-are considered in the study.

The remainder of this article includes four sections: Section 2 discusses the related literature, Section 3 presents the proposed NR-IQA method, Section 4 provides the experimental results, and Section 5 presents the conclusions.

1.png

3.1 Re-blur algorithm

A 're-blurred' image is created by purposely blurring the test image. The following equation describes the image degradation model:

$g=H * f+n$            (1)

where, g denotes the blurry image, n is the added noise, * represents the convolution operator, and H is the distortion factor, known as the point-spread function (PSF) [46]. In the spatial field, the PSF characterizes the rate at which the optical system blurs the spotlight [3, 47].

An initial input was made as a two-dimensional image f(x, y) as the following:

$g(x, y)=H(x, y) * f(x, y)+n(x, y)$             (2)

where, g(x, y) is the degraded image, * represents the convolution operator, H(x, y) indicates the distortion factor, known as PSF, and n(x, y) is the added noise [3, 46, 47]. Digital image recovery may be seen as a process in which we try to approximate f(x, y).

In this study, the parameters of the PSF, specifically the length and angle, are estimated through an initial, relatively accurate assessment of the angle using analysis in the Cepstrum domain, following the method presented by Kumar [48]. For a given angle, the blur length in the image is determined using the method presented by Chang et al. [3].

The proposed algorithm involves the acquisition of a blurred image through a blurring kernel (i.e., PSF) as well as convolving the original image. The Winner filter is applied to the blurred image in order to obtain the image, as represented by the following equation:

$G(u, v)=\frac{H^*(m, n)}{|H(m, n)|^2+N S R}$          (3)

where, NSR is the noise variance, and H refers to the blurring filter. When the filtered PSF rembles to real PSF (h), it can reproduce the identical blur in the reblurred image, and the restoration filter will produce decreased resonance and noise.

The image that has been blurred is subsequently processed through the Winner filter, which aims to remove the blur and generate an approximation of the original image. Following this, the PSF is convolved once more to modify the blur level in the image, resulting in the re-blurred image denoted as "g" or PRI, as depicted in Figure 2.

2.png

Figure 2. The steps of the re-blur algorithm used to generate the PRI

An example to illustrate the generation of the PRI is shown in Figure 3, where the I01 image in the KADID-10k dataset [31] is used as an input image (Figure 3 (a)). Figure 3 (b) shows the I01 image blurred with motion PSF at 46 degrees. Figure 3 (c) and (d) (e) (f) demonstrate the deblurred images due to PSF angles of 25, 36, 46, and 57 degrees, respectively. Figure 3 (e) demonstrates that the deblurred image is like the original image, and the ringing as well as the level of noise are tolerated. The image that has been deblurred using a blur kernel resembling the PSF is the one that will exhibit a comparable level of blurring when subjected to re-blurring.

3.png

Figure 3. Example of the image deblurring process used to generate the PRI

3.2 Combined NR IQA image quality metrics

As depicted in Figure 1, we employ SSIM, GMSD, and PSNR in this study to compute the quality scores q1, q2, and q3 and then combine them into one quality score value qs. Many metrics vary from 0 to 1, but not all. In the case of scales that do not have the same bounds, we make a normalization for them. This study uses the mean fusion function of three metrics to obtain the final quality score qs.

It should be noted that each of the three metrics used in this study has its own advantages: SSIM is designed to mimic human visual perception. It considers structure, contrast, and luminance, making it a better indicator of perceived image quality. GMSD is particularly sensitive to changes in the gradient magnitude of the image. It is good at capturing blurriness and other distortions that affect image edges. Since PSNR is based on a simple mathematical formula, it can be calculated quickly and with little processing complexity. In some circumstances, this simplicity may be helpful.

Below, we introduce the mathematical formulation of the SSIM, GMSD, and PSNR metrics and explain the combination of their quality scores to generate the final quality score (qs) to evaluate the quality of blurred images.

3.2.1 Structure similarity index method (SSIM)

The method known as the Structural Similarity Index is a model based on perception. The structural similarity index (SSIM), which has been widely studied, represents a shift in IQA from the pixel-based phase to the structure-based phase [24, 49]. The SSIM Quality Assessment Index depends on calculating three terms, namely the structural, contrast, and luminance terms [50]. This method involves the perceptional alteration of structural information, resulting in image degradation. Additionally, it collaborates with various other significant factors pertaining to perception, including contrast masking and luminance masking. SSIM values the perceived quality of images and videos. It measures the similarity between the original and the recovered images. The structural Similarity Index Method is defined through these three terms:

${SSIM}(x, y)=[l(x, y)]^\alpha \cdot[c(x, y)]^\beta \cdot[s(x, y)]^\gamma$            (4)

In this context, l represents luminance, which is utilized for comparing the brightness levels between two images. c denotes contrast, serving to distinguish the ranges between the brightest and darkest regions of the two images. s stands for structure, employed to compare the local luminance patterns between the images, revealing their similarities and dissimilarities. The positive constants α, β, and γ are also introduced [13]. The individual representations of luminance, contrast, and structure for an image are expressed as follows:

$l(x, y)=\frac{\left(2 \mu_x \mu_y+C_1\right)}{\left(\mu_x^2+\mu_y^2+C_1\right)}$       (5)

$c(x, y)=\frac{\left(2 \sigma_x \sigma_y+C_2\right)}{\left(\sigma_x^2+\sigma_y^2+C_2\right)}$           (6)

$s(x, y)=\frac{\left(\sigma_{x y}+C_3\right)}{\left(\sigma_x \sigma_y+C_3\right)}$           (7)

Here, $\mu_x$ and $\mu_y$ represent the local means, while $\alpha_x$ and $\alpha_y$ denote the standard deviations and $\alpha_{x y}$ represents the cross-covariance for images x and y respectively. If α=β=γ=1, the index is simplified as the following structure utilizing Eq. (2), Eq. (3) and Eq. (4):

${SSIM}(x, y)=\frac{\left(2 \mu_x \mu_y+C_1\right)\left(2 \sigma_x \sigma_y+C_2\right)}{\left(\mu_x^2+\mu_y^2+C_1\right)\left(\sigma_x^2+\sigma_y^2+C_2\right)}$          (8)

From Eq. (8), SSIM is on the normalized scale (the values between 0 to 1).

3.2.2 Gradient magnitude similarity deviation (GMSD)

GMSD, a method influenced by SSIM, utilizes the concept of structure similarity to evaluate digital image visual quality by measuring the similarity of their gradient magnitudes [25]. The gradient magnitudes of the reference image r and the distorted image d, denoted m_r and m_d, are determined as follows:

$m_r=\sqrt{\left(r * h_x\right)^2+\left(r * h_y\right)^2}$           (9)

$m_d=\sqrt{\left(d * h_x\right)^2+\left(d * h_y\right)^2}$        (10)

where, the symbol "*" denotes the convolution operation and h_x and h_y are the Prewitt filters along the horizontal and vertical directions, respectively. GMS is estimated as follows:

${GMS}(r, d)=\frac{2 m_r m_d+c}{m_r^2+m_d^2+c}$         (11)

where, c is a positive constant that provides numerical stability. The lighter the gray level in the GMS map, the higher the similarity and the higher the predicted local quality. The GMSD algorithm uses the standard deviation of the GMS map as the IQA index indicates.

${GMSD}(r, d)={STD}(G M S(r, d))$          (12)

In this context, STD(●) denotes the computation of the standard deviation for the input variable. The GMSD value serves as an indicator of the extent of distortion present in an image. A higher GMSD score corresponds to lower perceived image quality. In the absence of distortion, the GMSD value is 0, showcasing its effectiveness in detecting structural distortions with a minimal computational cost.

3.2.3 Peak signal-to-noise ratio (PSNR)

The equation representing PSNR can be written as [26]:

$P S N R=10 * \log _{10}\left[\frac{225^2}{M S E}\right]$          (13)

The value of MSE is estimated by the difference between the blurred and reblurred image as:

$MSE =\frac{1}{M N} \sum_{X=1}^M \sum_{Y=1}^N\left[T(x, y)-T^{\prime}(x, y)\right]^2$           (14)

where, $T(x, y)$ and $T^{\prime}(x, y)$ denote the pixel value at position (x, y) of the blurred and reblurred images, respectively.

3.3 Combining the quality scores

After computing the quality scores of SSIM, GMSD, and PSNR (i.e., q1, q2, and q3), we calculate the final quality score, qs, of the input blurred image using the mean fusion function:

$q s=\frac{1}{N S} \sum_{i=1}^{N S} q_i$           (15)

where, q_i is the quality score of i^th quality assessment method (SSIM, GMSD, or PSNR), and NS is the number of quality assessment methods. In this study, we experimentally set NS to 3.

Following Sheikh et al. [51], we use a nonlinear mapping before the calculation of evaluation metrics. Firstly, we intend to align the quality scores. After that, we utilize the following five-parameter logistic function for mapping the quality scores:

$q s=\beta_1\left(\frac{1}{2}-\frac{1}{1+\exp \left(\beta_2\left(q-\beta_3\right)\right)}\right)+\beta_4 q+\beta_5$,           (16)

where, qs and q stand for the original and mapped quality scores, respectively; $\left\{\beta_j \mid j=1,2, \ldots .5\right\}$ are five parameters identified based on curve fitting. In prior research, the qs values are considered for evaluation metrics' computation.

Notably, the MATLAB function 'nlinfit' is used in this study to estimate the coefficients of the nonlinear mapping function shown in (16).

3.4 Evaluation metrics

In this study, three evaluation metrics are used to compare the performance of the different indices: Pearson Linear Correlation Coefficient (PLCC), Spearman's Rank Ordered Correlation Coefficient (SROCC), and Root-Mean-Square Error (RMSE). Both PLCC and SROCC are statistical methods used to measure the correlation between two variables. In the context of IQA, PLCC and SROCC are widely used to evaluate the performance of image quality metrics by comparing their predicted scores with human-rated scores. Higher PLCC and SROCC values stand for better agreement between the predicted quality scores and human scores.

The PLCC index can be expressed as follows:

$\operatorname{PLCC}=\frac{1}{n-1} \sum_{j=1}^n\left(\frac{x_j-\bar{x}}{\sigma_x}\right)\left(\frac{y_j-\bar{y}}{\sigma_y}\right)$            (17)

where, $\left\{x_1, x_2, \ldots, x_n\right\}$ are subjective scores, $\left\{y_1, y_2, \ldots, y_n\right\}$ are objective scores, $\sigma_x$ and $\sigma_y$ are their variances and $\bar{x}$ and $\bar{y}$ are their average scores. PLCC is a metric measuring how well the objective scores are associated with the subjective scores.

The SROCC index can be formulated as follows:

$\operatorname{SROCC}=1-\frac{6}{n\left(n^2-1\right)} \sum_{j=1}^n\left(r_{x_j}-r_{y_j}\right)^2$,            (18)

where, $r_{x_j}$ and $r_{y_j}$ are the rank positions of $x_j$ and $y_j$ in Arrays {x} and {y}, respectively. SROCC is a metric measuring the relative monotonicity between the objective and subjective and objective sores.

RMSE can be expressed as follows:

$\operatorname{RMSE}=\left[\frac{1}{\mathrm{n}} \sum_{\mathrm{j}=1}^{\mathrm{n}}\left(\mathrm{x}_{\mathrm{j}}-\mathrm{y}_{\mathrm{j}}\right)^2\right] \frac{1}{2}$,           (19)

RMSE is a metric utilized to determine the absolute error between the objective as well as subjective scores. A good algorithm exhibits reduced RMSE value. RMSE is a prevalent measure representing the quadratic mean’s square root of the differences between the objective predicted scores and subjective quality scores.

Through the utilization of these three statistical measures, it becomes feasible to effectively examine the uniformity between the subjective quality score as well as the objectively predicted score signifying the IQA methods' performance.

This subsection provides an overview of evaluating NR-IQA metrics. NR-IQA ranking algorithms and performance assessment depend on the association between the ground as well as predicted truth quality scores. To quantitatively assess our proposed model performance, we examine the proposed IQA metrics utilizing RMSE, PLCC, SROCC.