Monte Carlo Optimization of a Combined Image Quality Assessment for Compressed Images Evaluation

In any imaging system, that uses storage or transmission, the compression process becomes an indispensable tool to reduce the required binary capacity to store and transmit these images. The importance of compression algorithms on the compression rates and processing times presents a significant challenge, especially with the increase in day-to-day big data concepts [1]. The compression process role represents an image using a few bits, which increases the compression ratio while keeping an acceptable quality of the image [2]. Unless the JPEG standards that use the discrete cosine transform and the JPEG2000 operate with the wavelet transform, most techniques proposed in the literature attempt to present and use other combinations of algorithms to optimize and minimize compression-induced degradation by having high compression ratios. Most of these techniques use Wavelet transforms of one kind or another [3, 4]. The compression algorithms that reduce the image size induce a modification and a distortion in the present data in these images. This process generates a degradation of the textural image quality, making them unusable [5].

To perform these algorithms and proceed to an evaluation of the obtained quality in compressed images, two approaches are usable; subjective methods and objective methods [6]. In the subjective context that involves the human observer for assigning a note to a degraded image, several measures are developed. These measurements respect especially the recommendations of the ISO 3664-2000 standard by the International Organization for Standardization, specifying the viewing conditions for images and the ITU-BT.500-13 standard, normalized by the International Telecommunication Union, that provides methodologies for the assessment of image quality including general methods of test, the grading scales, and the viewing conditions.

In the context of objective methods, scientists have developed numerical image quality assessment (IQA) methods to qualify various image-processing applications such as image compression methods. According to the use of a reference image, the most used objective quality indices are called full reference methods (FR-IQA) because these techniques use the total original image as a reference to compare modification in the compressed image [7]. Resumed from extensive literature sources, hundreds of full-reference image quality assessment, we can categorize the most proposed techniques into seven groups depending on how images are compared [8, 9]:

• Pixel difference-based measures exploiting direct features from the pixels of compared images such as energy difference or mean square distortion;
• Correlation-based measures using a structural correlation of pixels or the vector angular directions;
• The human visual system (HVS) based on human eye model measures conforms to HVS-weighted criteria functions like contrast, color, brightness, and frequency changes.
• Spectral distance-based measures with a transform analysis in the frequency domain, or spatio-frequently domain;
• Context-based measures based on various functions of the multidimensional context probability;
• Texture-based measures by analyzing the degradation of the textural composition of images and their consistency across direction and resolution levels;
• Edge-based measures that evaluate the preservation of the edge and contour components in images, across resolution levels and edge models.

We can note that each metric presented in the literature tends to respond specifically to certain distortion types. To scan and evaluate all distortion types in a single process, some authors proposed a new idea that combines several metrics into an optimized one [10, 11]. So complementary to the seven groups cited above, we can add another category of measures that fusion some elementary quality metrics to produce a new combined parameter that evaluates the image quality over different ways at one time. The most used examples and the basic concept are discussed in the next sections of this paper.

This work proposes another combined image quality that uses three image evaluation concepts; structural, textural, and edge analyses. To find the best form of image quality evaluation, we will try to optimize this combination and calculate the best weights combination by exploiting the Monte Carlo optimization method. For this aim, we organize this paper into three parts. The first one, resume some works that use the same concept for the combined metrics. In the second part, we present our methodology proposing three ways combined metrics and their optimization by the Monte Carlo method. Finally, in the simulation results part, we discuss obtained results in evaluating some chosen images to qualify their quality.

3.1 Metric fusion concept

The objective of combined metric IQA is to produce objective scores consistent with human ones and provide better results to contribute to the multi-measure approach. Several ways and concepts combining image quality metrics have been proposed presenting different models in metrics combination [10, 11, 21]. These models use mathematic functions approach, rules decision techniques, clustering, neural networks, or other learning techniques. With the mathematic model, there are two forms for this combination: 

The weighted additive approach that takes the global measure as an average of all used Qi metrics using a weighted sum like in Eq. (1):

$Q=\sum_{i=1}^{n} p_{i} \cdot Q_{i}=p_{1} \cdot Q_{1}+p_{2} \cdot Q_{2}+\cdots+p_{n} \cdot Q_{n}$   (1)

The weighted multiplicative approach that calculates the multiplication of different Qi metrics of which is associated with a combined weight, Eq. (2):

$Q=\prod_{i=1}^{n} Q_{i}^{p_{i}}=Q_{1}^{p_{1}} \times Q_{2}^{p_{2}} \times \ldots \times Q_{n}^{p_{n}}$    (2)

where, p={p_1,p_2,p_3,…,p_n } is the vector of weights combination that is optimized to have a better form of combination and Q_i.represents the quality metrics.

In our study, we focus the choice a combined three ways to evaluate the quality in the compressed images; (1) the structural-based evaluation with the (MS-SSIM) metric, (2) the textural-based evaluation using an image texture quality metric (ITQ), and (3) the edge-based quality evaluation by exploiting an edge image quality metric (EdgeIQA).

The most important quality metric in image quality assessment that evaluates the structural quality is the Multiscale Structural Similarity (MS-SSIM) proposed by Wang et al. [30]. This metric is an ameliorate case of the SSIM exploiting the brightness, contrast, and structure. Based on SSIM, it repeats iteratively repeated to M scales by applying a low-pass filter and a down-sampling by a factor of 2. This metric is presented by the following formula [30]:

$M S-\operatorname{SSIM}\left(X_{0}, X_{c}\right)=\frac{1}{M} \sum_{k=1}^{M} \operatorname{SSIM}\left(X_{0}^{(k)}, X_{c}^{(k)}\right)$     (3)

With $X_{0}^{(k)}$ and $X_{c}^{(k)}$ are respectively the original and the compressed images in a considered (k) scale.

The SSIM parameter is a single scale similarity index calculated by a combined comparison of the properties of brightness, contrast, and structure information between pair of vectors x and y of compared images:

SSIM(x,y)=l(x,y)×c(x,y)×s(x,y)    (4)

In a second way, and to understand the compressed image textural quality, we use a proposed image quality that uses the Gray Level Co-Occurrence Matrix (GLCM) textural features [31]. This parameter estimates properties of images relating to second-order statistics measuring the probability of appearance of pairs of pixel values located at a distance in the image using Contrast, Correlation, Energy, Entropy, and Homogeneity. The following equation defines the final image texture quality (ITQ) for five features with the feature index i={Cont,Corr,Ener,Homo,Entr}:

$I T Q=\sum_{i} a_{i} \times I T Q_{F(i)}$     (5)

With $\sum_{i} a_{i}=1$ and the value of the $I T Q_{F(i)}$ parameter calculates the feature quality by:

$I T Q_{F(i)}=1-\frac{\left|F(i)_{X_{0}}-F(i)_{X_{c}}\right|}{F(i)_{X_{0}}}$     (6)

where, $F(i)_{X_{0}}$ represents the i^th textural feature for the original image and $F(i)_{X_{C}}$ for the compressed image.

In a third way and to improve the compression algorithm's efficiency, we evaluate the edge degradation in a compressed image by using another proposed texture-based quality. This parameter compares the obtained binary edge images $E_{\text {org }}^{(t h)}$ and $E_{\text {comp }}^{(\text {th })}$ corresponding respectively to the original and compressed images. These edges are determined by a Canny detection technique [32] with an optimal calculation of the threshold by the grayscale histogram of the image's gradient [33].

This quality calculate the normalized common edges between $E_{\text {org }}^{(t h)} \text { and } E_{\text {comp }}^{(\text {th })}$ with the following formula using the sum of the edge points preserved in the compressed image compared to the edge points in the original image:

$E d g e I Q A=\frac{\sum\left(E_{\text {comp }} \cap E_{\text {org }}\right)}{\sum E_{\text {org }}}$    (7)

With these selected indices and conform to the combination concepts, we propose a combined metric by mathematical Eq. (1) applied on images chosen as shown in the next section. With this model, our proposed textural combined quality form is in the following equation:

$T C Q=\alpha \times M S_{-} S S I M+\beta \times I T Q+\gamma \times E d g e I Q A$    (8)

With α, β and γ<1 are weighted coefficients for the summed combination and as a condition α+β+γ=1.

According to the previous formulas, we obtain the combined quality scores that should return quality scores that are consistent with the human subjective evaluation and evaluate image quality in structural similarity, textural evaluation, and edge conservation. To find the best combination weights, we consider this problem as an optimization approach allowing us to find the weights coefficients to give the best correlation between the calculated objective scores and subjective scores. We use specific benchmark database images that contain reference images and some corresponding distorted equivalents with subjective human scores provided by mean opinion scores values (MOS or DMOS), according to the recommendation of the ITU Video Quality Experts Group (VQEG) [34]. These scores are calculated after a non-linear mapping between the vector of objective scores S ̂, and MOS or DMOS, denoted by S representing the subjective scores.

To do this, the Root Mean Square Error (RMSE) between the objective and subjective vectors is calculated by the following formula:

$R M S E\left(\hat{S}_{m}, S\right)=\sqrt{\frac{\left(\hat{S}_{m}-S\right)^{T}\left(\hat{S}_{m}-S\right)}{m}}$    (9)

With $\hat{S}_{m}$ is the non-linearly mapped S ̂.

Considering $d_{i}$ is the difference between the $i^{t h}$ image in S ̂ and S, in a total number of m images, the Spearman rank-order correlation coefficient (SROOC) is a normalized coefficient [-1,1], defined as:

$\operatorname{SROOC}(\hat{S}, S)=1-\frac{6 \sum_{i=1}^{N} d_{i}^{2}}{m\left(m^{2}-1\right)}$    (10)

The other used coefficient called Kendall’s rank correlation coefficient (KRCC) [-1,1], is defined as:

$\operatorname{KRCC}(\hat{S}, S)=\frac{n_{c}-n_{d}}{(n-1) \times \frac{n}{2}}$   (11)

With n_c and n_d represent respectively the number of concordant and discordant pairs of the n observations between the S ̂ and S vectors.

 Table 1. The used public IQA databases in this study

 Besides, another normalized coefficient called Pearson linear correlation coefficient (PLCC) values is as:

$\operatorname{PLCC}\left(\hat{S}_{m}, S\right)=\frac{\overline{\hat{S}}_{m}^{T} \bar{S}}{\sqrt{\hat{S}_{m}^{T} \bar{S}_{m} \bar{S}^{T} \bar{S}}}$   (12)

However, we note that the basic problem concerns the optimization approach by finding the best combination coefficients (in our case α, β, and γ), The objective of the optimization algorithms is to find the best weights combination able to minimize the RMSE and maximize the SROOC, the KRCC, and the PLCC correlation values. Some studies compared the evaluation of the efficacy of correlation methods PLCC, SROOC, and KRCC. Based on these works, we choose the use of the Pearson linear correlation coefficient (PLCC) as the best approach for the optimization problem [35-37].

3.2 Our optimized fusion model

In our optimization case, we opt for two computation approaches. The first one uses an experimental approach by recursive calculation of all possible coefficient combinations and find the optimal weighting values for the quality correlation factor. Each coefficient is varied from 0 to 1 with a step equal to 0.01, to take all possible combinations satisfying the condition α+β+γ=1. With this recursive method, we can calculate the values of $T C Q_{B F}$ corresponding to Eq. (8). This technique, called the brute force method, presents a critical problem that scales exponentially with the number of scores.

Our second case uses the Monte Carlo (MC) optimization method [28, 29] with an oriented approach to finding the best weight coefficients. This method approximates the weighted fusion of the quality by considering the problem as follows:

$\max _{\alpha, \beta, \gamma} P L C C\left(\hat{S}_{m}, S\right)$    (13)

With the oriented constraint conditions expressed as: 

$\left\{\begin{array}{l}\alpha+\beta+\gamma=1 \\\alpha, \beta, \gamma<1\end{array}\right.$    (14)

The basic concept of this algorithm, as shown in Figure 1, is generating random vectors α and β in a considered interval and calculate the γ value conforming to Eq. (12). The next step is the calculation of the objective quality values using vectors (α,β,γ) and determining the correlation vector between subjective and objective qualities. Finally, we determinate the weight coefficients that satisfy the maximum of the PLCC correlation. To enable calculus of the weight coefficients, we propose the use of four benchmark IQA databases, Table 1; Categorical Subjective Image Quality database (CSIQ) [38], Tampere Image Database 2008 and 2013 (TID2008) [39], (TID2013) [40], and the Konstanz Artificially Distorted Image quality Database (KADID-10k) [41].

The TID2008 database that contains 1700 distorted images generated from 25 reference images, with 4 level distortions from 17 distortion types [39]. In the TID2013 database, an ameliorate version of TID2008, containing 24 distortion types to produce 3000 test images from 25 reference images [40]. The CSIQ database with a total of 866 distorted images compared to 30 reference ones generated by 6 distortion types [38]. Finally, the KADID-10k database contains 10125 test images generated from 81 reference images, within a total of 25 types of distortions, grouped into 8 distortions classes, namely blurs, compression, noise-related, brightness changes, spatial distortions, sharpness, and contrast [41].

1.png

Figure 1. The Monte Carlo optimization algorithm

In this work, we proposed a new form of quality analyzing degradation in compressed images. It's based on the fusion of three quality forms; (1) the structural quality based on the structure similarity (MS-SSIM), (2) the texture-based quality based on the use of a GLCM textural image evaluation metric named (ITQ), and (3) the edge-based quality that use an edge image evaluation metric named EdgeIQA based on Canny edge conservation. This fusion is based on the weighted linear combination model. This proposed metric presents a good evaluation with four chosen databases (TID2008, CSIQ, TID2013, and KADID-10k).

The numerical experiment results investigate some wavelet-based compression algorithms coupled with SPIHT compared to the JPEG2000 standard compression. These results confirmed that the JPEG2000 and the Quincunx-based algorithm associated with SPIHT (QWT-SPIHT) related to the lifting Wavelet transform with SPIHT (LWT-SPIHT). We have also used four kinds of images to perform our study; a natural image, a medical image, a biometric image, and a texture image.

As a future perspective, we propose to analyze and study the efficiency of our combined metric according to the types of degradation through benchmark image databases to understand the textural deterioration according to these types of degradations. We suggest also adding a multiscale wavelet approach to allow a deep concept quality evaluation in the spectral-based quality.