Content-Based Image Recovery System with the Aid of Median Binary Design Pattern

Multimedia data, such as films and photographs, have significantly risen in our daily lives along with the growth and use of the Internet [1]. Today, a popular research area is the speedy and accurate retrieval of important information in mass multimedia. Content-based image retrieval (CBIR) is one of the most well-liked fundamental research fields. Although CBIR has seen a lot of research activity during the 1990s, the complexity of image data continues to present several complex problems [2]. These problems are associated with difficulties in several interdisciplinary study fields, including computer vision, image processing, image databases, machine learning, etc. [3, 4].

Unlike conventional keywords-based image retrieval, a typical CBIR retrieves images based on their visual content, such as color, form, and texture, etc. [5]. There are numerous CBIR methodologies, and practically all of them adhere to a standardized CBIR pipeline. The pipeline's first step is extracting features from the photo group [6-8]. Typically, only one instance of this step is required to produce a feature database. The database can be regenerated if there are new photographs in the collection [9]. Any data that can be calculated from an image's pixels and is pertinent to the image is referred to as a feature. A feature vector is created by organizing these features [10].

1.png

Figure 1. Sample images for CBIR

An input image is the first step in a CBIR pipeline inquiry. A feature vector is generated when features are extracted from this image [11]. Only feature vectors with a proximity to the input vector are returned after the input feature vector has been compared to every feature vector in the database. A method's success depends on how the distance is calculated [12]. Sample images of CBIR is shown in Figure 1.

Existing approaches cannot efficiently use the saliency zones of images, and human perception of images cannot be well stored [13, 14]. In addition, typical feature extraction techniques overlook the spatial structure of images, which is an essential quality for image retrieval [15-17]. Another popular feature is texture. Traditional approaches, including the Tamura texture feature, GLCM and Gabor filter texture feature, are used to define texture features. When compared to the suggested methodology, the method and its considerable computing complexity produce poor results [18, 19].

To solve these limitations, we propose a novel image retrieval method based on a multichannel decode local-binary pattern is recommended. To consolidate the LBPs from each channel of color images, this method utilizes adder and decoder-based techniques and functions admirably with color images. This paper proposes the median binary pattern (MBP), famous for its noise-resilient binary pattern, to make the framework noise-resilient.

Further content of this research work is systematized as Section-II examines interrelated work, Section-III talks about existing techniques, Section-IV discusses proposed strategies, Section-V examines test results and examinations, and Section-VI talks about the proposed strategy's conclusion and future extension.

The noise in digital images is mainly developed in the course of image procurement or image sharing. Because of the image noise, one can't distinguish the data in images. There are countless reasons for noise in images, like ecological conditions at the time of image procurement or based on the nature of the detecting components. The primary cause for noise in the image is interfering with the channel applied for image sharing. Noise image can be demonstrated below,

$\mathrm{A}(\mathrm{x}, \mathrm{y})+\mathrm{B}(\mathrm{x}, \mathrm{y})=\mathrm{C}(\mathrm{x}, \mathrm{y})$          (1)

2.png

Figure 2. The layout of the content-based image recovery process

Here A(x, y) addresses the first image pixel worth and B(x, y) is the noise found in the image, and C(x, y) is the subsequent noise image. Salt and pepper noise is used for the analyses in the present paper. The filtering method utilized in this paper is Median Binary Pattern (MBP). Middle Binary Pattern (MBP) administrator maps the concentration space to the local binary pattern (LBP) by developing the pixels. MBP generated pattern is utilized to describe color images. For the effective utilization of the LBP data from different channels, two multichannel decoded LBP methods are suggested in this paper: multichannel adder-based LBP (maLBP) and multichannel decoder-based LBP (mdLBP). The proposed approach was tested using image recovery on the Corel 1k dataset. Five classes were considered: beaches, monuments, buses, dinosaurs, and elephants, each with 20 images. Figure 2 explains the proposed architecture.

3.1 Problem statement

Given that the color histogram is a statistical result and has the advantages of being straightforward, reliable, and effective, it is acknowledged as a suitable representation of features. However, the fundamental issue with color histogram indexing is that it ignores spatial information. A system that handles any image of a user's interest as a query image and compares the image with the data to obtain precise matches or photos in the same general category as the query images is to be built. The construction of a system that admits any image as a query image and retrieves images by analyzing or matching image data using specific properties of the image offers a solution to the problem statement. These characteristics are derived from the saliency maps-based method of salient feature extraction.

3.2 Salt and pepper noise

This kind of noise is known as gunshot noise, spike noise, or impulse noise [10, 11], which is generally affected by wrong retention areas, failure of pixel constituents in the camera sensors, or there may be mistakes in timing during digitization. To mitigate noise, filtering methods are utilized. The filtering method used in this paper is Median Binary Pattern (MBP).

3.3 Middle Binary Pattern (MBP)

Middle Binary Pattern (MBP) [17, 18] is the method utilized to control the noise present in the image. Middle Binary Pattern (MBP) administrator maps the concentration space to the local binary pattern (LBP) by developing the pixels in contrast to their median value in a (normally 3X3) neighborhood rather than continually utilizing the middle pixel as I LBP to give greater affectability to noise robustness. The median value of 120 is observed in the patch. The MBP at pixel (x, y) is characterized as,

$\operatorname{MBP}(\mathrm{i}, \mathrm{j})=\sum_{k=0}^{L-1} 2^K H\left(b_k-\tau\right)$          (2)

where, L is the patch size or pixel area (i.e., L=9 for 3X3 fix) and $\ \tau$ to is the middle of the patch.

The MBP is utilized to measure the supply of these patterns in the image and structures the surface descriptor in Figure 4.

As indicated by the meaning of the LBP, an $L B P t \ (x, y)$  for a specific pixel $(x, y)$  in t^th network is created by figuring out a binary number $\operatorname{LBPtn}(x, y)$ assumed in the below formula,

$\operatorname{LBPt} \ (x, y)=\sum_{n=1}^N \operatorname{LBP}_t^n(x, y) \times f^n$, for all $\ t[1, c]$  

where, fⁿ is $L B P_t^n(x, y)= \begin{cases}1, \mathrm{I}_t^n(x, y) \geq \mathrm{I}_t(x, y) \\ 0,  \text { Otherwise }\end{cases}$ a premium task demarcated by the succeeding formula fⁿ =(2)^(n-1) for all $n∈\lceil 1, N\rceil$. Figure 3 represents the block diagram of median binary pattern.

3.png

Figure 3. Block diagram of median binary pattern

4.png

Figure 4. MBP for a 3x3 patch (i.e., L=9) with a median $\ \tau$₌₁₂₀ output binary pattern MBP=100011110₂ =286

5.png

Figure 5. The local neighbors

Image Source: https://github.com/arorayash/mdLBP

The N binary values LBPtn(x, y) is set in a specific pixel of (x, y) relating to every neighbor Itn(x, y) of t^th channel. Local neighbors of MBP were represented in Figure 5.

3.4 Multichannel Decoded Local Binary Pattern (mdLBP)

This pattern is utilized to describe color images. For the effective utilization of the LBP data from different channels, two multichannel decoded LBP [2, 15] methods are suggested in this paper: multichannel adder-based LBP (maLBP) and multichannel decoder-based LBP (mdLBP). A sum of (c+1) element and (2c) of resulted channels are created as of (c) number of input-based channels of (c≥2) utilizing a multichannel adder and decoder, respectively. Image 4 portrays the calculation interaction engaged with multichannel adder-based and decoder-based LBPs. Multichannel LBP adders and decoders are utilized to describe the color images, with LBPtn(x, y) for all $t ∈-[1, c]$ as the $c$ input-based channels. Hence the results of multichannel adder-based and multichannel decoder-based local binary patterns are indicated by maLBPt1n(x, y) and mdLBPt2n(x, y), separately, $\mathrm{t} 1 ∈ \ [1, \mathrm{c}+1]$ and $\mathrm{t} 2 ∈[1,2 \mathrm{c}]$. It ought to be noticed that the calculated numbers of LBPtn(x, y) may be binary values (for example, 0 or 1). Consequently, the binary numbers of maLBPt1n(x, y) & mdLBPt2n(x, y) have resulted from the multichannel adder map and multichannel decoder map decoded by maMn(x, y) & mdMn(x, y) relating to every pixel's neighbor n.(x, y). Numerically, maMn(x, y) & mdMn(x, y) are characterized as follows:

25_6.png

Figure 6. Flowchart of a mix of multichannel adder

Figure 6 shows the LBP highlight vector (for example, maLBP) and multichannel decoder-based LBP highlight vector (for example, mdLBP) of the picture's blue, red, and green channels. We denote $\operatorname{LBP}_t^n(x, y)$ of all $n ∈-[1, N]$ in a same way for all the $t ∈-[1, c+1]$ in adder pattern and $m d L B P_{t 2^n}(x, y)$ for all $n ∈-[1, N]$ and $t_2 ∈\left[1,2^c\right]$ in decoder pattern correspondingly. The multi-channel adder-based LBP $\operatorname{maLB} P_{t 1^n}(x, y)$  for pixel (x, y) from multi-channel adder map maMⁿ(x, y) and t₁ is defined as,

$\begin{aligned} & m d L B P_{t 1}^n(x, y) \\ & =\left\{\begin{array}{l}1, \  { maM }^n(x, y) \geq\left(t_1-1\right) \\ 0, \text { otherwise }\end{array} \ \text { for } \ \text { all } \ t_2\right. \\ & \in[1, c+1] \text { and }  \text { for }  \text { all } \ n \in[1, N] \\ & \end{aligned}$          (3)

Multi-channel decoder map mdMn (x, y) and t₂ may be calculated as,

$$

m d L B P_{t 2}^n(x, y)=\left\{\begin{array}{l}

1, \ m d M^n(x, y) \geq\left(t_2-1\right) \\

0,  \text { otherwise }

\end{array}\right.

$$

for all $t_2 \in\left[1,2^c\right\rceil$ and for all $n \in[1, N]$          (4)

The computation of maLBP_t1(x, y) for all $t 1 ∈-\lceil 1, c+1\rceil$ in the same way maLBP_t2(x, y) for all $t_2 \in\left[1,2^c\right]$ from input $L B P_t{ }^n \ (x, y)$ applying a sample as shown in Figure 7 for c=3 and N=8.

The multi-channel adder based LBPs ($\operatorname{maLBP}_{t 1} \ (x, y)$ for all $\ t 1 ∈-[1, c+1]$) for the middle pixel (x, y) is calculated by applying $\operatorname{maLB} P_{t 1} n(x, y)$ is defined as,

$\operatorname{maLBP}_{t 1}(x, y)=\sum_{n=1}^N \operatorname{maLBP}_{t 1^n} \ (x, y) \times f^n$          (5)

And multi-channel decoder based LBPs $\left(m d L B P_{t 2} \ (x, y) \ \right.$ for $\left.\ \operatorname{all} \ t 2 ∈ \left[1,2^c\right]\right)$ is calculated for the middle pixel (x, y) from $m d L B P_{t 2} n (x, y)$ is defined as,

7.png

Figure 7. Computation of based LBP and multichannel decoder-based LBP for c=3 and N=8

$\begin{gathered}n d L B P_{t 2}(x, y)=\sum_{n-1}^N m d L B P_{t 2}^n(x, y) \times \ f^n \text { multi - channel adder }\end{gathered}$          (6)

And then, the feature vectors are computed by applying the histogram function to the resulting adder-channel (i.e., maLBPt1) and the decoder-channel (i.e., mdLBPt2), respectively. Computation of LBP was shown in Figure 7.

3.5 Color features extraction

This research presents a CBIR approach based on the extracted color features. One of the most comprehensive elements is color, which affects an image's foreground, background, and objects. Additionally, resistant to rotation and picture translation is color. Additionally, different color properties are used in a variety of applications. The color space recognizes a specific combination of color methods and the mapping function.

The first and last photographs obtained are shown once the color edge has been found. In this case, it is essential to consider the distance between the database photos and the feature vector of the image provided as the query. If there is a sufficiently small distance between the query picture vector feature and the database images, the corresponding image in the database will be selected as a similar image. Instead of using an exact match, searches are frequently conducted on parity. Additionally, the Manhattan distance is indexed using the index of analogy. The search's outcomes are consequently arranged based on the same index.

3.6 Shape features extraction

The main objective of shape feature extraction is to capture the shape details of the various image elements, such as moment, region, boundary, etc. This extraction makes storing, transmitting, comparing and identifying the shape easier. The shape must have properties that can withstand translation, scaling and rotation. Research has been done to find the best way to pinpoint the properties of shape that would make it possible to store, transmit, or recognize the shape. In this sense, no mathematical change is involved in the selected shape attributes. The RGB color image is converted to a grayscale image to extract the shape features. The colored image has three values for every pixel.

The median filter is employed during the preprocessing stage to reduce noise. The speckle, pepper, and salt noise are reduced when the median filter is used. Additionally, the median filter is used when the blurring of edges is undesirable since it helps maintain tiny details and sharp edges in digital photographs. Almost all of the noisy data in the image is reduced when the median filter is used. To separate the pixels with extremely similar values and ignore undecided pixels from the grey image, the neuromorphic clustering algorithm is next applied.

3.7 Texture features extraction

The GLCM algorithm, which may be seen as a matrix with two dimensions of combined similarity among pairs of pixels and a distance d among them in a particular direction, is a trustworthy and effective way of classifying images using statistical analysis. 14 characteristics were retrieved and specified from the GLCM and used to categorize texture features.

1. Creating a grayscale version of the color image.

2. A 5 × 5 Gaussian filter is applied to the input image.

3. Blocks of the filtered Image measuring 4 × 4 are divided up.

4. GLCM calculates the energy, SD, mean value, homogeneity, and contrast for every block.

5. These properties are computed in 4 directions: diagonal (45 and 135 degrees), vertical (0 degrees), and horizontal (90 degrees).

6. The database contains the features that were extracted.