Rapid Classification of Massive Images Based on Cloud Computing Platform

With the progress of information technology and the rapid development of image data acquisition technology, a large amount of multimedia data is generated every day in all walks of life, among which digital image data accounts for a large part [1-6]. Face with the explosive growth of digital image data, the traditional stand-alone mode of image processing has exposed many problems, such as slow processing speed and poor concurrency, etc. [7-12]. Therefore, the traditional image processing mode has gradually failed to meet the needs of users. Now it is particularly necessary to seek a new and efficient image processing mode [13-20]. As a new calculation pattern, cloud computing is a typical distributed and parallel computing model, which can greatly shorten the execution time of computing tasks. With the continuously increasing data processing speed requirements, it is more and more suitable to process images on cloud computing platform with distributed parallel computing.

With the advent of the era of big data, various types of image data has been growing explosively. The traditional image classification algorithm has low efficiency and accuracy problems, which can not meet the processing requirements of massive image data. For the above problems, Wang et al. [21] proposed a massive image classification method based on parallel hybrid classifier algorithm, which combined Adaboost algorithm with Radial Basis Function (RBF) algorithm, and incorporated multiple RBF classifiers into a strong classifier. In addition, MapReduce parallel programming model was used to parallelize the Adaboost RBF algorithm. The performance test results showed that the algorithm met the automatic classification needs of massive images. For the problem of low image classification performance in single-node computing model, Hu et al. [22] integrated the gray level with image texture using Gray Level Concurrence Matrix (GLCM), and designed a MapReduce-based parallel fuzzy C-means clustering method, which improved the real-time classification performance of massive images. The results showed that this method had high real-time processing efficiency in massive image classification. The cloud computing platform, which has achieved excellent commercial results, was used for parallel classification processing of massive remote sensing data, which met the requirements of improving the parallel processing efficiency. Feng and Wu [23] aimed to study the application of distributed cloud computing architecture in big data-based hyperspectral remote sensing image classification. The co-authors described related cloud computing technologies, and proposed Spatial Correlation regularized Sparse Representation Classification (SCSRC) model of sparse representation classification method based on spatial correlation constraints. In addition, the co-authors proposed a distributed classification algorithm DP-SCSRC on Spark platform based on spatial correlation constraint sparse representation by analyzing the principle and the solving method of the SCSRC model, and designed optimization methods for the core steps. Thakur and Pillai [24] proposed that these methods showed some limitations when the input images contained a large amount of noise contents and fuzzy degrees, such as unsatisfactory image classification results and low accuracy. Then the co-authors analyzed and summarized various image classification technologies.

In conclusion, the application of distributed parallel processing technology to massive image processing will become an important research direction. As a typical distributed and parallel computing mode, cloud computing has efficient computing processing ability, which greatly shortens the execution time of computing tasks. With the need for continuous increase of data processing speed, it is more and more suitable for images to be processed on cloud computing platform with distributed parallel computing. Therefore, this paper did research on the rapid classification of massive images based on cloud computing platform. The second part of the paper presented the multi-mode information fusion of massive images, and gave the implementation steps of the proposed massive image fusion method based on information complementation. For the massive image classification tasks, the third part of the article proposed to classify the massive images based on the salient interactive feedback network. The constructed model not only simplified the complicated network parameter initialization and tuning process, but also avoided the classification failure caused by scarce labeled images, which is more suitable for application and practical application scenarios. The experimental results verified the model was effective in rapid massive image classification.

3.png

Figure 3. Classification model framework of massive images

For the tasks of classifying massive images, this paper proposed to classify them based on salient interactive feedback network. The network model mainly aimed to train a massive image classification model by combining the interaction with the feedback between the image target and the salient local region of the image. Figure 3 shows the framework of massive image classification model. Compared with the traditional classification model, this model not only simplified the complicated network parameter initialization and tuning process, but also avoided the classification failure caused by the scarcity of labeled images, showing it is more suitable for application and practical application scenarios.

The model set the salient feature extraction and refinement modules, which were used to capture the salience of image target and local region of the image and refine the extracted salient feature. Let V and U be the height and width of the input image, F, Q and D be the height, width and channel number of the corresponding feature map, respectively, Q_d be the trainable parameter in the convolutional neural network and * be the convolution, pooling and activation operation. In the salient feature extraction module, $S Q \in R^{V \times U \times 3}$, the given image fused in the previous section, was input into the pre-trained convolutional neural network to extract the primary feature $G \in R^{F \times Q \times 3}$. The following formula showed the extraction process of G:

$G=Q_d * S Q$                (8)

In order to obtain better extraction effects of salient feature, this paper explored the image target and candidate local region of the image in SQ using 1×1 convolution kernel, and represented the exploration region through the class activation map $X \in R^{F \times Q \times D}$. Let Q_g be the trainable parameter in the 1×1 convolution kernel, then:

$X=Q_g * G$                      (9)

In order to generate the class activation map and cover the expected discriminative region as much as possible, this paper generated the class activation map based on top- k average activation operation, and adjusted the response of the class activation map channel to different regions using the weight b, generated by the function softmax. Let b_i be the weight of the i-th channel and X_x be X after global average pooling operation, then the following formula gave the expression of the weight vector b=[b₁, b₂,..., b_i,..., b_D]:

$b_i=\frac{\exp \left(X_{x(i)}\right)}{\sum_{i=1}^D \exp \left(X_{x(i)}\right)}$                                     (10)

Due to b representing the discrimination of X, the salient feature extraction module adjusted the influence of class activation map on the exploration region using b. Initial attention map $N \in R^{F \times Q \times D}$, which balanced the coverage region and the discrimination ability of these regional features, was built through channel weighting of X, that is:

$N=b \otimes X$                              (11)

In order to obtain a complete salient image target, this paper designed an image target flow, which focused on the region related to semantics of the image target. Let X_i be the weight of the class activation map in the i-th channel and the cross-channel average pooling of X was performed, then:

$R=\frac{\sum_{i=1}^D X_i}{D}$                              (12)

In order to make full use of R, this paper also constructed an attention gating sub-module to locate the region related to the image target. In order to meet the functional requirements of both capturing the spatial relationship in the image and extracting the region highly responsive to the final classification, this module needed to include the attention sub-module shown in the following formula. Let $N_x \in R^{F \times Q}$ be the attention map with context and Q_r and Q_x be the learnable parameters corresponding to the two convolution layers contained in the attention sub-module, then:

$N_x=\operatorname{sigmoid}\left(Q_x \cdot \tanh \left(Q_r \cdot R\right)\right)$                              (13)

Let $N_t \in R^{F \times} Q$ be the attention map saliently related to the image target and G_tr be the salient feature related to semantics of the image target. This module summarized the features of the fused image in each channel direction, and N_t and G_tr were obtained by processing the summary $N_{\text {Total }} \in R^{F \times Q}$ using the gating sub-module. The following gave the corresponding calculation formula:

$N_t=\operatorname{sigmoid}\left(Q_n \cdot \tanh \left(\left(Q_m \cdot N_x\right) \otimes N_{s u m}\right)\right)$                              (14)

$G_{t r}=X \otimes N_t$                              (15)

In order to extract the salient feature of local region of the image, a local flow was set up to learn from multiple detectors, which could perceive the whole local region. Therefore, by taking into consideration the maximum response of different channels in the fused image, which reflected the most discriminative local region in different channels, the coordinates of each local region were represented as {(a₁, b₁), (a₂, b₂), (a_i, b_i), (a_m, b_m)}, where (a_i, b_i) are the coordinates of the i-th local region in the fused image, and m is the number of detected local regions. Let Q_E and Q_s be the parameters of the two convolution layers and $N_{e(i)} \in R^{F \times Q}$ be the attention map of the i-th channel. Similarly, the attention map $N_E \in R^{F \times Q \times m}$ related to local semantics in local flow could be obtained by the following formula:

$N_E=\operatorname{sigmoid}\left(Q_s \cdot \operatorname{tabh}\left(Q_E \cdot N_{\text {norm }}\right)\right)=\left\{N_{e(1)}, N_{e(2)}, \ldots, N_{e(i)}, \ldots, N_{e(m)}\right\}$                           (16)

The extraction of local salient features included two steps: initializing the flow parameters and adjusting the parameters with the activation ratio. Let N_e be m attention maps related to local semantics and G_er=[G_er(1), G_er(2), ..., G_er(m)] be the generated salient features related to local semantics, and G_er contained m local salient features, then the specific calculation formula was as follows:

$G_{e r}=X \otimes N_e$                              (17)

The design of salient feature refinement module aimed to refine the salient features of the image target and local region of the image, and synthesize them into advanced features with discrimination, which was conducive to the rapid recognition of massive images. Specifically, the salient amplifier module was introduced to generate image target and its local region based on salient amplification. Then the attention gating sub-module and the attention sub-module were used to locate the image target and its local region, and the target-level and local-level salient amplification features were generated.

Let G_tr be the target-level salient amplification feature, G_e be the local-level salient amplification feature, and $N_{e x} \in R^{F \times Q \times m}$ and $G_{e f} \in R^{D \times m}$ be the local attention map and its advanced feature, respectively. In order to locate each independent local region in the image target, this paper modeled the similarity between G_tr and G_e in the image target flow and the local flow, and guided G_e to be constrained in the image target. Let Q_t, Q_l and Q_k be the learnable weight parameters, respectively, then:

$N_{e x}=\operatorname{sigmoid}\left(Q_k \cdot \tanh \left(Q_t \cdot G_{t r}+Q_l \cdot G_e\right)\right)$                             (18)

$G_{e f}=G_e \otimes M_{e x}$                             (19)

The salient features of all local regions in G_er were supplemented further in the local-image target flow. The G_er(l) and G_t of each local region were integrated in the form of G_er(l)⨂G_t to generate the advanced feature $G_{t f} \in R^{D \times m}$ of the image target. Let h(*) be the global maximum pooling and then:

$G_{t f}=\left(\begin{array}{l}h\left(G_{e r(1)} \otimes G_t\right) \\ h\left(G_{e r(2)} \otimes G_t\right) \\ \vdots \\ h\left(G_{e r(m)} \otimes G_t\right)\end{array}\right)$                             (20)

In order to optimize the advanced features generated, the network information affecting the optimization effects of salient feature was readjusted in the back-propagation form layer by layer. This paper proposed a comprehensive calibration loss L_sK_r, which realized the construction of feedback between advanced feature and salient feature. Let K_x and K_g be the attention and feature calibration loss functions, respectively, then, K_r, which could achieve the calibration salient feature, was calculated by the following formula:

$K_r=K_x+K_g$                           (21)

Let n_ex(i,j) and n_e(i,j) be the elements in the attention map corresponding to N_ex and N_e of each local region, respectively, and n_tx(i,j) and n_t(i,j) be the elements in the attention map corresponding to N_tx and N_t of the image target, respectively, then K_x was represented as:

$\begin{aligned} & K_x\left(N_{e x}, N_e\right)=\sum_{i, j=1}^{(F, O)}\left\|n_{e x}(i, j)-n_e(i, j)\right\|^2 \\ & K_x\left(N_{t x}, N_t\right)=\sum_{i, j=1}^{(F, Q)}\left\|n_{t x}(i, j)-n_t(i, j)\right\|^2\end{aligned}$                          (22)

In addition, in order to achieve the goal of salient feature gradually approaching the advanced feature, this paper set the Hellinger distance between them as the feature calibration penalty term. Let G_er(j) and G_ef(j) be the salient feature and advanced feature of each local region on the j-th feature channel, respectively, G_tr(j) and G_tf(j) be the salient feature and advanced feature of the image target on the j-th feature channel, respectively, and D₁ and D₂ be the number of feature channels of the local region and image target, respectively, then:

$\begin{aligned} & K_g\left(N_{e r}, N_{e f}\right)=\frac{1}{\sqrt{2}} \sum_{j=1}^{D_1}\left\|\sqrt{G_{e r(j)}}-\sqrt{G_{e f(j)}}\right\|_2 \\ & K_g\left(N_{t r}, N_{t f}\right)=\frac{1}{\sqrt{2}} \sum_{j=1}^{D_2}\left\|\sqrt{G_{t r(j)}}-\sqrt{G_{t f(j)}}\right\|_2\end{aligned}$                          (23)