Spectral Similarity Based Multiscale Spatial-Spectral Preprocessing Framework for Hyperspectral Image Classification

The spectral information provides important features for identifying, detecting, and classifying material. Many spectral information similarity measures, such as Spectral Angle Mapper (SAM) [25] and Spectral Information Divergence (SID) [26], which are signature vector-based similarity measures, have been proposed for this purpose, and the Spectral Correlation Measure (SCM) [27] is a spectral correlation-based similarity measure. The SAM calculates the angle between two spectra, resulting in 0 indicating high similarity and 1 indicating poor similarity. The Spectral Angle between the spectral signature and the spectral band can be calculated as Eq. (1).

$\operatorname{SAM}\left(s_a, s_b\right)=\cos ^{-1}\left(\frac{\sum_{i=1}^n s_{a i} * s_{b i}}{\left(\sum_{i=1}^n s_{a i}^2\right)^{\frac{1}{2}} *\left(\sum_{i=1}^n s_{b i}^2\right)^{\frac{1}{2}}}\right)$        (1)

The SID calculates the distance between p and q, and the probability distributions of spectral signatures $S_a$ and $s_b$, which have $n$ spectral bands can be calculated via Eq. (2).

$\operatorname{SID}\left(s_a, s_b\right)=D\left(s_a \| s_b\right)+D\left(s_b \| s_a\right)$      (2)

where,

$D\left(s_a \| s_b\right)=\sum_{i=1}^n p_i * \log \left(\frac{p_i}{q_i}\right)$

and

$D\left(s_b \| s_a\right)=\sum_{i=1}^n q_i * \log \left(\frac{q_i}{p_i}\right)$

The SCM is calculated as the correlation coefficient between spectral signatures and spectral bands can be described by Eq. (3).

$\operatorname{SCM}\left(s_a, s_b\right)=\frac{n \sum_{i=1}^n s_{a i} * s_{b i}-\sum_{i=1}^n s_{a i} * \sum_{i=1}^n s_{b i}}{\sqrt{\left(n \sum_{i=1}^n s_{a i}^2-\left(\sum_{i=1}^n s_{a i}\right)^2\right) *\left(n \sum_{i=1}^n s_{b i}^2-\left(\sum_{i=1}^n s_{b i}\right)^2\right)}}$      (3)

The Fréchet distance is a significant similarity measure used in comparing two curves, A and B, by determining the minimum leash length required for a man on one curve to walk a dog on the other curve continuously from the starting to the ending points without backtracking [28-30]. This metric captures the minimal cost of a continuous deformation of one curve into another and defines the cost of deformation as the maximal distance between two related points [31]. The Fréchet distance is a well-studied and popular measure of similarity between curves, with various studies focusing on different aspects and extensions of this concept [32].

Alt and Godau [33] studied the Fréchet distance between polygonal curves. The discrete Fréchet distance was introduced by Eiter and Mannila [29] in 1994 as an approximate solution to the Fréchet distance, specifically based on polygonal curves where only the nodes are considered. According to Li et al. [34], spectral signatures can be treated as spectral curves. This is our main idea of using the Fréchet distance as a similarity measure of spectral signatures.

However, Neighborhood operations in hyperspectral image analysis are essential for capturing both spectral and spatial information to enhance the understanding and classification of hyperspectral data. Neighborhood operations are widely used in image processing. Simply, an operator takes the values of the neighborhood around the pixel performs some operations with them, and then writes the result back on the pixel. The general procedure is described in Figure 1.

Many neighborhood operators are implemented for image processing. For example, average, count, diversity, interspersion, maximum, median, minimum, mode, quartile, range, standard deviation, sum, variance, etc. The impact of spatial resolution on neighborhood operations is crucial as it directly affects the level of detail and accuracy in spatial data analysis. Higher spatial resolution enables more precise delineation of features within a neighborhood, facilitating better identification of patterns and relationships. However, finer spatial resolutions often result in larger datasets, leading to increased computational complexity and processing time [35]. Conversely, lower spatial resolutions may result in information loss and reduced ability to capture intricate spatial variations within neighborhoods [36].

01.jpg

Figure 1. The general procedure of neighborhood operators

Spectral similarity measures present a method to mitigate the constraints imposed by spatial resolution in neighborhood operations. By concentrating on the spectral attributes of data rather than solely relying on spatial information, spectral similarity measures aid in identifying similarities and differences between spatial entities based on their spectral signatures. This approach allows for the comparison of spatial features at a spectral level, enabling the detection of patterns and relationships that may not be discernible from spatial data alone.

To address this limitation, we propose a similarity-based weighted neighborhood algorithm. This algorithm integrates spectral similarity measures into the neighborhood analysis, assigning weights to neighboring pixels based on their spectral similarity to the target pixel.

This similarity-based weighted neighborhood algorithm effectively enhances the analysis by emphasizing spectrally similar pixels, thereby improving the accuracy and reliability of neighborhood operations without solely depending on spatial resolution.

The influence of spatial resolution on neighborhood operations is multifaceted, impacting the level of detail, computational demands, and information loss in spatial analyses. Spectral similarity measures offer a complementary strategy to address the limitations of spatial resolution by focusing on spectral characteristics. By integrating a similarity-based weighted neighborhood algorithm, we can further enhance the analysis and interpretation of spatial data in neighborhood studies, providing a more accurate and detailed understanding of spatial patterns and relationships. For some improvement, the observations can be weighted according to their distances to the current target pixel. Therefore, nonlinear functions can be applied as weighting functions for neighborhood operators. The most classical method is the Gaussian filter. The Gaussian filter computes a weighted average of the neighborhood surrounding each pixel, with the central pixel's value exerting a greater influence on the average. The main idea of this paper is to use spectral similarities, especially the Fréchet distance, as a weighting for the weighted average neighborhood operator.

In this section, we introduce a novel multiscale spectral-spatial preprocessing technique founded upon a weighted average neighborhood operator incorporating spectral similarity-based weighting. The algorithm is predicated on the premise that individual pixel vectors within a hyperspectral image exhibit spectral attributes akin to those of neighboring pixel vectors, given the prevalent continuity observed in hyperspectral imagery. For this reason, each pixel vector in a hyperspectral image will likely have similar characteristics to the weighted average of neighboring pixel vectors. To improve the characteristic similarity, the weighting function is highly important. Due to their high spectral dimensionality, spectral signatures can be represented as curves. The Fréchet distance is one of the well-known similarity measures of curves such as L-Curve, and Dynamic Time Warping (DTW). The Fréchet distance is a widely recognized curve similarity measurement method known for its effectiveness in assessing the similarity between two curves [39-41]. This metric has been extensively studied and utilized in various fields, showcasing its versatility and applicability in curve analysis. The Fréchet distance is particularly favored for its robustness and ability to accurately capture the resemblance between complex curves.

The L-curve method has gained significant popularity as a prevalent technique for choosing the regularization parameter in various applications [42, 43]. This method, known for its efficiency and reliability, utilizes curve characteristics to determine optimal parameters, contributing to enhanced model performance and accuracy.

In addition, Dynamic Time Warping (DTW) is widely adopted as a similarity measure for time series data, offering a robust approach to aligning and comparing temporal sequences [44, 45]. DTW has proven to be instrumental in various fields, including computer science and signal processing, where precise alignment and comparison of time-dependent data are crucial.

As a result of this information, the Fréchet distance can be used as the weight of neighboring pixels in the weighted average operation. The mechanics of the weighted average operator are given in Figure 2.

02.jpg

Figure 2. The mechanics of the weighted average operator

The flowchart illustrating the proposed methodology for the hyperspectral image classification can be observed in Figure 3.

03.jpg

Figure 3. Flowchart of the proposed methodology

The proposed method involves the following steps:

1. Determining the Scale Size of the Window: The process begins by defining the size of the neighborhood window, which determines the spatial extent considered for analysis. This step involves selecting an appropriate scale to capture relevant spatial information effectively.

2. Selecting the Pixel Vector of the Interest-Centered Neighborhood Window: Once the window size is determined, the pixel vector of interest is selected within this neighborhood window. This pixel vector serves as the focal point for subsequent calculations.

3. Calculating Distances and Obtaining Weight Values: Distances between the pixel vector of interest and its neighboring pixel vectors are calculated using the Fréchet distance. These distances are used to derive weight values for each neighboring pixel vector, indicating their influence on the focal pixel.

4. Applying a Weighted Average Operator: A weighted average operator is then applied to calculate the new value of the pixel vector of interest. This operator considers the calculated weight values to adjust the contribution of neighboring pixels to the final value.

5. Multiscale Processing: Apply steps 2-4 iteratively for all scales.

6. Returning the Resulting Image to the Classifier: After applying steps for all scales, the resulting image, modified based on the weighted averaging process, is returned to the classifier. This modified image is used to create a classification map, enabling the classification of different regions within the hyperspectral image based on the processed pixel values.

In the first step, the neighborhood window size is defined, and the image is spatially padded. In the second step, the pixel vector of interest is selected and the neighborhood window is centrally located. In the third step, distances are calculated using the Fréchet distance, and the weight values of each neighboring pixel vector are obtained via Eq. (7).

$w_y=1-f_{\mathrm{SS}}(x, y)$       (7)

where, $x$ is the current pixel vector of interest, $y$ is one of the neighboring pixel vectors, and $f_{S S}$ is the spectral similarity function used to calculate the weight of pixel $y$, $w_y \in[0,1]$.

Finally, the weighted average operation is applied via Eq. (8).

$g=\frac{\sum_{i=1}^n w_i I_i}{\sum_{i=1}^n w_i}$      (8)

where, I is the neighborhood window which contains $n$ pixel vectors, $w_i$ is the weight of the ith pixel vector and $g$ is the weighted average vector of the pixel vector of interest.

These steps are repeated by a neighborhood window for each pixel vector in the image until all scale sizes are applied. According to the above descriptions, more similar neighboring pixel vectors have greater weights because of their proximity to zero. In the realm of hyperspectral image classification, the method can be described in Figure 4.

04.jpg

Figure 4. Pseudocodes of the proposed method

6.1 Indian pines dataset

In the context of the Indian Pines [46] dataset, a fraction of the data, specifically 0.5%, 1%, and 5% of the samples, were randomly selected within each class to construct the learning sets, while the remaining data constituted the test set. The classification maps resulting from the application of the Support Vector Machine (SVM) classifier on the Indian Pines dataset, particularly when 5% of the samples were chosen within each class for the training set, are visually represented in Figure 5. Additionally, Table 13. provides a detailed overview of the classification accuracy for each class within the Indian Pines dataset, specifically focusing on scenarios where 5% of the samples were selected within each class for the training set, with a specific emphasis on the SVM classifier.

The experimental setup described aligns with the methodology commonly employed in hyperspectral image analysis research, where the selection of training samples from each class is crucial for evaluating the performance of classification algorithms [51]. By utilizing the SVM classifier and varying the percentage of samples chosen for training within each class, a comprehensive assessment of the classification accuracy across different classes within the Indian Pines dataset was conducted. This rigorous evaluation approach provides valuable insights into the robustness and efficacy of the proposed methodology in handling hyperspectral data with varying sAample sizes and class distributions, contributing significantly to the advancement of research in hyperspectral image classification.

05.jpg

Figure 5. Classification maps obtained from the SVM classifier on the Indian Pines dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 13. Classwise, Average and Overall classification accuracy (%) of SVM on the Indian Pines dataset

06.jpg

Figure 6. Classification maps obtained from the KELM classifier on the Indian Pines dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 14. Classwise, Average and Overall classification accuracy (%) of the KELM classifier on the Indian Pines dataset

Figure 6 depicts the outcomes of classification maps derived from the Kernel Extreme Learning Machine (KELM) classifier applied to the Indian Pines dataset, with a 5% sample selection in each class for the training set. The classwise classification accuracy of the KELM algorithm on the Indian Pines dataset under the same 5% sample selection in each class for the training set is presented in Table 14.

A comparison between the proposed method and other known methods is given in Table 15. Statistical comparison of proposed method is given in Table 16.

Table 15. Comparison between the proposed method and other known methods on the Indian Pines dataset

Table 16. Statistical Comparison of Proposed Method

* P-Value < 0,05 denoted as +, otherwise denoted as -

6.2 PaviaU dataset

In the PaviaU [47] dataset, the entirety of the dataset is composed of samples randomly selected at rates of 0.5%, 1%, and 5% from each class to form the learning sets, with the remaining data constituting the test set. The classification maps resulting from the SVM classifier applied to the PaviaU dataset, particularly when 5% of the samples are chosen from each class for the training set, are visually depicted in Figure 7. Additionally, Table 17. provides a detailed overview of the classification accuracy of each class within the PaviaU dataset when 5% of the samples are utilized from each class for the training set, employing the SVM classifier. This meticulous experimental setup ensures a comprehensive evaluation of the proposed methodology's performance across various class distributions, shedding light on its effectiveness in handling different training set sizes and class imbalances in hyperspectral image classification scenarios.

In this section, Figure 8 illustrates the outcomes of classification maps derived from the Kernel Extreme Learning Machine (KELM) classifier applied to the PaviaU dataset, with a 5% sample selection in each class for the training set. Furthermore, Table 18 presents the classwise classification accuracy on the PaviaU dataset under the condition of 5% sample selection in each class for the training set, specifically for the KELM classifier.

07.jpg

Figure 7. Classification maps generated by the SVM classifier on the PaviaU dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 17. Classwise, Average and Overall classification accuracy (%) of the SVM classifier on the PaviaU dataset

80.jpg

Figure 8. Classification maps obtained from the KELM classifier on the PaviaU dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 18. Classwise, Average and Overall classification accuracy (%) of the KELM classifier on the PaviaU dataset

09.jpg

Figure 9. Classification maps obtained from the SVM classifier on the Salinas dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 19. Comparison between the proposed method and other known methods on the PaviaU dataset

Table 20. Statistical comparison of proposed method

* P-Value < 0,05 denoted as +, otherwise denoted as –

A comparison between the proposed method and other known methods is given in Table 19. Statistical comparison of proposed method is given in Table 20.

6.3 Salinas dataset

In the Salinas [47] dataset, a fraction of 0.5%, 1%, and 5% of the samples were randomly chosen within each class to form the training sets, while the remaining data was designated for the test set. The classification maps generated by the Support Vector Machine (SVM) classifier on the Salinas dataset, particularly when 5% of the samples were selected within each class for the training set, are visually depicted in Figure 9. Additionally, Table 21. provides a detailed overview of the classification accuracy achieved for each class in the Salinas dataset under the scenario where 5% of the samples were utilized within each class for the training set, specifically focusing on the SVM model.

Figure 10 depicts the outcomes of classification maps derived from the Kernel Extreme Learning Machine (KELM) classifier applied to the Salinas dataset, with a 5% sample selection in each class for the training set. The class-specific classification accuracy for the Salinas dataset under the same conditions is presented in Table 22 for the KELM classifier.

A comparison between the proposed method and other known methods is given in Table 23. Statistical comparison of proposed method is given in Table 24.

Table 21. Classwise, Average and Overall classification accuracy (%) of the SVM classifier on the Salinas dataset

010.jpg

Figure 10. Classification maps obtained from the KELM classifier on the Salinas dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 22. Classwise, Average and Overall classification accuracy (%) of the KELM classifier on the Salinas dataset

Table 23. Comparison between the proposed method and other known methods on the Salinas dataset

Table 24. Statistical Comparison of Proposed Method

* P-Value < 0,05 denoted as +, otherwise denoted as -

6.4 Kennedy space center dataset

In the analysis of the Kennedy Space Center (KSC) [48]  dataset, a sampling approach was employed wherein data subsets of 0.5%, 1%, and 5% were randomly selected within each class for the training sets, with the remaining data comprising the test set. The outcomes of the classification maps generated by the Support Vector Machine (SVM) classifier on the Kennedy Space Center dataset, utilizing a 5% sample selection in each class for the training set, are depicted in Figure 11. Furthermore, Table 25. presents the classwise classification accuracy of the SVM on the Kennedy Space Center dataset under the condition of 5% sample selection in each class for the training set.

011.jpg

Figure 11. Classification maps obtained from the SVM classifier on the Kennedy Space Center dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

012.jpg

Figure 12. Classification maps obtained from the KELM classifier on the Kennedy Space Center dataset

a) Proposed, b) SAM, c) SID, d) SCM, and e) Groundtruth

Table 25. Classwise, Average and Overall classification accuracy (%) of the SVM classifier on the KSC dataset

Table 26. Classwise, Average and Overall classification accuracy (%) of the KELM classifier on the KSC dataset