Dust Detection on Solar Panels: A Computer Vision Approach

Automated inspection techniques have been widely adopted in the industry sector in the last two decades to detect defect regions on solar panels or to identify the defect type and its existence on these panels. These defects are formulated, from the perspective of image processing, as local textural features [1]. Automated inspection of solar panels is an alternate of human visual inspection, where real-time feedback is required when a certain defect occurs because of labor costs and human distraction that affect the decision [2]. Solar panels are devices engineered to transform sunlight into electricity, capturing solar energy and converting it into usable electrical power.

In Middle East countries, although solar radiation intensity is very high, desert sands and dust accumulation represent a natural barrier that can significantly impair their capacity to generate energy [3]. Over time, dust, grime, and various particles settle on the panel surfaces, forming a fine layer that hampers sunlight from reaching the photovoltaic cells. This layer diminishes the panels' efficiency in absorbing and transforming solar energy into electricity. Consequently, the Photovoltaic (PV) system's energy output declines, resulting in a substantial reduction in energy production [4, 5]. Consistent monitoring and cleaning are imperative to mitigate the detrimental impacts of dust accumulation, ensuring that solar panels sustain their optimal functionality, thereby maximizing their contribution to clean and sustainable energy generation.

Existing technologies for dust detection range from manual inspection to automated systems utilizing drones, robots, or sensors. Manual inspections are cost-effective but time-consuming and often impractical for large-scale installations. Automated methods relying on vision technology offer advantages such as increased efficiency, accuracy, early detection of defects, and reduced labor costs [6]. However, they may come with disadvantages including initial investment costs, maintenance requirements, and potential limitations due to lighting conditions and variations in surface texture detecting certain types of contaminants or in challenging environmental conditions [7]. In addition, current proposed dust detection techniques have many shortcomings including short-range acquisition and considering moderate and heavy dust densities on solar panels in desert areas. Nevertheless, despite these limitations, solar panel inspections remain indispensable for detecting dust accumulation, optimizing performance, initiating cleaning actions, and ensuring the secure and efficient operation of PV systems. Striking a balance between effectiveness, cost, and practicality is essential in implementing a reliable dust detection strategy for PV panels.

This paper proposes a computer vision approach to inspect the solar panel condition in terms of dust accumulation. The proposed approach has the ability to classify automatically solar panels to clean and dust-accumulated regardless of lighting conditions and texture variations. The proposed approach introduces a feature description by the fusion of both gray level co-occurrence matrix (GLCM) textural features and uniform local binary patterns (LBP). This approach can be integrated with automated monitoring and cleaning systems of solar panels that combine technologies such as an unmanned aerial vehicle (UAV) and digital imaging. This work has the following contributions:

• Constructing a visible light image dataset of solar panels with moderate and heavy dust accumulation. These images were acquired vertically on the solar panel with an acquisition range between 1.5-4 m.
• Implementing a dust detection model that has the ability to classify solar panels to either clean or dust-accumulated from visible light images.
• Proposing an optimized descriptor of solar panels by fusing the local features of LBP with the global features of GLCM.
• Validating the efficiency of feature descriptor in the domain of dust detection under different lighting conditions.

Furthermore, the paper is structured as follows: Section 2 demonstrates a review of prior research. Section 3 presents a mathematical background of the computer vision and artificial intelligence techniques used in this work. The proposed dust detection approach is presented in Section 4 while the experimental findings along with performance measures are demonstrated in Section 5. Finally, Section 6 provides the conclusions.

3.1 Gray level co-occurrence matrix

The gray level co-occurrence matrix (GLCM) is typically deployed to extract global textural features of an image. It is defined as the distribution of co-occurring gray levels at a given offset: direction (θ) and distance (d). The relative frequencies of two neighboring pixels, one with gray level i and the other with gray level j, are calculated from the quantized image to construct the co-occurrence matrix P_ij. This co-occurrence matrix (P_ij) can be calculated with different values of d such as: 1, 2, and 3 and different values of θ such as: 0, 45, 90, and 135 as shown in Figure 1.

1.png

Figure 1. GLCM with different offsets

Several textural features can be derived from the co-occurrence matrix P_ij, including Energy, Contrast, Correlation, Homogeneity, Entropy, Autocorrelation, Dissimilarity, and Cluster Shade [20]. The following equations define some of these features where p(i, j) is the (i, j)th cell in the co-occurrence matrix, µ_x and µ_y are the mean values for rows and columns of the matrix, σ_x and σ_y are the standard deviation values for rows and columns of the matrix, N_g is the number of gray levels in the quantized image [20-22]:

$\mu_x=\sum_i \sum_j i \cdot p(i, j)$          (1)

$\mu_y=\sum_i \sum_j j \cdot p(i, j)$                (2)

$\sigma_{\mathrm{x}}=\sum_{\mathrm{i}} \sum_{\mathrm{j}}\left(\mathrm{i}-\mu_{\mathrm{x}}\right)^2 \cdot \mathrm{p}(\mathrm{i}, \mathrm{j})$                       (3)

$\sigma_y=\sum_i \sum_j\left(i-\mu_y\right)^2 \cdot p(i, j)$                         (4)

The textural features are defined as follows [20-22]:

• Energy

$\mathrm{f}_1=\sum_{\mathrm{i}} \sum_{\mathrm{j}} \mathrm{p}(\mathrm{i}, \mathrm{j})^2$                   (5)

• Contrast

$f_2=\sum_{n=0}^{N_g-1} n^2\left\{\sum_{i=1}^{N_g} \sum_{\substack{j=1 \\|i-j|=n}}^{N_g} p(i, j)\right\}$                           (6)

• Correlation

$\mathrm{f}_3=\frac{\sum_{\mathrm{i}} \sum_{\mathrm{j}}(\mathrm{ij}) \mathrm{p}(\mathrm{i}, \mathrm{j})-\mu_{\mathrm{x}} \mu_{\mathrm{y}}}{\sigma_{\mathrm{x}} \sigma_{\mathrm{y}}}$                         (7)

• Homogeneity

$f_4=\sum_i \sum_j \frac{1}{1+(i-j)^2} p(i, j)$                    （8）

• Entropy

$f_5=-\sum_i \sum_j p(i, j) \log (p(i, j))$                    （9）

• Autocorrelation

$\mathrm{f}_6=\sum_{\mathrm{i}} \sum_{\mathrm{j}}(\mathrm{ij}) \cdot \mathrm{p}(\mathrm{i}, \mathrm{j})$                    （10）

• Dissimilarity

$f_7=\sum_i \sum_j|i-j| \cdot p(i, j)$                  （11）

• Cluster Shade

$f_8=\sum_i \sum_j\left(i+j-\mu_x-\mu_y\right)^3 \cdot p(i, j)$                         （12）

3.2 Local binary pattern

Local Binary Pattern (LBP) is a feature extraction operator designed for grayscale images that is invariant to both rotation and scale [23, 24]. It aims to capture local texture characteristics within an image. To compute the LBP value for a specific pixel location (X_c, Y_c), a comparison is made between a pixel value and its neighbors. If the neighbor pixel value surpasses the target pixel value, it is labeled as 1; otherwise, it is labeled as 0. The number of neighboring pixels considered in this comparison can range from 4, 8, 12, 16, to 24. When comparing with the eight immediate neighboring pixels, this process yields an 8-bit binary code. This binary code can then be converted into a decimal value, which is subsequently assigned as the new pixel value for the central pixel, as shown in Figure 2.

1.png

Figure 2. Schematic of LBP with eight pixel surroundings

LBP can be computed according to Eq. (13) as:

$\operatorname{LBP}_{\mathrm{P}}\left(\mathrm{X}_{\mathrm{c}}, \mathrm{Y}_{\mathrm{c}}\right)=\sum_{\mathrm{p}=0}^{\mathrm{P}-1} \mathrm{~S}\left(\mathrm{~g}_{\mathrm{p}}-\mathrm{g}_{\mathrm{c}}\right) 2^{\mathrm{p}}$                    (13)

where, ɡ_c is the gray level of the centered pixel, ɡ_p is the gray value of neighboring pixels, P is the number of surrounding pixels, and S is a function defined as:

$S(x)= \begin{cases}0, & x<0 \\ 1, & x \geq 0\end{cases}$                    (14)

The LBP_p operator produces 2P different local binary patterns. Rotation-invariant LBP can be computed according to Eq. (15) as:

$\mathrm{LBP}_{\mathrm{P}}^{\mathrm{ri}}=\min \left\{\operatorname{ROR}\left(\mathrm{LBP}_{\mathrm{p}}, \mathrm{i}\right) \mid \mathrm{i}=0,1, \ldots \ldots, \mathrm{P}-1\right\}$                     (15)

where, ROR(x,i) function applies a circular right shift on the P-bit number I times and thus; achieving 36 unique rotation invariant local binary patterns in case of LBP_P. These Rotation-invariant LBP describe bright spots, dark spots, and edges [25].

In the study by Ojala et al. [23], an enhanced version of the Local Binary Pattern (LBP) is introduced, which incorporates a uniformity measure denoted as “U.” This measure is defined as the count of spatial transitions, representing bitwise changes from 0 to 1 or vice versa, within the local binary pattern. For instance, a pattern like “111111112” is assigned a U value of 0, while a pattern like “000011112” is given a U value of 2. This uniformity measure serves to categorize local binary patterns into two distinct groups: “uniform” patterns, which are intrinsic properties of texture and exhibit a U value of no more than 2, and “nonuniform” patterns. The formula for the modified LBP is then expressed as follows:

$\operatorname{LBP}_{\mathrm{P}}^{\mathrm{riu} 2}=\left\{\begin{array}{lr}\sum_{\mathrm{p}=0}^{\mathrm{P}-1} \mathrm{~S}\left(\mathrm{~g}_{\mathrm{p}}-\mathrm{g}_{\mathrm{c}}\right) & \text { if } \mathrm{U}\left(\mathrm{LBP}_{\mathrm{P}}\right) \leq 2 \\ \mathrm{P}+1 & \text { otherwise }\end{array}\right.$                 (16)

This definition results in the creation of P+1 uniform patterns and 1 nonuniform pattern. The nonuniform pattern encompasses all patterns with a U value greater than 2. As a result, the total number of LBP patterns is reduced to P+2 patterns. In case of P=8, nine uniform patterns and one Non-uniform pattern, as shown in Figure 3, would be produced.

1b.png

Figure 3. Examples of LBP8 patterns: (a) The nine Uniform patterns and (b) Non-uniform patterns

The histogram of LBP patterns, collected across a surface image, serves as the feature descriptor for characterizing surface texture. In the study by Ojala et al. [23], the researchers came to the conclusion that the histogram of uniform patterns outperforms histograms based on other LBP pattern types in terms of discrimination capabilities.

3.3 Support vector machine

The Support Vector Machine (SVM) is essentially a statistical binary classifier that employs kernel functions. When employing a sigmoid kernel function, an SVM shows similarities to the response of a two-layer feedforward neural network. However, SVM demonstrates greater efficiency than neural networks when dealing with intricate problems involving extensive datasets and high dimensionality, and it is adept at mitigating overfitting concerns [26-28].

The core concept of SVM is to identify the optimal hyperplane that maximizes the margin between the hyperplane and the closest data points from both classes. SVM can be formulated as follows:

If input space is: {x₁, x₂, x₃, …, x_n}

and output space is: $y \in\{-1,1\}$

The hyperplane that separates the two classes is expressed as [28-30]:

$\overrightarrow{\mathrm{w}} \cdot \overrightarrow{\mathrm{x}}+\mathrm{b}=0$              (17)

here, w (weight) denotes the orthogonal vector defining the orientation of the hyperplane, while b (bias) represents the distance from the origin to the hyperplane.

Training instances must satisfy:

$\overrightarrow{\mathrm{w}} \cdot \overrightarrow{\mathrm{x}}+\mathrm{b} \geq 1$ for $\mathrm{y}=+1$                   (18)

$\overrightarrow{\mathrm{w}} \cdot \overrightarrow{\mathrm{x}}+\mathrm{b} \leq-1$ for $\mathrm{y}=-1$                       (19)

The previous inequalities are merged to obtain:

$y(\vec{w} \cdot \vec{x}+b)-1 \geq 0$              (20)

This can be formulated as:

$\operatorname{maximize} \frac{2}{\|\mathrm{w}\|}$ or minimize $\frac{1}{2}\|\mathrm{w}\|^2$                  (21)

Such that: $\mathrm{y}(\overrightarrow{\mathrm{w}} \cdot \overrightarrow{\mathrm{x}}+\mathrm{b})-1 \geq 0$

For solving Eq. (21), a Lagrange multiplier is deployed and it yields:

$\min L_p \equiv \frac{1}{2}\|\vec{w}\|^2-\sum_{i=1}^l \alpha_i y_i\left(\vec{w} \cdot \overrightarrow{x_1}+b\right)+\sum_{i=1}^l \alpha_i$            (22)

Such that αi ≥ 0 where α is the Lagrange multiplier.

The result is a decision function of Lagrange multipliers α and bias (b) for test instance x_t:

$y_t=\sum_{i=1}^1\left(\alpha_i y_i<\overrightarrow{x_1} \cdot \overrightarrow{x_t}>+b\right)$             (23)

The proposed approach for inspecting solar panels’ conditions in terms of dust accumulation has undergone validation using images of both clean and dust-accumulated solar panels. These images were captured using a camera and adjusted to a 650 × 870 pixel scale through numerical fractioning to counteract object distortion. Implemented within MATLAB software running on a 2.4-GHz i3 CPU, this inspection approach utilized a dataset comprising 213 solar panel images for both training and testing phases. We have allocated 75% of the images for the training set and 25% for the testing set. Table 1 shows the exact division of dataset images among the classification phases and the solar panels’ conditions.

Table 1. Solar panel images dataset division

Each solar panel image in the dataset has had its feature descriptor extracted, becoming its representative signature. This descriptor is a combination of the histogram of uniform LBP patterns and eight textural features computed from the GLCM.

In Figure 7, examples of clean and dust-accumulated solar panel images are displayed alongside the features extracted from both the LBP Uniforms histogram and GLCM textural analysis. It is apparent that both LBP and GLCM feature descriptors are different for clean and dust-accumulated solar panels.

The extracted features were fed to implement a binary classification model using linear support vector machine. This model aims to classify solar panels either to clean or dust-accumulated condition. Four experiments were implemented in the classification stage. The first experiment was implemented based on all Uniform and Non-uniform patterns of LBP. The second experiment was implemented by concatenating the ten LBP patterns and the eight GLCM textural features. The third and fourth experiments were implemented by adopting Recursive Feature Elimination (RFE) with number of features equals 15 and 12, respectively [31]. RFE selects features by recursively evaluating progressively smaller sets of features. The least significant features are eliminated from the current set.

1a.png

Figure 7. Examples of feature description results of solar panel images for both dust-accumulated and clean surfaces

1.png

Figure 8. Confusion matrices of the four solar panel inspection experiments with: (a) All LBP features; (b) All LBP and GLCM features; (c) 15 RFE features; (d) 12 RFE features

Table 2. Performane metrics of the dust detection approach cross the four experiments

Figure 8 shows the confusion matrices of the classification results on the 53 test images over the four experiments. Table 2 shows Precision, Recall, accuracy and F1 score for each experiment. It is clear that experiment 3 has scored the highest precision and accuracy with 91.7 % and 94.3 %, respectively. Experiment 2 shows better precision, accuracy and F1 score over experiment 1. This improvement demonstrates the positive effect of GLCM features. In experiment 3, RFE has removed f2, f6 and f7 features and the model outperformed all other models with F1 score equals 0.94. This implies that these features have no effect in discrimination between clean and dust-accumulated solar panels. Additionally, RFE has been deployed to remove the six least effect features which were f2, f5, f6, f7, U2, and U6. It is apparent that f5, U2 and U6 features have a good influence on the classification since precision, recall, accuracy and F1 score values have declined in comparison with their values in experiment 3.

The proposed approach has the ability in detecting dust-accumulated solar panels accurately cross the four experiments. The number of Fn (the number of incorrectly predicted clean solar panels) was one in the first three experiments and two in the last experiment. The number of Fp (the number of incorrectly predicted dust-accumulated solar panels) was acceptable in both experiments 2 and 3. This means that the proposed approach is more accurate in detecting duct-accumulated panels than clean panels.