Employing Data Mining Techniques for Engineering Soil Classification: A Unified Soil Classification System Approach

Soil classification systems serve as a common language among engineers, enabling them to categorize soils based on defining characteristics. Such classification is performed through a systematic approach, focusing on specific properties and usage criteria as stipulated by the American Society for Testing and Materials (ASTM). The physical and mechanical properties of the soil dictate its suitability for use as structural fill or foundation soil [1]. A comprehensive understanding of soil classification thus furnishes professionals with insights into its predicted behavior [2].

In the realm of civil engineering, the Unified Soil Classification System (USCS) and the American Association of State Highway and Transportation Officials (AASHTO) approach are commonly employed. Both methodologies hinge on granulometry and Atterberg limits, fundamental elements in soil characterization. Granulometry provides an understanding of the particle size distribution within the soil, thereby identifying the relative proportions of sand, silt, and clay. Conversely, Atterberg limits signify three specific moisture content points at which the soil's behavior alters: the liquid limit, the plastic limit, and the shrinkage limit. These limits inform about the soil's plasticity and water retention capacity [3].

The USCS classification process commences with soil sampling from the study area, after which the samples undergo laboratory testing. Granulometric analysis separates the sample into distinct sizes (sand, silt, and clay), determining the percentage of each fraction. Concurrently, Atterberg limit tests ascertain the liquid, plastic, and shrinkage limits, offering insights into soil plasticity and its response to water content variations. The outcomes of these tests feed into a flow diagram, with soils coded by symbols and letters indicating their characteristics.

Given its pivotal role in the design and execution of infrastructure projects, accurate soil classification holds immense value in civil and geotechnical engineering. Consequently, advancements in classification systems utilizing novel methodologies can significantly enhance the precision and efficiency of geotechnical analysis and design.

In light of this, the current research proposes a novel classification method employing data mining techniques to analyze and extract patterns and relationships within soil property data. This approach enables soil classification in line with the criteria set by the USCS system. Data mining, a technique adept at uncovering patterns and relationships within large datasets, is particularly beneficial when analyzing and categorizing numerous soil samples with various characteristics. The applied data mining algorithms, namely support vector machine, random forest, k-nearest neighbors, and decision trees, identify intricate relationships between the physical and mechanical properties of soils.

The study utilizes artificial intelligence (AI) techniques centered on data mining. It applies soil classification algorithms for engineering purposes to enhance and automate the soil classification process by integrating data mining with the USCS. This approach expedites the determination of soil categories based on their properties, offering a more efficient method. Such advancements bear significant practical implications for civil and geotechnical engineering, where accurate soil classification is essential for effective decision-making in construction and infrastructure projects. Additionally, this methodology may reveal patterns and relationships that could potentially remain undetected in manual analysis, thereby enriching our understanding of soil characteristics and behavior across varying conditions and locations.

3.1 Soil classification

Soil classification was performed according to ASTM standards to determine physical and mechanical properties. A representative sample of the soil was taken and air-dried to eliminate surface water content. Then the granulometric test (ASTM D 422) was performed using sieves of different sizes, weighing the fractions retained in each sieve. The results were expressed as cumulative percentages of retained weight as a function of particle size. The Atterberg liquid and plastic limit test (ASTM D4318) was then performed. For the liquid limit, a sample is taken that passes the 40 mesh sieve, and the soil is kneaded with water until a homogeneous paste is obtained, using the Casagrande cup, the sample is molded in the Cascador with the spatula and a groove of 2 mm is made along the sample, then it is rotated and the number of blows is noted until it closes at 13 mm, three repetitions are made for a different number of blows, at the end three samples are taken to determine the moisture content (ASTM D 2216). The plastic limit is carried out by taking a wet soil sample in the form of an ellipsoid and then rolling it with the fingers of the hand on a smooth surface with the pressure strictly necessary to form cylinders. If the cylinder has not crumbled before it reaches a diameter of about 3.2 mm (1/8"), it is turned back into an ellipsoid, and the process is repeated as many times as necessary until it crumbles to about that diameter. The portion obtained is placed on watch glasses until about 6g of soil is collected to determine its moisture content (ASTM D 2216).

3.2 Algorithms

Data mining is a set of tools designed for rapid, automated, and exploratory data analysis. These tools include decision trees, rule induction, inductive logic programming, k-nearest neighbors, clustering algorithms, neural networks, genetic algorithms, and Bayesian networks, among others [4].

The algorithms used in the research are k-nearest neighbors, support vector machine, decision trees, and random forest.

K-nearest neighbors (KNN) is a nonparametric classifier that predicts the label of an entry by identifying its k nearest neighbors based on specified distance metrics, such as Euclidean distance or cosine similarity. It then determines the majority vote of the neighboring labels to assign the entry to a particular class [10]. It is often used for its simplicity and ability to obtain faster results, unlike other methods by storing a set of prototypes representing the knowledge of the problem [5]. Based on the idea that patterns closer to a target pattern x', provide more information about the label. KNN assigns the class label of most patterns closest to K the data space. This metric is used to calculate the distance or similarity between data points. Choosing an appropriate distance metric is critical because it determines how close or similar two data points are in the feature space. In this R^q, it is reasonable to employ the Minkowski metric (p-norm) corresponding to the Euclidean distance p=2. In other data spaces, we have to choose suitable distance functions, e.g., the Hamming distance in B^q. In the case of binary classification, the label set y={1, -1} is used, and the KNN is defined as with neighborhood size K and with the index set $N_K\left(x^{\prime}\right)$ of the K nearest patterns see Eq. (1) and Eq. (2). It is recommended to employ a value K=5 in the models because it produces the best performance with a low standard deviation between K=1, 2, ..., 20 [11].

$\left\|x^{\prime}-x_i\right\|^p=\left(\sum_{i=1}^q\left|\left(x_i\right)^{\prime}-\left(x_i\right)_i\right|^p\right)^{1 / p}$                (1)

$f_{K N N}\left(X^{\prime}\right)=\left\{\begin{array}{l}1 \text { if } \sum i \in N_K\left(x^{\prime}\right)^{y i} \geq 0 \\ -1 \text { if } \sum i \in N_K\left(x^{\prime}\right)^{y i}<0\end{array}\right.$             (2)

Support vector machine (SVM) is a method used in various fields such as object classification, face recognition, and text categorization. SVM aims to find an optimal hyperplane that best separates data points of different classes in the feature space [12].

The SVM training procedure for pattern recognition involves solving a quadratic optimization problem. The main goal of SVM is to find the optimal hyperplane that maximizes the distance between data points of different classes $\left\{\left(x_1, y_1\right), \ldots,\left(x_i, y_i\right)\right\}$, where $x \in R^N, \mathrm{y} \in\{-1,1\}$. Kernel mapping is a technique used to identify training data from the input space to a higher dimensional feature space where the mapped training data can become linearly separable. This is particularly useful when working with data that is not linearly separable in the original feature space.

Maximize:

$W(\alpha)=\sum_{i=1}^l \alpha_i-\frac{1}{2} \sum_{i=1}^l \sum_{i=1}^l \alpha_i y_i \alpha_j y_j K\left(X_i, X_j\right)$              (3)

subject to $\sum_{i=1}^l \alpha_j y_j=0$.

The decision function becomes:

$f(x)=\operatorname{sng}\left(\sum_{i=1}^l \alpha_i y_j K\left(X_i, X\right)+b\right)$                (4)

Decision trees are classification and prediction techniques that can handle both numerical and nominal data. They are constructed in two stages: Induction and Pruning. During the induction phase, the tree is built by recursively partitioning the data according to features and selecting the best partitioning criterion at each node. On the other hand, in the pruning phase, excessive complexity is removed from the tree to improve its generalizability. The most commonly used decision tree algorithms are C4.5, ID3, CART, and J48 [13].

Random forest is a robust learning algorithm that uses multiple randomized decision trees and combines their predictions by averaging. Its operation consists of two phases: construction, where multiple decision trees are built using different training instances to ensure an accurate and less overfit model, and prediction, which is determined by averaging the results of each tree. Performance is evaluated based on several criteria, variations in classifier parameter values, sensitivity to noise, and changes in training size. To measure efficiency, they are compared to classification trees by analyzing the performance relative to classification trees [14].

3.3 Data matrix

Soil classification information according to SUCS was collected using a data collection sheet, from different laboratories certified by the National Quality Institute (INACAL) and certified soil mechanics files. A total of 474 soil classifications were obtained from 2018 to 2022, forming a data matrix of 474 instances and 9 variables. The variables collected were: gravel (GV), sand (AR), fines (FN), liquid limit (LL), plastic limit (LP), plasticity index (PI), maximum dry density (MDS), optimum moisture content (OCH), and soil type extracted from granulometry (ASTM D 422), Atterberg Limits (ASTM D4318), and Proctor (ASTM D1557) tests, see Figure 1 and Tables 1-2. For repeatability, each test was repeated three times until the results obtained varied within ±0.5% in the laboratory [15].

1.png

Figure 1. Descriptive statistics of the variables

Table 1. Description of variables collected

Table 2. Statistical data from the database

2.png

Figure 2. Soil types contained in the database

3.png

Figure 3. Types of distribution of each variable

The soil classification present in the data matrix contains 11 soil types according to SUCS: silty sand (SM), clay (CL), clayey gravel (GC), clayey sand (SC), gravel graded with silt (GW-GM), silt (ML), silty gravel (GM), well-graded gravel (GW), poorly graded gravel with silt (GP-GM), clayey sand with silt (SC-SM), and silty gravel with clay (GM-GC), see Figure 2.

3.4 Methodology for employing data mining

The knowledge discovery database (KDD) methodology was used, which is known for its iterative nature, where certain phases can lead to revisiting previous steps. In many cases, multiple iterations are required to efficiently extract high-quality, interactive knowledge. The involvement of subject matter experts is crucial throughout the process, as they contribute to data preparation and validate the extracted knowledge [16].

The KDD methodology uses a set of techniques, collectively referred to as data mining, to discover trends in large amounts of data [17]. The term KDD was coined in 1989 to convey the idea that knowledge is the ultimate result of data-driven discovery. The KDD process consists of extracting patterns, in the form of rules or functions, from data for analysis by the user [18].

The phases involved in the KDD process are:

3.4.1 Selection

Identified the need to classify soils for engineering purposes through data mining.

A data set was created, on which the discovery process was carried out. For the selection of the attributes, the distribution of each variable was analyzed, see Figure 3.

Then the Sklearn library was used in the Jupyter Lab interface and the “RandomForest” algorithm was run, which evaluates the value of the variables by measuring the gain concerning the output variable (soil type) using the command “model.feature_importances_”. The variables with a Ranked value greater than 8.0 were selected.

3.4.2 Preprocessing and cleaning

During data collection, it is common for the obtained set to contain null or anomalous instances, which can cause noise in the knowledge extraction process. In this phase, data cleaning was applied to improve the quality of the data using the Python programming language with the Jupyter Lab interface and the Numpy, pandas, and seaborn libraries, which use statistical analysis as a whisker box to clean and remove anomalous data. Missing data were imputed using three different methods: average, median, and regression algorithms. In addition, duplicate data were removed using the Set command in Python. To deal with anomalous data, the Panda’s library was used, and the boxplot was used, which allows visualization of the distribution and detection of outliers by identifying points that fall outside the boundaries of the graph (see Figure 4). In addition, the library assigned visual attributes to the data, calculated statistical transformations, and decorated the graph with informative labels on the axes [19].

4.png

Figure 4. Database without outliers

3.4.3 Transformation and reduction

Useful characteristics were sought to represent the data about soil type, using transformation methods to reduce the effective number of variables and invariant representations of the data (see Table 3); the variables were grouped according to the type of distribution in Figure 3.

3.4.4 Data mining

We used the Python programming language with the Jupyter Lab interface and the Scikit-learn 1.1.2 package developed for machine learning with Python. Scikit-learn offers a wide range of machine learning algorithms, including supervised and unsupervised techniques, all presented through a consistent, task-oriented interface. This design facilitates a direct comparison of methods for a specific application, ensuring ease of use and flexibility in the machine-learning process [20]. The prediction algorithms used for training and pattern extraction were k-nearest neighbors, support vector machine, decision trees, and random forest, which consist of three parts: knowledge representation, evaluation, and search, maintaining minimal bias without affecting variance; adapting to data types.

To build the models, the data were divided into training (70%) and test (30%) sets, and then the hyperparameters were configured as shown in the following Table 4.

Table 3. Grouping of variables according to their importance

Table 4. Hyperparameters of the algorithms

Table 5. Confusion matrix

3.4.5 Intervention and evaluation of data

After training the five groups PL_1, PL_2, PL_3, PL_4, and PL_5 with the collected database and the proposed data mining models, the predicted results were compared with the actual values using performance metrics such as the confusion matrix, also known as the error matrix. This has a special table design that allows visualizing the performance of the algorithm, where each row represents the instances of an actual class, while each column represents the instances of a predicted class. This design allows a clear evaluation of the performance of the algorithm in correctly classifying the data points into different classes. The analysis of the confusion matrix provides insight into the accuracy, precision, recall, and other performance parameters of the data mining models [21]. The distribution of the errors committed by the algorithms is expressed by true positives (VP), false negatives (FN), false positives (FP), and true negatives (VN), see Table 5.

Three main metrics were used to evaluate the performance of each algorithm: Recall, Specificity, and Precision. Recall measures the proportion of positive cases correctly identified, Specificity measures the proportion of negative cases correctly identified, and Precision measures the closeness of measured values to established or known values. By using these measures. These metrics allow a better understanding of the capabilities of the algorithms and facilitate informed decision-making in their practical application [22], see Eqs. (5)-(8).

$Recall(R)=\frac{V P}{V P+F N}$            (5)

$Specificity (E)=\frac{V N}{V N+F P}$        (6)

$Precision (P)=\frac{V P}{V P+F P}$          (7)

$F-measure =2 * \frac{R+P}{R * P}$                  (8)

The various metrics used to evaluate the algorithms were: Recall is the ability of the algorithm to accurately predict the correct results, especially in identifying positive cases. Specificity is the ability of the algorithm to correctly identify negative results, thereby reducing false positives. Precision quantifies the number of correct positive results relative to the total number of positive instances in the population, providing a measure of accuracy in identifying positive instances. The F-measure indicates the overall goodness of fit of the algorithm, combining the precision and recall measures in a balanced manner.

The data matrix consisted of 474 soil classifications and 9 variables collected from 2018 to 2022 from certified laboratories and mechanical studies, based on tests standardized by the Ministry of Transport and Communications (MTC) and the Peruvian Technical Standard (NTP). The variables collected were of a discrete and nominal type, similar to Bui et al. [4], who use discrete variables in their research.

The algorithms chosen to estimate the engineering soil type were k-nearest neighbors, support vector machines, decision trees, and random forests recommended by Al-Shamiri [9]. Al-Shamiri conducted research on pattern extraction from databases using various algorithms, such as decision tree algorithms, k-nearest neighbors, neural networks, naive Bayes, support vector machines, random forests, regression, AIS, SETM, Apriori, FP-Growth, and K-Means and found that these algorithms performed best during the training and validation phase. Like Rao and Chaparala [23], they used a support vector machine to construct decision tables that minimize features extracted from large data sets.

The modeling of soil types for engineering purposes was performed with 5 groups (PL_1, PL_2, PL_3, PL_4, and PL_5) conformed by 4 different variables and trained with four algorithms from the Scikit-learn library in Jupyter Lab with the Python programming language, obtaining a percentage of correct classification of 68. 22%, 71. 03%, 84.11%, 63.51%, and 61.68%, respectively, which exceed by 84.11% the results obtained by Manjula and Narsimha [24], of 75.39% and 75.38% of correctly classified instances, and are in the range of the results obtained by Palomino Ojeda and Rosario Bocanegra [6], Hernández Pereira and Medina González [7] and Cortés Henao [8], guaranteeing the validity of the models.

Promising results have been obtained in soil classification by data mining. However, it is important to consider some limitations. To improve the generalization and robustness of the models, it is necessary to collect other variables that represent all the conditions and types of soils present in the region studied, such as chemical compounds and electrical resistivity, among others. This will allow for obtaining a broad and diverse database. In future research, it is recommended to vary the hyperparameters of the models. Although hyperparameter optimization was performed, the entire search space may not have been exhaustively explored, and advanced optimization techniques, such as Bayesian optimization, could be used to find more optimal hyperparameter combinations and further improve model performance.