An Imbalance-Aware Multi-Model Machine Learning Framework for Smartphone-Based Road Surface Classification via Inertial Sensor Fusion

Classification of road surfaces has been widely researched owing to the potential applications for safety, optimal design of transportation infrastructure, and development of intelligent transportation systems. There are several studies that have successfully utilized smartphone sensors and ML algorithms in classifying road surfaces. This literature review summarizes the main approaches, data used, results, and limitations of recent papers dedicated to this topic.

Horizontal positioning of the accelerometer and gyroscope sensors found in smartphones proved highly efficient in the task of classifying road surfaces [14]. The authors of this paper recorded their data at a frequency of 100 Hz, gathering timestamps, three axes of acceleration, gyroscope readings, car speed, and GPS coordinates. These measures allowed attaining a 94% accuracy rate when determining the levels of road roughness. Ecological variables such as light or weather were sources of noise that interfered with the successful implementation of the algorithm. Accelerometer and gyroscope integration into ATVs allows pothole detection and International Roughness Index calculation [14] with an accuracy rate of 90.5% and an error rate of 8.41%, showcasing the utility of sensor-based methods across diverse vehicular conditions. Road condition evaluation using accelerometers and gyroscopes with the International Roughness Index metric was explored by Douangphachanh and Oneyama [15]. The use of AndroSensor in smartphones in different cars in areas around the gearshift led to R² coefficients ranging from 0.721 to 0.869. However, locating devices near the gearshift led to increased levels of engine noise, which may lead to inaccurate data; conversely, the dashboard was a better position to get more vibration data related to the road.

In the study by Ghafoor [16], pothole detection based on data from accelerometers and ultrasonic sensors resulted in impressive performance using Extreme Gradient Boosting (XGBoost); the model achieved an accuracy of 94.56%, recall of 97.41%, precision of 96.40%, and F1-score of 96.90%. In Bengaluru, India, Nericell [17] detected various forms of road surface damage using information from accelerometers in three types of vehicles. Although the number of false negatives decreased to 29% when well-oriented sensors were used, positioning variations continued to pose problems since the false negatives were high above 25 km/h. In the study by Afenika et al. [18], the classification of road surfaces into smooth and damaged conditions based on accelerometer data showed promising results. The authors’ dataset included 300 data sets, 200 sets for smooth roads and 100 for the damaged ones, and was analyzed through the SVM approach, resulting in 93% accuracy and demonstrating the high efficiency of the approach in road condition differentiation.

Classification of the road surface irregularities, such as potholes and bumps, was performed using the K-Nearest Neighbors (KNN) approach in combination with accelerometer sensors, which resulted in 96.03% pothole accuracy and 94.12% bump accuracy [19]. A related research study collected accelerometer and GPS data from an Android application, with a sampling frequency of 10 Hz on the roads of Chandigarh, India. Detection of potholes and bumps based on a Dynamic Time Warping (DTW) approach led to 88.66% accuracy for potholes and 88.89% for bumps; however, high computational costs prevented the authors from scaling up the test set [20]. The same DTW algorithm was implemented by other researchers to detect road markings, cracks, and potholes with the use of a smartphone gyroscope and frequency at 30 Hz [21].

The work [11] detected road conditions in Malaysia using an Android app, including acceleration signals from the 3-axis accelerometer. Their models achieved over 99% accuracy in classifying various road types, highlighting the potential of accelerometer-based classification. The study [22] examined how crowdsourced data from accelerometers and gyroscopes can be used for monitoring the condition of cycleways through GyroTracker software. They classified road segments into true bad, true okay, and true good, achieving a classification rate of about 90%. Another study investigated the use of deep learning algorithms for road surface detection and classification, such as detecting potholes and speed bumps [23]. Reservoir computing, Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM) achieved excellent results with accuracies up to 98%. However, the real-time implementation faced challenges related to computational requirements and processing speed. A crowdsensing-driven platform that utilizes smartphone GPS and accelerometer data was presented in Texas, USA, by the study [24] for detecting potholes and bumps. The system recorded geo-referenced Z-axis accelerations to evaluate road quality in poor, bad, and good categories. Basavaraju et al. [25] employed multi-class supervised learning approaches to classify road structures using smartphone data. Methods such as SVM, Decision Tree (DT), and Multilayer Perceptron (MLP) models were used to classify smooth roads, cracks, and potholes, with MLP achieving the highest average test accuracy of 91.49%. Limited training data was a key factor affecting accuracy. In another study, researchers focused on analyzing vehicle vibrations to detect humps and potholes, where prediction errors increased with increasing speeds up to 34.8% at 15 km/h and declined to 1.6% at 20 km/h. This study considered only accelerometer data but ignored the extra information that could be obtained from gyroscopes [26].

The study [27] suggested the utilization of accelerometer data and video statistics to detect road surface irregularities. Several approaches, such as thresholding, SVM, RF, LSTM, and joint optimization, have been investigated. Under the condition where the smartphone is fixed within the holder, LSTM and joint optimization resulted in accuracies of 0.82 and 0.92, respectively. For the case where the smartphone was placed within the car door compartment, LSTM and joint optimization yielded accuracies of 0.87 and 0.88, respectively. Table 1 summarizes the methodologies, dataset sizes, accuracies, key features, and limitations of recent studies in road surface classification, highlighting the strengths and gaps the proposed approach addresses.

Research on road anomaly monitoring has evolved from simple accelerometer peak detection and GPS-correlated thresholds to supervised ML and deep learning approaches, though early methods struggled with multiclass discrimination under varying conditions [1]. Subsequent supervised learning studies have improved multiclass separability using engineered temporal, statistical, and spectral features with models such as SVMs, RFs, and boosting algorithms, but class imbalance, where smooth-road samples vastly outnumber potholes and speed bumps, remains a persistent challenge that biases models toward majority classes despite high reported accuracy. Across these previous studies, several recurring limitations can be identified: dependence on a single inertial sensor instead of integrating data from multiple sensors, assessment conducted on insufficiently large datasets to reveal behaviors of minority classes, lack of strategies for managing imbalances with consideration for leakage, and minimal exploration of how the developed model would be implemented on the device as opposed to being evaluated in an offline context. This study also builds on our own progressive line of work on smartphone-based road surface classification. Earlier studies established a baseline for road surface classification using smartphone accelerometer and gyroscope data. Class imbalance was subsequently addressed through Synthetic Minority Over-sampling Technique (SMOTE)-augmented KNN classification, followed by an Adaptive Synthetic Sampling (ADASYN)-driven XGBoost framework for pothole detection [6].

Table 1. Comparison of prior road surface classification studies

Note: SVM = Support Vector Machine, XGBoost = Extreme Gradient Boosting, CNN = Convolutional Neural Network, LSTM = Long Short-Term Memory, ML = machine learning, DT = Decision Tree, MLP = Multilayer Perceptron, DTW = Dynamic Time Warping, KNN = K-Nearest Neighbors.

This study addresses these gaps through a comprehensive pipeline featuring six-channel inertial sensing (accelerometer + gyroscope feature-level fusion), leakage-free post-split SMOTEENN resampling, and comparative evaluation of heterogeneous ensembles (AdaBoost, SVM-RBF, RF, Stacking), while emphasizing practical mobile app deployment for real-world sensing, annotation, and inference, bridging the critical gap between offline accuracy and end-user deployment scenarios, and complementing the dual-filter deployment-focused study [28] with a broader comparative evaluation of classical imbalance-aware ensemble methods. This work also complies with the United Nations Sustainable Development Goals 9 and 11 [6, 29].

The accelerometers measure the rate at which a moving vehicle accelerates or decelerates in all three axes, giving us insights into peculiarities of the road surface. With a study of the frequency and intensity of vibrations, even minute variations such as bumps and other imperceptible characteristics can be observed. For instance, Figure 4(a) shows that when a vehicle is moving on a normal road, its values in the three axes remain stable. Once the car goes over a speed bump, the data produces a distinctive peak due to the rise of the car over the bump, then another peak indicates the compression of the suspension and the descent. After that, another peak illustrates the recoil of the suspension and the recovery of the car once it passes the speed bump. Conversely, Figure 4(b) represents the readings collected while the car passes through rumble strips. In contrast to speed bumps, rumble strips indicate a pattern of recurring crests and troughs on the sensor readings, where each crest or trough is related to the deflection of each rumble strip.

tu_pian_52.png

(a)

tu_pian_53.png

(b)

Figure 4. Illustration of accelerometer graphs depicting (a) speed bumps and (b) rumble strips

The recurrent waveform observed corresponds with the recurring nature of the arrangement of the rumble strips that are generally installed in close proximity to each other. The experiments covered a 500-meter section while maintaining an average speed of 30 kmph. The experimental site was selected to gather data encompassing all four class labels. The experiment was replicated over 100 instances on Rajpur Road, Dehradun, Uttarakhand (India), leading to a sizable dataset.

Ground-truth labels were assigned to each recording session during post-processing through a combination of observations made by on-road annotators, GPS-correlated event records, and visual analysis of the accelerometer waveforms. Events were identified using GPS waypoints on the route map for each session. The front-seated annotator pressed a button in the PRISM app at both the beginning and end of every event. Synchronization of GPS timestamps was maintained within approximately ±0.5 seconds around each marked event. Additionally, as part of quality control, all windows drawn from minority classes were re-evaluated post hoc for a visible impulse response in the vertical acceleration channel; any windows that did not display a distinct impulse were relabeled as Smooth Road.

In this work, we implement the data-processing pipeline summarized in Figure 5, in which raw tri-axial accelerometer and gyroscope data are first segmented using a sliding-window approach and then transformed via time- and frequency-domain feature extraction. Building on this pipeline, the inertial signals are fused into magnitude-based, orientation-invariant representations, followed by train–test splitting, standardization, class-imbalance handling, and principal component analysis (PCA)-based dimensionality reduction, after which multiple machine-learning models (AdaBoost, SVM with RBF kernel, RF, and a stacking ensemble) are trained to classify road-surface conditions as smooth, speed bump, rumble strip, or pothole, as illustrated in the diagram.

tu_pian_55.png

Figure 5. Preprocessing and feature-engineering pipeline

4.1 Dataset and label definition

The dataset contains six motion channels and one label column. The sensor inputs are Acc-X, Acc-Y, Acc-Z, Gyro-X, Gyro-Y, and Gyro-Z, and the target label is Road Condition. The encoded classes are: 0 = Smooth, 1 = Speed Bump, 2 = Rumble Strip, and 3 = Pothole. The raw dataset contains 76,466 rows. As presented in Figure 6, the class distribution is strongly imbalanced: 71,983 smooth samples, 1,273 speed bump samples, 2,527 rumble strip samples, and 683 pothole samples. This motivates the use of imbalance-aware training and macro-sensitive evaluation.

tu_pian_56.png

Figure 6. Raw class distribution of the dataset

4.2 Sliding-window segmentation

The continuous sensor stream is transformed into overlapping windows using a fixed-size sliding window mechanism. Let the multichannel sample stream be denoted by

$X=\left\{ {{x}_{1}},{{x}_{2}},\ldots ,{{x}_{N}} \right\},{{x}_{t}}\in {{R}^{6}}$

where, each sample contains three accelerometer and three gyroscope values. For a window size of W and a stride S, the k-th window is defined as

${{X}^{\left( k \right)}}=\left\{ {{x}_{kS+1}},{{x}_{kS+2}},\ldots ,{{x}_{kS+W}} \right\}$          (1)

In this study, W = 64 and S = 16; a majority-purity criterion is applied before assigning a label to a window. If the most frequent class in the window appears with count n_max, then the purity is computed as

${{P}^{\left( k \right)}}=\frac{{{n}_{\text{max}}}}{W}$           (2)

A window is retained only if P^(k) ≥ 0.60. The assigned label is the majority class of that window. In Figure 7, the windowed class distribution of the dataset is presented.

tu_pian_57.png

Figure 7. Windowed class distribution of the dataset

4.3 Feature extraction

For each channel in every retained window, the pipeline extracts time-domain and frequency-domain descriptors. If the samples in a window for one channel are z₁, z₂, …, z_W, the mean and standard deviation are computed as

$\mu =\frac{1}{W}\underset{i=1}{\overset{W}{\mathop \sum }}\,{{z}_{i}}$          (3)

$\sigma =\sqrt{\frac{1}{W}\underset{i=1}{\overset{W}{\mathop \sum }}\,{{({{z}_{i}}-\mu )}^{2}}}$         (4)

The root mean square (RMS) is given by

$\text{RMS}=\sqrt{\frac{1}{W}\underset{i=1}{\overset{W}{\mathop \sum }}\,z_{i}^{2}}$          (5)

The signal energy is computed as

$E=\underset{i=1}{\overset{W}{\mathop \sum }}\,z_{i}^{2}$            (6)

The mean absolute successive difference (MASD) and mean squared successive difference (MSSD) are defined as

$\text{MASD}=\frac{1}{W-1}\underset{i=1}{\overset{W-1}{\mathop \sum }}\,\left| {{z}_{i+1}}-{{z}_{i}} \right|$            (7)

$\text{MSSD}=\frac{1}{W-1}\underset{i=1}{\overset{W-1}{\mathop \sum }}\,{{({{z}_{i+1}}-{{z}_{i}})}^{2}}$          (8)

Frequency-domain features are computed using Welch spectral estimation. Let $P\left( f \right)$ denote the resulting power spectral density. The dominant frequency is

${{f}_{\text{dom}}}=\text{arg}\underset{f}{\mathop{\text{max}}}\,P\left( f \right)$           (9)

The spectral energy is

${{E}_{f}}=\underset{f}{\mathop \sum }\,P\left( f \right)$           (10)

and the spectral entropy is computed from the normalized spectrum, ${{p}_{f}}$, where

${{p}_{f}}=\frac{P\left( f \right)}{\mathop{\sum }_{f}P\left( f \right)+\varepsilon }$           (11)

${{H}_{f}}=-\underset{f}{\mathop \sum }\,{{p}_{f}}\text{lo}{{\text{g}}_{2}}\left( {{p}_{f}}+\varepsilon  \right)$             (12)

where, $\varepsilon $ is a small constant for numerical stability. Figure 8 represents the boxplot distribution of the first 10 extracted Acc-X features across all windows.

tu_pian_58.png

Figure 8. Boxplot distribution of the first 10 extracted Acc-X features across all windows

4.4 Inertial fusion through magnitude features

In addition to per-axis features, the accelerometer and gyroscope magnitude signals are computed to provide orientation-robust summaries of translational and rotational intensity. For each timestamp, the accelerometer magnitude is

${{M}_{acc}}=\sqrt{Acc_{x}^{2}+Acc_{y}^{2}+Acc_{z}^{2}}$            (13)

and the gyroscope magnitude is

${{M}_{gyro}}=\sqrt{Gyro_{x}^{2}+Gyro_{y}^{2}+Gyro_{z}^{2}}$          (14)

Statistical descriptors are then extracted from these magnitude signals in the same way as for the original channels. This constitutes feature-level sensor fusion of translational and rotational motion information.

4.5 Train-test split and standardization

To avoid leakage, the window-level feature matrix is split into training and testing subsets prior to imbalance treatment. Let the feature vector for sample $i$ be ${{\mathbf{x}}_{i}}$. Standardization is then applied to each feature dimension using the training-set mean ${{\mu }_{j}}$ and standard deviation ${{\sigma }_{j}}$:

$\hat{x}_{i j}=\frac{x_{i j}-\mu_j}{\sigma_j}$          (15)

The same transformation is then applied to the test data using the training-derived statistics.

4.6 Imbalance handling with Synthetic Minority Over-sampling Technique with Edited Nearest Neighbors

Class imbalance is addressed only in the training partition. The primary method used in the implemented pipeline is SMOTEENN, which combines synthetic minority oversampling with Edited Nearest Neighbors cleaning. SMOTE creates synthetic minority samples by interpolation between neighboring minority instances. Given a minority instance ${{\mathbf{x}}_{i}}$ and one of its nearest minority neighbors ${{\mathbf{x}}_{nn}}$, a synthetic sample is generated as

${{\mathbf{x}}_{new}}={{\mathbf{x}}_{i}}+\lambda \left( {{\mathbf{x}}_{nn}}-{{\mathbf{x}}_{i}} \right),\text{ }\!\!~\!\!\text{ }\lambda \in \left[ 0,1 \right]$            (16)

tu_pian_59.png

Figure 9. Class distribution after Synthetic Minority Over-sampling Technique with Edited Nearest Neighbors (SMOTEENN)-based balancing

ENN then removes samples that disagree with the dominant class among their nearest neighbors, thereby reducing local noise and boundary ambiguity.

SMOTEENN was selected based on recent comparative studies demonstrating its superiority over standalone SMOTE or ENN in handling imbalanced classification tasks [30]. The combined approach addresses both the data-scarcity problem via SMOTE oversampling and the boundary-region noise problem via ENN cleaning, resulting in more robust decision boundaries [31]. For multiclass imbalanced scenarios similar to our 4-class road classification task, hybrid resampling methods have consistently outperformed single-strategy approaches, with SMOTEENN showing particularly strong performance when the minority class includes both central-region and boundary-region samples [31]. Figure 9 demonstrates the class distribution after SMOTEENN-based balancing.

This cautious approach is especially significant for the Pothole and Speed Bump categories within this dataset. In these cases, a straightforward application of SMOTE often generates samples that lie on either side of the decision boundary associated with Rumble Strip. This occurs because impulse-like transients from all three types of anomalies occupy similar areas in the feature space. The Edited Nearest Neighbors component of SMOTEENN effectively eliminates these unclear synthetic and original samples, which is the reason it was favored over simple oversampling methods for this particular task.

4.7 Dimensionality reduction

After standardization and resampling, PCA is applied to preserve 95% of the variance while reducing redundancy [32]. The cumulative explained variance curve is presented in Figure 10, where the retained number of principal components is determined based on the 95% variance threshold.

If the covariance matrix of the standardized feature space is decomposed into eigenpairs, the retained components are those satisfying

$\frac{\mathop{\sum }_{i=1}^{m}{{\lambda }_{i}}}{\mathop{\sum }_{i=1}^{d}{{\lambda }_{i}}}\ge 0.95$         (17)

where, $d$ is the total number of original features and $m$ is the number of retained principal components.

tu_pian_60.png

Figure 10. Principal component analysis (PCA) cumulative explained variance curve

4.8 Classification models

Four classifiers are evaluated in the same processed feature space: AdaBoost, SVM with radial basis function kernel, RF, stacking ensemble with AdaBoost, SVM, and RF as base learners, and logistic regression as the meta-learner. The stacking prediction can be conceptually expressed as

$\hat{y}=g\left(h_1(\mathbf{x}), h_2(\mathbf{x}), h_3(\mathbf{x})\right)$         (18)

where, ${{h}_{1}}$, ${{h}_{2}}$, and ${{h}_{3}}$ are the base learners, and $g$ is the logistic-regression meta-model.

Four types of classifiers were used in order to cover different ML approaches. AdaBoost is based on the concept of boosting, while SVM, using the RBF kernel, represents the method of margin maximization in the feature space. The next RF classifier corresponds to bagging with random feature selection, and finally, Stacking combines various and diverse base learners into an ensemble through a process of meta-learning. Recently, there has been increasing interest in ensemble methods such as stacking, in which combinations of base learners with mutually reinforcing strengths can work very effectively [33]. In particular, experiments across multiple domains revealed that the stacking ensemble approach outperforms its base learners individually, and RF provides the benchmark solution due to its robustness to hyperparameters and the fact that it does not overfit [13, 34]. As the meta-classifier in Stacking, logistic regression was employed owing to its simplicity and fast performance, which makes it preferable compared to complex meta-classifiers [33]. Table 3 demonstrates the key hyperparameters of the four evaluated classifiers

Table 3. Key hyperparameters of the four evaluated classifiers

Note: SVM = Support Vector Machine, RF = Random Forest, RBF = Radial Basis Function.

All models were trained using fixed random seeds to guarantee reproducibility. No further hyperparameter tuning was conducted apart from the previously mentioned configurations, which were determined through initial experiments on the validation set.

The entire process comprising feature extraction, resampling, model training, and evaluation was implemented in Python 3 with the help of scikit-learn and imbalanced-learn libraries, all executed on Google Colab's standard free-tier CPU environment. GPU acceleration was not utilized since none of the four classifiers (AdaBoost, SVM-RBF, RF, and the stacking ensemble) required it.

4.9 Evaluation metrics

Three principal metrics are used: accuracy, macro-F1, and weighted-F1. If $T{{P}_{c}}$, $F{{P}_{c}}$, and $F{{N}_{c}}$ denote the class-wise true positives, false positives, and false negatives for class $c$, then precision and recall are defined as

$\text{Precisio}{{\text{n}}_{c}}=\frac{T{{P}_{c}}}{T{{P}_{c}}+F{{P}_{c}}}$          (19)

$\text{Recal}{{\text{l}}_{c}}=\frac{T{{P}_{c}}}{T{{P}_{c}}+F{{N}_{c}}}$            (20)

The class-wise F1-score is

$F{{1}_{c}}=\frac{2\cdot \text{Precisio}{{\text{n}}_{c}}\cdot \text{Recal}{{\text{l}}_{c}}}{\text{Precisio}{{\text{n}}_{c}}+\text{Recal}{{\text{l}}_{c}}}$            (21)

Macro-F1 averages the class-wise F1-scores equally:

$\text{Macro-F}1=\frac{1}{C}\underset{c=1}{\overset{C}{\mathop \sum }}\,F{{1}_{c}}$          (22)

Weighted-F1 uses class support ${{n}_{c}}$:

$\text{Weighted-F}1=\frac{1}{\mathop{\sum }_{c=1}^{C}{{n}_{c}}}\underset{c=1}{\overset{C}{\mathop \sum }}\,{{n}_{c}}F{{1}_{c}}$        (23)

Accuracy is computed as

Accuracy $=\frac{\text { Number of correct predictions }}{\text { Total number of predictions }}$           (24)

PRISM is a complete Android application based on the Flutter platform for road surface classification on a mobile phone. This does not involve the use of any extra equipment, nor does it utilize any processing performed in the cloud, and the application runs solely on the device, storing information from sensors, performing classification, alerting the driver of the anomaly, and transmitting logs to local/remote servers via the Internet. Our application uses a trained model integrated directly into the ONNX format, which will perform classification tasks on the smartphone itself. The Internet connection is not needed for anomaly detection in our case, and no private information will be sent to the server with regard to the sensors' data. The application constantly reads the accelerometer and gyroscope sensors' data at 100 Hz while moving and feeds the information to the model.

The current application has only two filters used to ensure that the alerts raised are relevant and not distracting. For one, the use of a debouncing method ensures that the classifier must predict the same anomaly class three times within half a second. In addition, the speed detection filter ensures that any alert raised below five kilometers per hour and above eighty kilometers per hour does not occur if the model operates outside its speed calibration range. The combination of these filters ensures the reliability of our alert system. Figure 17 shows the interface of the application with an alert for rumble strips.

In our application, once the anomaly occurs, the vehicle's GPS coordinates and timestamps automatically tag it. Then, the collected information gets saved to a CSV file in the device's memory. Finally, after connecting the device to the Internet, uploading the session files to Google Drive becomes effortless. Therefore, such an application ensures that users collect a significant amount of anomaly data without even realizing it. Hence, the data gathered is readily available to analyze road conditions. The application consists of ten modules, including sensor acquisition, inference, alerting, logging, cloud synchronization, map visualization, and many others.

tu_pian_67.png

Figure 17. PRISM application map screen with anomaly alert