AI Ladder-Based Intelligent Enterprise Resource Planning for Predictive Railway Track Maintenance from UAV Imagery

The railway industry is a vital component of Indonesia’s transportation infrastructure, particularly on Java and Sumatra, Investments in both super-and sub-structures, including ballast, slab track, and subgrade, are essential to maintaining operational reliability and long-term capacity of the national rail system [1]. Recent developments, such as the inauguration of Southeast Asia’s first high-speed railway connecting Jakarta and Bandung, underscore Indonesia’s commitment for the adoption of advanced engineering standards for track construction and system performance [2, 3]. Despite these advancements, safety remains a pressing concern. Historical incidents, highlight persistent vulnerabilities in Indonesia’s railway operations. A total of 143 train accidents occurred in Indonesia between 2015 and 2021, causing 132 fatalities and nearly 300 injuries [4], highlighting the continuing instability of railway safety performance. Earlier, around 2004 to 2010, over 700 railway accidents were reported, mostly involving derailments and vehicle collisions. Although train-to-train crashes made up only about 5% of the total, road-related collisions accounted for nearly 20%, resulting in more than 360 deaths and 1,200 injuries [5]. It reflects long-standing structural and operational safety deficiencies in the national rail system.

Similar incidents have also occurred globally, further underscoring the universal nature of railway safety challenges. Studies have shown that fatalities and injuries among track maintenance workers often occur under high-risk working conditions (especially at night) and can be exacerbated by communication failures and insufficient safety coordination [6]. In South Korea, 240 human errors were found on Korean Railway system [7]. In the United Kingdom, Network Rail was fined £3.75 million after two maintenance workers were fatally struck by a train in Margam, Wales; the investigation revealed systemic safety failures and led to significant operational reforms [8].

Railway defects are a critical issue in transportation safety [9], as they can lead to derailments, service disruptions, and costly repairs [10]. Traditional inspection methods are often inefficient and prone to human error [11]. In the modern context, human safety and industrial efficiency are global concerns, as emphasized in the Sustainable Development Goals (SDGs). In particular, SDG 9 (Industry, Innovation and Infrastructure) underlines the importance of resilient infrastructure and technological innovation. In the railway sector, this translates into maintaining advanced, safe, and sustainable transportation systems.

Recent studies have explored visual inspection methods for railway defect detection using image processing and computer vision. Zhuang et al. [12] developed an automated visual inspection system using a cascading classifier ensemble trained with LogitBoost to identify cracks and deformations. Infrared thermography (IRT) has also been applied to detect hidden cracks, although it requires specialized equipment and is sensitive to environmental conditions [13]. Classical techniques such as canny edge detection and the Hough transform face challenges in recognizing complex or novel defect types due to reliance on handcrafted features [14]. Collectively, these studies demonstrate promise but also reveal limitations in generalizability, real-time applicability, and integration with maintenance decision-making systems.

To support intelligent and scalable infrastructure maintenance, this study adopts the AI Ladder Model comprising four stages: Collect, Organize, Analyze, and Infuse as a guiding framework for embedding AI into operational workflows [15]. In this framework, unmanned aerial vehicles (UAVs) provide real-time data acquisition (Collect); feature extraction, annotation, and preprocessing tools prepare and structure image data (Organize); machine-learning (ML) algorithms enable pattern recognition and classification (Analyze); and outputs are translated into actionable maintenance strategies (Infuse). This structure ensures end-to-end alignment between data, analytics, and infrastructure decisions, while supporting SDG 9 through digital innovation. Building on this framework, we evaluate four ML algorithms: Artificial Neural Networks (ANNs), Support Vector Machines (SVMs), Random Forest (RF), and K-Nearest Neighbors (KNNs) for railway defect detection. RF is recognized for handling high-dimensional data and mitigating overfitting [16]; KNN is effective for similarity-based classification [17]; ANN excels at learning nonlinear patterns and extracting complex features [18]; and SVM performs well with limited datasets and high-dimensional feature spaces [19]. We further incorporate a deep-learning-based feature embedding to enhance performance and generalizability across diverse track conditions, with all classification outcomes expressed consistently in defective (bad) /non-defective (good) terms.

Unlike most railway inspection studies that do not frame their approach within a systematic framework, this work explicitly employs the AI Ladder (Collect-Organize-Analyze-Infuse) [15] as an end-to-end backbone: In the Collect phase, we use UAVs for scalable and high-frequency RGB image acquisition; The Organize phase arranges annotation/normalization and extracts deep feature embeddings with standardized outputs of good/bad across track conditions; The Analyze phase conducts a controlled comparison of four models (ANN, SVM, RF, KNN) based on AUC, accuracy, F1-score, and MCC, rather than merely reporting a single accuracy number [16, 19]; and the Infuse phase links predictions to enterprise resource planning (ERP) to trigger work orders, scheduling, and resource prioritization, components that are generally absent in the literature yet crucial to bridge the perception-to-action gap [20]. Thus, our uniqueness lies not only in UAVs, deep embeddings, and multi-model comparison, but especially in the use of the AI Ladder as an operational framework that ensures analytical findings culminate in measurable maintenance impact.

The integration of predictive-maintenance outputs into ERP systems, particularly maintenance modules, is crucial for operational reliability and decision-making in infrastructure-intensive sectors. Prior work shows that combining ML with SAP plant maintenance can improve maintenance scheduling and failure prediction [21], ERP-based modules reduce non-value-added tasks and streamline workflows [22], and integrating Reliability-Centered Maintenance (RCM) with Computerized Maintenance Management Systems (CMMS) improves metrics such as Mean Time Between Failures (MTBF) and Mean Time to Repair (MTTR) [23]. A review further highlights that intelligent ERP optimization using ML improves failure forecasting and resource allocation for preventive and corrective actions [24].

Despite the progress in perception, recent surveys note that most UAV studies stop at reporting detection accuracy without demonstrating how predictions trigger scheduled maintenance in enterprise systems. The absence of end-to-end pipelines that connect computer-vision outputs to asset-management workflows (ERP) remains a key barrier to operational adoption [20]. While prior studies demonstrate effective visual inspection techniques and promising ML-based approaches [12], three key gaps persist:

(i) limited use of UAVs for scalable, high-frequency data collection directly tied to automated defective/non-defective classification;

(ii) insufficient generalization across varied track conditions without deep feature embeddings;

(iii) weak integration between defect-detection outputs and ERP maintenance modules for actionable scheduling and resource optimization [20-22].

This study proposes a comprehensive UAV-based predictive-maintenance framework that integrates UAV data acquisition, deep feature embedding, and comparative evaluation of ANN, SVM, RF, and KNN within the AI Ladder workflow, and infuses predictive insights into ERP maintenance modules.

This work makes three key contributions that explicitly bridge the perception-to-action gap. First, it advances the theoretical/methodological frontier by introducing an end-to-end, layered analytics pipeline that couples UAV-captured imagery with deep-learning-based feature embeddings and multiple ML classifiers, enabling robust defective (bad)/non-defective (good) classification across diverse track conditions. Second, it delivers operational system integration by infusing classifier outputs into ERP maintenance modules, converting predictions into scheduled actions automated work orders, prioritization, and optimized resource allocation consistent with prior evidence on ERP-enabled reliability improvements [20, 22]. Third, it provides practical impact through a scalable, future-ready approach that increases inspection frequency and accuracy while reducing reliance on labor-intensive methods, thereby advancing SDG 9 objectives. Collectively, these contributions close the loop from UAV-based perception to enterprise maintenance execution.

This part will demonstrate the outcomes obtained from each stage of the AI Ladder, which includes collect, organize, analyze, and infuse data. These stages serve as the foundational framework for implementing AI within the company. By thoroughly examining each phase, the study aims to provide a clear understanding of how data is transformed into actionable insights and integrated into business processes, ultimately driving more informed decision-making and enhancing organizational performance.

3.1 Collect

3.1.1 Dataset collected from UAVs

The dataset utilized in this study comprises rail track images captured by the UAVs. The images in the dataset exhibit various forms of defects, such as cracks, deformations, and surface wear, which could compromise the operational integrity of railway infrastructure.

Each image captured by the UAV is annotated using a corresponding binary mask, an image of the same dimensions consisting solely of black (0) and white (1) pixel values (Figure 5). White pixels indicate regions where visible rail defects such as cracks, deformations, or wear are present, while black pixels represent defect-free areas. They serve as the ground truth labels during model training [33], enabling the machine learning algorithms to learn the visual patterns associated with defective and non-defective track segments.

5.png

Figure 5. Image data and its binary mask (0 (black), 1 (white)) as ground truth labels

3.2 Organize

3.2.1 Feature embedding and extraction

After transferring data to the core, all data is processed and embedded to adjust with the model easily, it is the next step of AI Ladder: organize. The size column represents the file size of each image in bytes. Larger file sizes generally indicate more detailed images [32], which could result from higher resolution or more complex visual content. Meanwhile, the width and height columns specify the image dimensions in pixels. In this dataset, the width is consistently 160 pixels, while the height varies across images, ranging from 333 to 427 pixels. It means that the images may have different aspect ratios, possibly due to variations in how they were captured or processed, as shown in Table 2.

Table 2. Images after embedded and converted into numerical values

While the numerical columns labeled n0, n1, n2, n3, n4, and extending up to n2047, correspond to extracted feature values obtained through an image embedding process. These values are derived using a feature extraction technique, likely based on a deep learning model, where pre-trained data converted high-dimensional image data into lower-dimensional vectors [38]. These vectors capture essential features of the images, such as texture, shape, and patterns, while discarding unnecessary information. Each floating-point number encodes specific visual characteristics of the image, such as texture, structural patterns, or edges.

3.2.2 EDA

Before classification, this research uses distance metrics, scatter plots, and hierarchical clustering (Figure 6) to verify data suitability, cleanliness, and the distribution of "non-defective" and "defective" railway track features.

6a.png

6b.png

Figure 6. Distance and clustering of data through heatmap and scatter plot

The analysis, particularly the distance matrix and dendrogram in Figure 6, reveals distinct clusters and well-defined separability. This confirms that the chosen features effectively differentiate between track conditions, validating the dataset's readiness and providing guidance for supervised learning models [39].

A hierarchical clustering heatmap (Figure 6), ranging from blue (high similarity) to yellow (high variability), alongside dendrograms, reveals strong intra-cluster similarity and distinct groupings within rail track image data. This indicates that the images naturally cluster, suggesting good potential for separating non-defective and defective tracks using unsupervised methods. This visualization helps in evaluating how well the data points cluster together [40].

Furthermore, a scatter plot clearly distinguishes non-defective (red) and defective (blue) classes, demonstrating high separability in the feature space. This strong visual distinction reinforces the idea that classification models should perform well on this dataset.

Normalization using Euclidean distance is crucial [40]. It standardizes feature scales, preventing larger-magnitude features from skewing similarity calculations and ensuring that distance-based models operate effectively and without bias. This preprocessing step helps maintain meaningful relationships and supports accurate clustering and classification [41].

Visual analysis, employing both a hierarchical clustering heatmap and a scatter plot (Figure 7), strongly supports the feasibility of classifying rail track images. The heatmap (blue = similar, yellow = different) and its accompanying dendrograms reveal significant intra-cluster similarity, identifying two primary groups: C1 (largely "non-defective" tracks) and C2 (mainly "defective" tracks), while also highlighting potential transitional states through its hierarchical structure. This clear separability, further emphasized by distinct red ("non-defective") and blue ("defective") groups in the scatter plot, is achieved through crucial Euclidean distance normalization, which ensures unbiased feature representation. These findings confirm that extracted features effectively differentiate track conditions and provide practical guidance for classification: they help determine the appropriate number of classes, inform hyperparameter tuning, aid in selecting balanced training data, and suggest that different defect types may require distinct handling approaches.

7.png

Figure 7. The dendrogram of hierarchical clustering

3.3 Analyze

3.3.1 Model evaluation

Model evaluation is part of Analyze phase in AI Ladder, where machine learning models are applied to interpret and extract meaningful insights from collected and organized data, using performance metrics and visualizations to facilitate understanding and guide decision-making.

Precision-recall curve

Precision–recall curves for RF, ANNs, KNNs, and SVMs demonstrate strong classification performance for the “non-defective (good)” class (left panel), it can be seen in Figure 8. All models maintain high precision across a wide range of recall, indicating accurate identification of non-defective tracks with minimal false positives. The curves’ proximity to the top-right region reflects robust predictive power and a favorable precision-recall trade-off for this class [42].

8a.png

8b.png

Figure 8. Precision-recall curve (“non-defective (good)” on the left, “defective (bad)” on the right)

In contrast, the precision-recall curves for the defective class reveal markedly weaker performance. Across most recall levels, precision is low, implying a higher rate of false positives when predicting defective tracks. The curves’ closeness to the baseline suggests the classifiers struggle to effectively separate defective instances [43]. This highlights a clear disparity: models excel on the non-defective class but face substantial challenges on the defective class.

ROC curve diagram

For the ROC curves, the non-defective class (left) shows stronger separability than the defective class (right). ANN (orange) and SVM (pink) trace the closest trajectories to the top-left corner, achieving higher True Positive Rates (TPR) at lower False Positive Rates (FPR). RF attains mid-tier performance, while KNN consistently yields the weakest curves as shown in Figure 9.

9a.png

9b.png

Figure 9. ROC curve and comparison between models ("non-defective (good)" on the left, "defective (bad)" on the right)

KNN shows the weakest performance, struggling to differentiate classes. While models handle non-defective tracks well, their sensitivity drops when predicting the defective class, increasing the risk of false negatives (missed defects), particularly for RF and KNN.

The confusion matrices provide a detailed breakdown of the classification performance of each model by illustrating the distribution of true positives, true negatives, false positives, and false negatives [43].

The confusion matrices in Table 3 provide a detailed comparison of the four classifiers in identifying rail tracks as defective or non-defective. ANN achieves the most balanced results, correctly classifying 84 defective and 89 non-defective tracks, with 37 misclassifications (21 FN, 16 FP). RF follows closely (85/86 correct; 39 errors: 20 FN, 19 FP), showing moderate, well-balanced performance. SVM attains 84 correct defective and 83 correct non-defective predictions, with 43 errors (21 FN, 22 FP). KNN performs the weakest, correctly identifying 69 defective and 95 non-defective tracks but incurring 46 errors, driven largely by 36 false negatives. Using SMOTE, we obtained a class distribution that is approximately even across categories [35]. Overall, ANN emerged as the most reliable models for rail defect detection.

Table 3. Confusion matrices of four models

The ANN achieved the best performance due to its natural ability to exploit the complexity of feature representations produced through embedding. The embeddings generated in the Organize stage are nonlinear and high-dimensional, enabling models with greater representational capacity to form more flexible decision boundaries compared to other algorithms. This explains why RF, SVM, and KNN lagged behind: while RF provides stability, SVM offers margin-based robustness, and KNN delivers local simplicity, none of these possess the layered learning mechanism required to capture subtle patterns and complex nonlinear interactions within the embeddings [44].

Although overfitting is a potential concern for ANN, this issue was mitigated through model regularization strategies, including cross-validation and careful monitoring of performance. The outcomes across Accuracy, F1, MCC, and AUC indicate that the model did not merely “memorize” the training data but truly generalized. We also observed that ANN maintained stability under small input perturbations, reflecting robustness and an indication of generalization, consistent with the findings of Novak et al. [45], who demonstrated that the model can generalize effectively.

The sample prediction results (Table 4) provide a detailed comparison of how different machine learning models classify rail track images as either defective or non-defective. Observing the classification trends, the Neural Network and SVM models show higher consistency in labeling the images correctly, aligning with their previously reported higher accuracy, F1-score, and MCC values. The Neural Network, in particular, consistently outperforms others by correctly classifying more images as non-defective or defective without frequent misclassification.

Table 4. Prediction result with implementing each model

Conversely, the KNN model appears to be the least reliable, as it often disagrees with the more accurate models, frequently misclassifying defective tracks as non-defective. This reflects its lower recall and precision scores, making it less suitable for defect detection. The RF model performs moderately well but still exhibits occasional misclassifications. These results reinforce the conclusion that neural network models are the best choices for rail defect detection due to their superior classification accuracy, reducing the risk of safety hazards caused by undetected rail defects.

Figure 10 displays a set of randomly selected, unlabeled test images used for classification by the models, serving as a basis to evaluate prediction accuracy. Accurate railway track defect detection is vital for safety and structural integrity. Early identification allows timely maintenance, reducing derailment risks and operational costs. Machine learning models like RF, SVM, ANN, and KNN enhance detection precision and efficiency. Their implementation supports safer railways and more effective maintenance, contributing to a more reliable and efficient rail network.

10.png

Figure 10. Sample image test data

Model performance

To assess the effectiveness in classifying railway track conditions, four machine learning models (ANN, SVM, RF, and KNN) were compared. This evaluation used key metrics like Area Under the Curve (AUC) (distinguishing ability), Classification Accuracy (CA) (overall accuracy), F1-score (balancing precision and recall), Precision, Recall, and Matthews Correlation Coefficient (MCC).

As shown in Table 5, the ANN attains the strongest results, with AUC 0.903, CA 0.824, F1-score 0.824, Precision 0.825, Recall 0.824, and MCC 0.648, indicating robust discrimination and a balanced precision-recall trade-off. The RF follows closely, achieving AUC 0.895, CA 0.814, F1-score 0.814, Precision 0.814, Recall 0.814, and MCC 0.629, reflecting stable and well-balanced predictions. SVM records AUC 0.856, CA 0.795, F1-score 0.795, Precision 0.795, Recall 0.795, and MCC 0.591, showing competent discrimination while trailing ANN and RF. KNN model yields the weakest scores, with AUC 0.869, CA 0.781, F1-score 0.778, Precision 0.799, Recall 0.781, and MCC 0.580, suggesting higher sensitivity to local noise and class overlap in this context. Taken together, the results point to ANN as the most effective and balanced classifier for this task, with RF a close second. SVM delivers solid but lower scores, while kNN is least suited here. If prioritizing missed defects, consider threshold tuning or recall-oriented settings for the selected model.

Table 5. Performance metrics of four models

3.4 Infuse

3.4.1 Focus group discussion

Recommended ERP dashboard features

Experts agreed that the dashboard should prioritize clarity and quick decision-making [46]. Since the classification is binary (defective/non-defective), the interface can remain simple yet informative. Recommended features are included in Table 6.

Table 6. Recommended ERP dashboard elements

Required ERP integration components

The discussion emphasized smooth and automated integration between UAVs-ML outputs and ERP modules, ensuring that detected defective segments trigger relevant business processes without manual intervention [47]. The key ERP modules and their roles are shown in Table 7.

Table 7. Required ERP integration components

The discussion concluded that a minimalist yet actionable dashboard is suitable for this project, as the system only classifies track segments into non-defective or defective conditions. Key emphasis was placed on real-time visual feedback (via map and segment list), automatic ERP-triggered responses (such as maintenance work orders), and alert mechanisms to ensure operational agility [47]. The required integration aligns with the Analyze and Infuse stages of the AI Ladder Model, where simple AI outputs are effectively embedded into core business processes to improve decision-making and responsiveness [48]. The simplicity of the classification model is not a limitation; rather, it supports faster deployment and higher user acceptance in field operations.

Business process re-engineering flow

The following section presents the reengineering of the railway track maintenance process, transitioning from a conventional manual approach to a data-driven, AI-integrated workflow.

3.4.2 As-Is process (Before AI and ERP integration)

Before digital transformation, railway track inspection and maintenance relied heavily on manual visual checks and periodic patrols. This process posed several limitations in speed, accuracy, consistency, and safety.

The manual workflow often resulted in delayed detection of track issues, inefficient use of maintenance resources, and safety. This reactive model (Table 8) lacked the agility and intelligence needed for modern railway infrastructure management [49].

Table 8. As-Is process

To-be process (After AI, UAVs and ERP integration)

After reengineering, the system integrates UAVs for automated data collection, machine learning for defect detection, and ERP for seamless action execution (Table 9).

In this proposed model, each component plays a strategic role, UAVs offer wide and efficient scanning, ML ensures fast and consistent classification of track conditions, and ERP handles operational execution from asset status updates to automated work orders. This end-to-end pipeline enables predictive and intelligent maintenance. It significantly enhances predictive maintenance by enabling real-time asset monitoring and automated decision-making [47].

Table 9. To-Be process

3.4.3 ERP system blueprint

This section defines the system blueprint, which outlines how data captured from UAVs inspections is processed and utilized within the ERP ecosystem to enable predictive maintenance.

Input pipeline

The input pipeline begins with UAVs capturing aerial images of the track, which are then analyzed by machine learning models and converted into structured data inputs (Table 10). It supports efficient transformation of unstructured visual data into actionable insights for railway maintenance [20].

This pipeline ensures that only processed, meaningful data enters the ERP system. The use of an integration layer standardizes the format, enabling consistent updates of track segment conditions and activation of business logic.

Table 10. Input pipeline

ERP module used

The ERP system comprises several interconnected modules, each with a specific function in the maintenance cycle as shown in Table 11.

Together, these modules allow for automated and intelligent workflows. For example, a "Bad" classification triggers a new maintenance request, which is then scheduled, tracked, and reported, all within the ERP environment. This closed-loop system enhances operational efficiency and accountability [12].

Table 11. ERP module used

Dashboard elements

The dashboard is the visual layer of the system, enabling users to monitor track conditions, review alerts, and access historical records.

By combining geospatial visualization (map viewer), data analytics (charts and stats), and operational logs, the dashboard empowers decision-makers with real-time insights. This helps prioritize responses and streamline coordination across departments [50].

3.4.4 UAT matrix

UAT was conducted to evaluate whether the developed system functions according to its intended design and meets user requirements.

3.4.5 Dashboard element

Table 12 lists each core component of the dashboard and summarizes its test results as shown in Table 13. All dashboard elements performed as expected during UAT. Users were able to interact with the map, access filtered tables, receive condition alerts, and trace updates.

This confirmed that the dashboard is both user-friendly and functionally reliable. It significantly enhanced usability and reliability in operational contexts [51].

Table 12. Dashboard elements

Table 13. Dashboard elements of UAT

ERP modules

Each ERP module was tested individually to ensure proper response to AI input and seamless integration across workflows.

The system successfully automated key actions such as condition updates and work order creation. Notifications and reports were generated accurately, confirming that the ERP system can reliably manage predictive maintenance workflows triggered by ML-based track assessments. Accurate generation of notifications and reports confirms that ERP systems integrated with machine learning assessments can effectively support and automate predictive maintenance workflows [52] as shown in Table 14.

Table 14. ERP module used for UAT