An Advanced Hybrid LSTM–XGBoost Framework for Data-Driven Hydrological Inflow Classification in Complex Dam Systems: The Case of Beni Haroun, Algeria

Water resource management is essential to the sustainable development of the world, and is especially important in areas where climate change significantly affects dam operations and water supply. The precise forecasting of water inflow to dams is vital for effective operation, flood risk reduction, and ensuring a reliable water supply for domestic, agricultural, and industrial uses. However, modeling inflow dynamics remains challenging because hydrometeorological parameters are highly nonlinear and interdependent. In this context, inflow classification plays a critical role in modern hydrological management systems, enabling dam operators to anticipate different inflow conditions, such as low, medium, or high flows, thereby supporting strategic planning for water release, storage, and flood control. Precise inflow classification also improves drought preparedness, supports equitable water distribution among sectors, and ensures downstream ecosystem stability. Consequently, constructing reliable inflow classification models is essential for smart and sustainable reservoir operations under increasing climatic uncertainty.

Artificial intelligence (AI) has emerged as a powerful tool for capturing complex nonlinear interactions among hydrometeorological variables [1]. Numerous studies have highlighted the predictive capabilities of machine learning (ML) and deep learning (DL) approaches for evaporation and inflow forecasting. Tezel and Buyukyildiz [2] demonstrated that artificial neural networks (ANNs) and support vector machines (SVMs) can accurately forecast monthly pan evaporation, outperforming traditional methods. Similarly, Deo et al. [3] applied a relevance vector machine (RVM), extreme learning machine (ELM), and multivariate adaptive regression splines (MARS) to predict evaporation, showing that ML can identify the most relevant predictors and improve accuracy. Wu et al. [4] developed hybrid ELM-based models that incorporate whale optimization algorithm (WOA) and flower pollination algorithm (FPA) for monthly evaporation forecasting in the Poyang Lake Basin, achieving higher success rates than traditional techniques. Shabani et al. [5] used random forest (RF), Gaussian process regression (GPR), and k-nearest neighbors (KNN) to predict evaporation, confirming ML’s strength in capturing nonlinear relationships among climatic parameters. Despite these advances, challenges remain in selecting reliable ML models for hydrological forecasting, particularly regarding model interpretability, computational efficiency, and sensitivity to data quality [6, 7].

The considerable size and complexity of hydrometeorological data further complicate the analysis and modeling. These datasets often consist of variables measured at different time points, requiring efficient techniques to identify patterns and relationships that enhance hydrological understanding and improve forecasting accuracy. ML effectively handles nonlinear relationships among climatic and hydrological variables, whereas DL excels at handling unstructured or complex data. The choice between ML and DL depends on the data structure, model complexity, and available computational resources. To leverage the strengths of both, hybrid models that integrate ML and DL are gaining attention for their improved adaptability, accuracy, and robustness in hydrological forecasting.

This study applies a hybrid deep ensemble learning framework to the Beni Haroun Dam, combining LSTM’s temporal feature extraction with XGBoost's robust classification performance to enhance inflow classification performance. The main contributions of this work are as follows: (1) the development of a hybrid LSTM–XGBoost model that exploits the complementary strengths of ML and DL, (2) a systematic evaluation of the hybrid model against standalone ML and DL approaches to quantify performance improvements, and (3) the application of the framework to a 17-year hydrometeorological dataset of the Beni Haroun Dam, offering insights for intelligent dam management and sustainable water resource planning in the region. The outcomes demonstrate the potential of data-driven models to support decision-making in reservoir operations under climatic variability in the Beni Haroun Dam.

The remainder of this paper is structured as follows: Section 2 presents the materials and methods used in this study. Section 3 provides the background knowledge relevant to the proposed approach. Section 4 describes the modeling framework and implementation details. Section 5 presents and discusses the experimental results. Finally, Section 6 concludes the paper by summarizing the main findings of the study and outlining directions for future work.

This section defines the basic concepts needed to elaborate on our classification methods and presents the theoretical foundations of the models used, with particular emphasis on the formal mathematical formulations of the LSTM network and the XGBoost algorithm. Their rigorous representation clarifies underlying mechanisms and supports the methodological developments that follow.

3.1 Long short-term memory

Recurrent Neural Networks (RNNs) are commonly used to model sequential dependencies by linking inputs across time steps. However, standard RNNs often face challenges in capturing long-term dependencies due to vanishing or exploding gradients. To address these limitations, the LSTM network illustrated in Figure 3 was developed as an improved RNN variant, capable of effectively retaining and leveraging past information over extended sequences.

An LSTM unit comprises three primary gating mechanisms, input gate $\left(i_t\right)$, forget gate $\left(f_t\right)$, and output gate $\left(o_t\right)$, which collectively regulate the flow of information within the network. The input gate determines the extent of new information to be incorporated into the memory cell ($c_t$), while the forget gate decides which parts of the previous memory should be discarded. The interaction between these two gates results in an updated cell state that encapsulates both new and retained information. The output gate controls the degree to which the internal memory is exposed to the next layer or time step, thus generating the final output $\left(h_t\right)$ of the LSTM unit.

Figure 3. Long short-term memory (LSTM) architecture

Mathematically, the behavior of an LSTM cell can be described as follows:

$\begin{aligned} f_t & =\phi\left(W_f y_{t-1}+b_f\right) \\ i_t & =\phi\left(W_i y_{t-1}+b_i\right) \\ o_t & =\phi\left(W_o y_{t-1}+b_o\right) \\ \tilde{c}_t & =\psi\left(W_c y_{t-1}+b_c\right) \\ c_t & =f_t \otimes c_{t-1} \oplus i_t \otimes \tilde{c}_t \\ h_t & =o_t \otimes \psi\left(c_t\right)\end{aligned}$

Here, $\phi$ and $\psi$ represent the sigmoid and hyperbolic tangent activation functions, respectively; ⊗ and ⊕ denote element-wise multiplication and addition; and $W$ and $b$ correspond to the weight matrices and bias vectors associated with the respective gates. Figure 3 illustrates the fundamental architecture of the LSTM model.

The incorporation of this memory mechanism enables LSTM networks to preserve contextual information over long sequences, making them highly effective for tasks involving temporal or sequential data.

3.2 Extreme gradient boosting

XGBoost is a decision tree-based boosting ensemble algorithm that offers greater efficiency, scalability, and flexibility than traditional GB Decision Trees [8]. Being a distributive GB framework, it is known for fast computation and high predictive performance. XGBoost follows the basic concept, whereby an ensemble of weak learners, mostly decision trees, is constructed sequentially. Each newly grown tree is taught to target the residual errors of the previous trees to fine-tune the overall accuracy [9, 10]. It employs a sparsity-aware algorithm and a weighted quantile sketch for approximate tree learning, whereby large-scale datasets with billions of examples can be efficiently handled [8].

Additionally, XGBoost optimizes an arbitrary differentiable loss function, allowing it to capture complex data patterns more effectively than conventional methods [11]. Its popularity in ML competitions can be attributed to its robust predictive performance, compatibility with distributed processing frameworks, and advanced system-level features such as cache-aware access patterns, data compression, and sharding [8].

In the context of a classification task, consider an ensemble composed of $K$ additive classification trees. The prediction for the $i$-th instance $x_i$ is defined as:

$\hat{y}_i=\sum_{k=1}^K f_k\left(x_i\right)$              (1)

where, $f_k$ denotes the function learned by the $k$-th decision tree. The overall objective function is a combination of the loss function and a regularization term:

${Obj}(\theta)=\sum_{i=1}^n l\left(y_i, \hat{y}_i\right)+\sum_{k=1}^K \omega\left(f_k\right)$           (2)

Here, $l$ is a differentiable convex loss function measuring the discrepancy between the true label $y_1$ and prediction $\hat{y}_1$, and $\omega\left(f_k\right)$ is a regularization function penalizing the complexity of the $k$-th tree. At the $t$-th boosting round, the prediction is updated as:

$\hat{y}_i^{(t)}=\hat{y}_i^{(t-1)}+f_t\left(x_i\right)$                (3)

The objective at this step is:

${Obj}^{(t)}=\sum_{i=1}^n l\left(y_i, \hat{y}_i^{(t-1)}+f_t\left(x_i\right)\right)+\omega\left(f_t\right)$              (4)

To facilitate optimization, the loss function is approximated using a second order Taylor expansion:

$x_{1,2}=\frac{-b \pm \sqrt{b^2-4 a c}}{2 a}$               (5)

${Obj}^{(t)} \approx \sum_{i=1}^n\left[g_i f_t\left(x_i\right)+\frac{1}{2} h_i f_t^2\left(x_i\right)\right]+\omega\left(f_t\right)+const$                    (6)

where the first and second-order gradients, respectively.

$g_i=\frac{\partial}{\partial \hat{y}_i^{(t-1)}} l\left(y_i, \hat{y}_i^{(t-1)}\right)$              (7)

$h_i=\frac{\partial^2}{\partial \hat{y}_i^{(t-1)^2}} l\left(y_i, \hat{y}_i^{(t-1)}\right)$                  (8)

Neglecting constant terms, the simplified objective becomes:

${Obj}^{(t)} \approx \sum_{i=1}^n\left[g_i f_t\left(x_i\right)+\frac{1}{2} h_i f_t^2\left(x_i\right)\right]+\omega\left(f_t\right)$                (9)

The regularization term is defined as:

$\omega(f)=\gamma T+\frac{1}{2} \lambda \sum_{j-1}^T w_j^2$                  (10)

where, $T$ is the number of leaves, $w_j$ is the weight of the $j$-th leaf, $\gamma$ controls tree complexity, and $\lambda$ is the L2 regularization parameter. Let $I_j=\left\{i \mid q\left(x_i\right)=j\right\}$ be the set of instances in leaf $j$. The objective can then be rewritten as:

${Obj}^{(\mathrm{t})} \approx \sum_{j=1}^T\left[\left(\sum_{i \in I_j} g_i\right) w_j+\frac{1}{2}\left(\sum_{i \in I_j} h_i+\lambda\right) w_j^2\right]+\gamma T$                  (11)

Let $G_j=\sum_{i \in I_j} g_i$ and $H_j=\sum_{i \in I_j} h_i$, then:

$O b j^{(t)}=\sum_{j-1}^T\left[G_j w_j+\frac{1}{2}\left(H_j+\lambda\right) w_j^2\right]+\gamma T$                   (12)

This formulation allows for efficient learning through greedy expansion, leveraging both first and second-order gradient information.

5.1 Performance of single LSTM and XGBoost models

The hyperparameters for both the LSTM and XGBoost models were selected using a trial-and-error approach. For the LSTM, the number of units, dropout rate, batch size, and learning rate were adjusted to achieve stable convergence and prevent overfitting, using the validation set to guide tuning. For XGBoost, parameters including the number of estimators, learning rate, maximum tree depth, subsample ratio, and regularization terms (reg_alpha, reg_lambda) were tuned iteratively using features extracted from the LSTM on the training set, with validation data employed for early stopping and hyperparameter selection. The final model performance was then evaluated on the independent test set. This strategy ensured a robust hybrid model with optimized predictive accuracy while maintaining manageable computational costs.

LSTM architecture was configured with two stacked LSTM layers comprising 128 and 64 memory cells, respectively, each followed by a 0.3 dropout regularization rate. A fully connected dense layer with 32 neurons and ReLU activation was applied before the final softmax output layer, which generated class probabilities for the three target categories. The network was optimized using the Adam optimizer with a fixed learning rate of 0.001 and trained with a categorical cross-entropy loss function.

The training process employed mini-batches of 32 samples for 100 epochs, with early stopping applied to halt training upon convergence of the validation loss. Weight initialization followed the Glorot Uniform scheme, and L2 regularization λ of 0.001 was imposed on the dense layer to ensure generalization and stability.

Table 2 summarizes the performance of LSTM, XGBoost, and the proposed hybrid model.

The LSTM network achieved an accuracy of 97.20%, demonstrating its ability to capture the complex temporal dependencies and sequential relationships inherent in the dataset. The high precision (97.13%) and recall (97.38%) indicate that the model effectively identified true positives while minimizing false positives, reflecting both high sensitivity and robust generalization. The F1-score of 97.24% and specificity of 98.50% further confirmed a balanced performance in distinguishing positive and negative samples. From a statistical perspective, Cohen’s Kappa of 0.96 and an MCE of 2.80% indicate strong agreement between the predicted and true labels with minimal misclassification.

The training and validation curves presented in Figures 6 and 7 illustrate the convergence behavior of the proposed LSTM network during model optimization, showing the evolution of model accuracy across 100 epochs for both training and validation sets. The two curves rapidly increase within the first 10 epochs, stabilizing around a high accuracy level thereafter. There are several indications of model generalization to unseen data, given the absence of divergence between training and validation accuracies. The training process was smooth, and no noticeable divergence between the training and validation curves was observed, which is an indicator that overfitting was not present, and the network successfully captured the underlying temporal dependencies without memorizing the training data.

These plots confirm that the LSTM achieved stable learning and excellent convergence behavior, maintaining a balance between bias and variance. This stability validates the LSTM’s suitability as a feature extraction mechanism, enabling it to learn compact, discriminative temporal representations of the inflow-related variables. In subsequent stages, these deep LSTM features are leveraged by the XGBoost classifier, which capitalizes on their informative structure to achieve enhanced classification performance. The absence of overfitting in the LSTM phase ensures that the extracted features remain generalizable, thereby improving the reliability and interpretability of the hybrid LSTM–XGBoost framework.

Table 2. Performance comparison of LSTM, XGBoost, and the proposed hybrid model across training and testing phases

Note: LSTM = long short-term memory; XGBoost = networks with extreme gradient boosting

6.png

Figure 6. Training and validation accuracy curves of the LSTM model during the learning phase

7.png

Figure 7. Training and validation loss curves of the LSTM model during the learning phase

An XGBoost classifier was employed to model complex nonlinear relationships within the dataset by aggregating a large number of weak learners into a powerful ensemble. The classifier was configured with 500 estimators and a maximum tree depth of six, balancing model expressiveness and generalization capability. A learning rate of 0.05 was adopted to ensure gradual convergence and prevent overfitting. To enhance diversity among trees, subsample and column-sample-by-tree ratios were both set to 0.8, allowing partial feature and sample usage per iteration. Regularization terms (α = 0.001, λ = 1.0) were applied to control model complexity and reduce variance, while Gamma (0.1) imposed a minimum loss threshold for split creation. The XGBoost classifier achieved an overall accuracy of 93.75%, capturing the complex nonlinear relationships in the structured tabular data. The precision (93.95%) and recall (93.47%) reflect its ability to balance false positives and true positives, whereas an F1-score of 93.70% and specificity of 96.44% indicate strong performance across all classes. Cohen’s Kappa of 0.90 and MCE of 6.25% confirm the model’s stability and generalization capability.

5.2 Interpretation of hybrid LSTM-XGBoost model performance

The hybrid LSTM–XGBoost model achieved a high accuracy of 99.12%, representing the most advanced and comprehensive predictive framework among the single LSTM and XGBoost models.

This hybrid integration capitalizes on the complementary strengths of both paradigms. LSTM’s ability to autonomously extract high-level temporal and contextual embeddings, and XGBoost’s powerful ensemble-based discrimination capability optimized for structured tabular data. In the proposed hybrid framework, the LSTM network first performs deep feature extraction, transforming the raw input into a compact, information-rich latent representation. These deep features are then passed to the XGBoost classifier, which operates as a meta-learner capable of refining decision boundaries based on the extracted representations. This architecture effectively combines DL’s representational expressiveness with XGBoost’s gradient-based interpretability, producing a combination enhancement in model performance. As summarized in Table 2, the hybrid model achieved a precision of 99.15%, a recall of 99.09%, and an F1-score of 99.12%, confirming its ability to accurately classify both positive and negative in the hybrid model stances with minimal misclassification. The specificity of 99.50% further underscores the model’s outstanding capability to correctly identify negative samples, ensuring highly reliable discrimination across all classes. The Cohen’s Kappa coefficient of 0.99 signifies near-perfect agreement between the predicted and actual labels, reflecting exceptional consistency and reliability beyond random chance. Additionally, the MCE of just 0.88% demonstrates almost negligible misclassification, emphasizing the hybrid model’s robustness and stability. This level of performance surpasses the individual LSTM and XGBoost models and highlights the latent potential of hybrid deep ensemble architectures in high-stakes data modeling tasks.

Figure 8. LSTM confusion matrix

Figure 9. Networks with extreme gradient boosting (XGBoost) confusion matrix

Figure 10. The proposed model confusion matrix

The results substantiate that deep sequential learning can significantly enrich the feature space of GB algorithms, enhance class separability, and mitigate residual classification errors. Such findings challenge the prevailing assumption that ensemble boosting methods are inherently self-sufficient in feature extraction. Instead, the hybrid results demonstrate that even sophisticated tree-based learners can substantially benefit from deep, non-linear embeddings generated by neural architectures like LSTM.

The confusion matrices displayed in Figures 8-10 illustrate the comparative performance of the three models. Figure 10 corresponds to the proposed hybrid model, while Figures 8 and 9 represent the standalone LSTM and XGBoost models, respectively. The confusion matrix shows that the LSTM model achieves strong results, with 295, 310, and 607 correct predictions for Classes 0, 1, and 2, respectively. However, several Class 2 samples are misclassified as Class 1, indicating difficulty in distinguishing closely related inflow categories. Similarly, XGBoost produce 283, 293, and 594 correct classifications, but shows greater confusion between Classes 0, 1, and 2 due to its lack of temporal feature learning. In contrast, the hybrid model achieves the highest classification accuracy, with 300, 314, and 623 correctly predicted instances and minimal misclassifications across all classes.

These visualizations suggest that the hybrid architecture effectively captures both the sequential dynamics and nonlinear relationships inherent in hydrological inflow data.

5.3 Comparison with machine learning models

The proposed hybrid model was benchmarked against conventional algorithms, including GB, LR, SVM, KNN, and GNB. As shown in Table 3, it consistently outperformed all alternatives in terms of the evaluation metrics, with respective accuracy of 95.43%, 94.55%, 88.42%, 86.06%, and 58.09%.

These results highlight the hybrid model’s capacity to capture complex nonlinear interactions missed by traditional approaches, delivering highly reliable and high-precision predictions suitable for advanced water resource management.

Table 3. Performance comparison of the proposed hybrid model and baseline classifiers during training and testing phases