An Optimized Multi-Scale Hierarchical Transformer Framework for Accurate Battery State-of-Charge Estimation Using Adaptive Hyperparameter Optimization

Numerous studies have utilized advanced approaches to improve prediction accuracy and computational efficiency. Here, Louis and Sampathkumar [9] presented a DL based SOC prediction named the Arithmetic Optimized Deep Belief Network. This method is used to enhance prediction accuracy and reduce convergence time. Arithmetic optimization is used to fine-tune the model to attain an accurate SOC prediction for EV applications.

Wu et al. [10] conducted a review of models like CNNs, RNNs, and transformers for SOC prediction. Therefore, the hybrid and adaptive learning model performed well in dynamic temperature and load environments. Also, a filtered DL model-based SOC prediction is developed by Korkmaz [11]. In this work, advanced filtering mechanisms are used to eliminate sensor noise and outliers from voltage and current signals before prediction. The method achieved a stable result by reducing error propagation.

Madani et al. [12] presented an analysis of machine learning (ML) and DL models for SOH estimation. In this work, DL models like CNN-LSTM hybrids can outperform ML models under varying temperature and load conditions. The DL method can capture temporal degradation patterns and nonlinear relationships among electrochemical variables better than ML models.

Sylvestrin et al. [13] provided an SOH estimation using ML that emphasized the need for lightweight, embedded AI solutions to enable on-board diagnostics in EVs. Also, Zhang et al. [14] presented a DL model to predict the Remaining Useful Life (RUL) of lithium-ion batteries using sparse segment data. These data are processed via a cloud computing infrastructure to accelerate training and improve model robustness in SOC prediction.

Amin et al. [15] reviewed various EV range prediction models using ML/DL, mathematical, and simulation-based approaches. They used an ensemble and hybrid learning method to achieve better generalization for SOC in EVs. Meanwhile, How et al. [16] conducted a review of SOC estimation by classifying it into Kalman filter variants, adaptive observers, and learning-based estimators. This method is used to balance the trade-off between model interpretability and computational demand in real-time EVs. Also, ML-based SOC prediction is carried out by Szumska et al. [17] to attain accurate EV prediction. It revealed that neural-based predictors improve forecasting accuracy significantly with sensor fusion techniques.

Giazitzis et al. [18] developed TinyML models for SOC estimation based on Electrochemical Impedance Spectroscopy (EIS) data. Here, the neural models demonstrated that edge AI can deliver low-latency inference effectively, which is suitable for embedded battery management systems.

Table 1. Comparison of State of Charge (SOC) prediction methods

Finally, Caferler et al. [19] reviewed DL models like regression, reinforcement, and classification methods that adaptively predict energy demand. It highlighted the need for lightweight architectures to balance accuracy with on-board computational limits. The overall summary of the survey, with their limitations, is given in Table 1.

Overall, the current SOC prediction models suffer from limited multi-scale temporal modeling and lack cross-domain feature fusion between voltage, current, and temperature. These approaches focus on single-scale models and do not fully capture the complex interactions across different sensor modalities. In addition, existing methods fail to use advanced hyperparameter optimization techniques. To solve these issues, the MSHT model based on multi-scale attention mechanisms is proposed to capture both fine-grained and coarse temporal dependencies. Also, it integrates cross-domain attention for effective fusion of voltage, current, and temperature data. It improves the model's ability to learn inter-domain correlations.

The experimental validation of the optimized MSHT for battery SOC estimation using the NASA Li-ion battery dataset (https://www.nasa.gov/content/prognostics-center-of-excellence-data-set-repository) is presented. This dataset contains multivariate measurements from four lithium-ion batteries, like #5, #6, #7, and #18. It contains the details of controlled charging, discharging, and impedance tests. It has attributes of voltage (V), current (I), temperature (T), and impedance across cycles for SOC regression tasks. To avoid data leakage caused by sequential correlations in battery measurements, the dataset is split based on chronological battery cycles rather than random sample selection. To ensure reproducibility and avoid temporal leakage, the battery cycles are divided chronologically into training, validation, and testing subsets. The first 70% of battery cycles are used for model training. The subsequent 15% of cycles are assigned for validation and RDO-based hyperparameter optimization. The remaining 15% of cycles are reserved for final testing. No random shuffling was applied during dataset partitioning. The validation set is used for RDO-based hyperparameter tuning. The testing set contains unseen cycles for unbiased SOC prediction evaluation. The results of the proposed and existing methods are validated using the metrics that are given in the equations below:

$M A E=\frac{1}{N} \sum_{i=1}^N\left|\widehat{S O C}_i-S O C_i\right|$        (30)

$M S E=\frac{1}{N} \sum_{i=1}^N\left(\widehat{S O C}_i-S O C_i\right)^2$        (31)

$R M S E=\sqrt{\frac{1}{N} \sum_{i=1}^N\left(\widehat{S O C}_i-S O C_i\right)^2}$         (32)

$M A P E=\frac{100 \%}{N} \sum_{i=1}^N\left|\frac{\widehat{S O C}_i-S O C_i}{S O C_i}\right|$         (33)

$R^2=1-\frac{\sum_{i=1}^N\left(\widehat{S O C}_i-S O C_i\right)^2}{\sum_{i=1}^N\left(S O C_i-S \bar{O} C\right)^2}$        (34)

where, $N$ denotes the total number of samples, $S O C_i$ denotes an actual value, and $\widehat{S O C}{ }_i$ represents the predicted SOC.

Table 3. Raindrop Optimization (RDO) tuned parameters

Table 3 presents the MSHT’s hyperparameters tuned by the RDO Algorithm. The MSHT parameters include embedding dimension $\left(d_{\text {model}}\right)$, number of attention heads (num_heads), learning rate $(\eta)$, dropout rate $(\delta)$, and cross-domain fusion flag, respectively.

In this section, the experimental results of optimized MSHT are trained and evaluated using four different datasets. But for visualization, the NASA B0005 dataset is used in this section. To further confirm the model’s effectiveness, additional experiments were conducted on the NASA B0006, B0007, and B0018 datasets.

Figure 2 demonstrates the RDO effectiveness in tuning the MSHT model's hyperparameters. It shows that the 64.7% improvement in Mean Absolute Error (MAE) metrics the value of the metaheuristic approach.

Figure 3 shows the training loss and validation loss against the number of epochs. Both the training and validation loss decrease rapidly up to about Epoch 5 and then stabilize at a very low value (around 0.1). Crucially, the validation loss tracks closely with the training loss and remains low that the model is learning effectively and not significantly overfitting to the training data.

Figure 4 shows the Training MAE and Validation MAE against the number of epochs. MAE is a direct measure of prediction error. Similar to the loss plot, both MAE values decrease quickly and then flatten. The final Validation MAE is very low (around 0.085), which aligns with the overall excellent performance. The curves tracking closely confirm robust generalization.

2.png

Figure 2. Raindrop Optimization (RDO) algorithm

3.png

Figure 3. Training loss and validation loss against the number of epochs

4.png

Figure 4. Training Mean Absolute Error (MAE) and validation MAE

Figure 5 shows that the optimized MSHT demonstrates high prediction performance, with an R² of 0.9896. This model can explain 98.96% of the variation in actual SOC values. The MAE of 0.0237 shows an average prediction error of only 2.37%, while the Root Mean Square Error (RMSE) of 0.0298 shows that larger deviations are rare.

Figure 6 shows the time series plot to compare the True SOC and the predicted SOC for the first 100 samples. The predicted SOC lines show close agreement between predicted and actual SOC values. This proves that the model is able to track rapid changes in the battery's state accurately.

5.png

Figure 5. Performance metrics

6.png

Figure 6. True State of Charge (SOC) vs predicted SOC

7.png

Figure 7. True State of Charge (SOC) vs. predicted SOC for R²

In Figure 7, the True SOC vs. Predicted SOC data points are clustered very tightly along the "Perfect Prediction" line. Here, the proposed work has a high R² value of 0.9896. This method confirms a SOC value is accurate across the entire range of 0.0 to 1.0.

Figure 8 shows the frequency of various Prediction Errors. The error distribution attained a highly centered distribution around zero, which indicates an unbiased model. Also, Figure 9 presents the Absolute Error for the first 200 samples with an overall MAE (0.0237). The majority of individual absolute errors are consistently below the overall MAE, which has small and well-contained errors.

8.png

Figure 8. Error distribution

9.png

Figure 9. Absolute errors over samples

10.png

Figure 10. Input features (first sample)

Figure 10 presents the time series plot showing the normalized input features (Voltage, Current, and Temperature) across 50-time steps for a single prediction sample. It is used to visualize the multivariate time series data that the MSHT processes. The highly non-linear and coupled nature of the features justifies the multi-scale attention mechanism to find the complex dependencies required for accurate SOC estimation.

The importance score is assigned to each of the 50-time steps in the sequence, shown in Figure 11. The MSHT does not assign uniform importance to all time steps. The fluctuation shows that the model is selectively focusing its attention on different historical time steps (e.g., higher importance around time steps 15, 37, and 45). It is used to extract the most relevant information for the current SOC prediction.

11.png

Figure 11. Temporal importance scores

For dataset #5, the proposed MSHT model achieved the lowest MAE and RMSE, as shown in Table 4. The proposed MSHT model outperformed the Bidirectional Long Short-Term Memory (Bi-LSTM) and Gated Recurrent Unit (GRU) by about 18% and 14%, respectively. The higher R² value (0.9896) signifies a strong correlation between predicted and true SOC values. It confirms model stability and precision. The Generative Adversarial Network (GAN)-based models struggle to capture fine-grained temporal correlations as effectively as MSHT's attention mechanisms. The Generative Adversarial Network (GAN) shows the poorest performance in terms of MAE, RMSE, and R². It may be limited due to its relatively simple architecture. It does not fully capture the complex and multi-domain dependencies inherent in SOC prediction.

Table 4. State of Charge (SOC) prediction performance comparison (NASA B0005 dataset)

Table 5. State of Charge (SOC) prediction performance comparison (NASA B0006 dataset)

Table 6. State of Charge (SOC) prediction performance comparison (NASA B0007 dataset)

Table 7. State of Charge (SOC) prediction performance comparison (NASA B0018 dataset)

Table 8. Before vs after Raindrop Optimization (RDO) ablation study for each dataset

Table 9. Ablation study of the Multi-Scale Hierarchical Transformer (MSHT) model (NASA #5 dataset)

Table 5 shows the performance of the proposed model for dataset #6; the MSHT model maintained high consistency, improving the MAE by 17.5% compared to Bi-LSTM. The optimized hyperparameters allowed better learning of long-term charge–discharge trends, reflected in an R² of 0.9884, which confirms a nearly perfect fit.

In Table 6, the performance of the existing and proposed models for dataset #7 is presented. The MSHT exhibited robust adaptability under varying discharge conditions. It captured subtle voltage fluctuations with a 15% reduction in RMSE and an R² of 0.9879, which indicates high prediction reliability.

Table 7 shows the performance on dataset #1 by the optimized MSHT and the existing models. The MSHT model attained the lowest MAE (0.0241) and Mean Absolute Percentage Error (MAPE) (1.88%). It improved SOC prediction accuracy by 16–20%, that consistent with learning across diverse battery profiles.

Table 8 shows an ablation analysis for four datasets before and after RDO tuning. It is clear that the RDO model enhanced its accuracy and reduced prediction errors across all four datasets. Before optimization, the average MAE was 0.0293, which dropped to 0.0248 after tuning, a performance gain of approximately 15–18%. Also, the ability to maintain low error rates (MAE < 0.026) and high R² values (>0.987) demonstrates its robustness and suitability effectively.

Table 9 shows an ablation analysis of the MSHT on the NASA #5 dataset. This result reveals the respective impact on prediction accuracy and model generalization.

• Full MSHT: It achieved the lowest error rates with an MAE of 0.0237, RMSE of 0.0298, and MAPE of 1.85%. It proved that all modules enhance spatial-temporal representation learning collectively, with higher metric values.
• Without multi-scale attention: By removing the multiscale feature extractor, it has a 15.9% increase in MAE. It shows that used to capture fine-grained temporal dependencies and reduce overfitting to specific time windows.
• Without cascade attention: Removing the cascade attention slightly degraded performance (MAE = 0.0269, MAPE = 2.30%). The hierarchical attention is used to refine the flow of contextual information among encoder layers.
• Without cross-domain attention: When cross-domain attention was excluded, the MSHT performance dropped to R² = 0.9848, MAPE = 2.57%. This shows that inter-domain correlation learning is essential for robust fusion of heterogeneous data sources.
• Without the temporal importance scorer, it used to contribute moderately to stability and convergence. Without it, the model’s MAPE rose to 2.10%, whereas temporal weighting enhances the prioritization of influential time frames during forecasting.

Overall, every module within MSHT contributes positively to prediction accuracy. The combination of all components in the full MSHT configuration ensures the best generalization with minimal prediction error. It improved its performance on MAE by around 16–20% and MAPE by 15–25% compared to partial configurations.

To assess the statistical reliability of the MSHT model, t-tests are performed for each evaluation metric across the four datasets as given in Table 10. The null hypothesis (H₀) assumes that there is no significant difference between the performance of MSHT and the baseline models. The alternative hypothesis (H₁) states that there is a significant difference between MSHT and the baseline models. The absolute prediction errors generated by the proposed MSHT and each baseline model were considered as statistical samples. The test was not performed only on the final average evaluation metrics or single experimental runs. For each battery dataset, the sample-wise prediction errors of all test instances were compared between MSHT and the corresponding baseline models. The p-values from the t-tests are used to determine whether the observed differences in performance metrics are statistically significant. A p-value < 0.05 indicates that the results are statistically significant.

It is observed that the p-values are less than 0.05 for all metrics. The MSHT model's performance is statistically significantly better than the baseline models. In addition, the Wilcoxon signed-rank test is adopted for validation. The test compares paired prediction errors between MSHT and baseline models using sample-wise absolute errors from the same test instances (Table 11). A significance level of 0.05 was considered, where p-values below 0.05 indicate statistically significant improvement.

Table 10. t-test results for each dataset

Table 11. Wilcoxon signed-rank test results

Table 12. Performance comparison of the model for the BMW i3 dataset

Table 13. Comparison of different hyperparameter optimization strategies for Multi-Scale Hierarchical Transformer (MSHT)

To further validate the model real time application, the model is tested on the BMW i3 Dataset. The dataset consists of measurements from 70 journeys made by a BMW i3 EV equipped with a 60 Ah battery pack. The data is collected via mounted EV sensors through the OBD port at a 1Hz sampling rate for the simulation experiments. The dataset includes dynamic operating conditions like vehicle acceleration, deceleration, charging and discharging cycles, and varying power demands during real driving scenarios. The battery measurements contain important parameters like voltage, current, temperature, and SOC under different load variations. Compared to laboratory-controlled NASA battery datasets, the BMW i3 dataset represents practical EV usage conditions with changing driving patterns. The temperature variations and fluctuating current demands add additional challenges for SOC estimation. The comparison results are given in Table 12. The MSHT has the lowest MAE (0.0271) for prediction. Compared to Bi-LSTM, the MSHT is 14.39% more accurate. Likewise, the MSHT has the lowest RMSE of 0.0332. Compared to ESN, the MSHT has a better RMSE improvement of about 26.5%. Similarly, the MSHT model shows the highest R² of 0.9869 when compared to existing models.

To further validate the effectiveness of the RDO-based hyperparameter tuning strategy, the proposed MSHT model was optimized using different hyperparameter optimization approaches like Grid Search, Random Search, and Bayesian Optimization. The comparison is performed on the NASA B0005 dataset using identical training and evaluation settings. The results are presented in Table 13. The RDO-based MSHT achieved the lowest prediction error and highest R² value compared with other optimization strategies. This proves its ability to efficiently explore the hyperparameter space and identify suitable model configurations.