Accuracy Rate of Least Square Support Vector Machine Method and Its Various Modifications: A Forecasting Evaluation on Multi-Type Data

ABSTRACT


INTRODUCTION
Forecasting is a crucial aspect of conducting predictive analysis [1].According to Taylor & Letham [2], forecasting is a common task in the field of data science that aids organizations in capacity planning, goal setting, and anomaly detection.Moran et al. [3] assert that forecasting involves the ability to predict future occurrences based on the analysis of past and current data.On the other hand, prediction is a broader concept, referring to assigning a probability distribution to an outcome based on model estimates, applicable to both realized and unrealized outcomes [4].
The Least Squares (LS) method is commonly employed for estimating coefficients in regression models [5].According to Fujii [6], the LS method is one statistical approach used to estimate correlations among various data sets.Meanwhile, Sulaimon Mutiu [7] asserts that the LS method serves as a standard approach in regression analysis for solving overdetermined system approximation, wherein there are more equations than unknowns.The LS method proves valuable in analyzing experimental data to summarize the effects of factors and test linear contrasts among predictions [8].Furthermore, according to Kong et al. [9], the LS method is a way to find the best-fitting function by minimizing the total sum of squared errors between measured points and the corresponding straight line.Consistent with Abazid et al. [10], the LS method finds widespread application in data fitting, aiming for the best fit that minimizes the sum of squared residuals.
Research on the LS method has been applied across various domains for economic data forecasting, such as business cycle data [11], carbon price [12][13][14][15], and stock price estimation [16].Additionally, studies employing the LS method are prevalent in predicting shallow landslides induced by rainfall [17], streamflow forecasting [18,19], and groundwater surface elevation forecasting [20].Su et al. [21] utilized the Least Square Regression Boosting (LSBoost) method in their research on natural gas price forecasting, obtaining low MSE and RMSE values of 1.12% and 1.06%, respectively.Furthermore, findings from a study on predicting cow dung pyrolysis for biochar production indicated that the LS-SVM model outperformed the ANN model with R2 values of 0.96 and 0.80, respectively [22].
Furthermore, the application of the LS method is widespread in climate data analysis, including wind energy data [23,24] and air humidity data [25].Utilizing LS in forecasting hourly fluctuations in the Air Quality Index (AQI) achieved an accuracy level of 93.24%, signifying a remarkably high precision [26].In their study on monthly temperature estimation using the Partial Least Square (PLS) method, Ertaç et al. [27] reported an RMSE value of 1.80% and a high accuracy level of 94%.Moreover, the LS method finds application in forecasting endeavors in the United States and Europe, including the USA [28][29][30][31][32], the Netherlands [33,34], Germany [35,36], and Brazil [37].In Peña-Guzmá n et al. [38] research on forecasting water demand in residential, commercial, and industrial areas using LS-SVM, an accuracy level of 98% and a percentage error below 12% were achieved, indicating a high level of precision.
The LS method yields superior results and presents reasonable long-term elasticity concerning natural resource demand for energy consumption forecasts [39].The LS method is extensively employed in forecasting in Asia, including China [40][41][42][43][44], Turkey [45], and Korea [46,47].Additionally, the integration of the LS method with other approaches in the forecasting domain is prevalent.Examples include LS-SVM for river flow prediction [48], Multi-Task Learning and LS-SVM for electricity load forecasting [49], LS Support Vector Machine Coupled with Data-Preprocessing Techniques (LSSVM-DWA) for river flow forecasting, Volterra-LS Support Vector Machine Model Combined with Signal Decomposition (EMD-LMD-LSSVM-Volterra) for hourly solar radiation forecasting [50], Fuzzy Clustering and Least Square Support Vector Machine Optimized with Wolf Pack Algorithm (FC-WPA-LSSVM) for short-term load forecasting at electric bus charging stations [51], and Least Square Support Vector Machine, Deep Belief Network, Singular Spectrum Analysis, and Locality-Sensitive Hashing (LSSVM-DBN-SSA-LSH) for wind power estimation [52].Research outcomes by Tien Bui et al. [53] indicate that the predictive strength of LSSVM-BC surpasses that of SVM, achieving an accuracy rate of 93.8% in predicting landslides due to rainfall.The IGA-LS-SVM algorithm utilized by Lin et al. [54] provides a more significant MSE value of 0.83%, making it suitable for short-term power load prediction.
Numerous studies on the application of the LS method, either standalone or in combination with various algorithms, have yielded diverse accuracy levels.Earlier research findings indicate that the LS method, combined with algorithms such as Support Vector Machine (SVM), fuzzy logic, Gravitational Search Algorithm (GSA), Genetic Programming (GP), Evolutionary Seasonal Decomposition (ESD), and Signal Decomposition (SD), achieves high accuracy levels based on Mean Absolute Percentage Error (MAPE) values.However, to date, there has been a lack of focus on accuracy levels based on MAPE values.The combination of the LS method with other techniques also impacts the number of iterations, training duration, and testing of data.Therefore, the author has undertaken data collection to discuss the research outcomes of the LS-SVM method, both in its standalone form and in combination with other algorithms.The objective of this research is to assess the comparative accuracy levels of the LS method and its modifications across various types of forecasted data.This research aims to provide a deeper understanding of the effectiveness of the LS-SVM method and its modifications in forecasting various types of data.Consequently, this study not only makes a practical contribution to selecting the optimal algorithm for time series forecasting but also strengthens the theoretical foundation in the development of more advanced forecasting techniques.The anticipated in outcome of this research is an improvement in prediction accuracy, which in turn is expected to yield significant benefits in decision-making processes.

METHOD
This research adopts a quantitative approach utilizing metaanalysis to delve into the error rates of the LS, LS-SVM, and the combination of LS with other algorithms in various forecasting domains such as economics, industry, social sciences, demography, agriculture, and mining.Typically, the classification function of LS-SVM can be formulated as follows [55]: where, (x) is referred to as a nonlinear function mapping from the input space X to a high-dimensional feature space.
The coefficients w and b are obtained by minimizing the upper bound of the generalization error.Thus, Eq. ( 1) can be derived by solving the following optimization problem: where,   is the error variable and  is the penalty parameter.By employing the Lagrangian function and Karush-Kuhn-Tucker (KKT) conditions for optimality in Eq. ( 2), we can derive the final classification solution of the primal problem as follows: In Eq. ( 3), K represents the kernel function, which serves to simplify the utilization of mapping.The research methodology, illustrated in Figure 1, outlines the procedural steps undertaken in this study.

Figure 1. Research procedure
The data collection process involved searching for relevant research findings in both national and international indexed databases.Inclusion criteria encompassed studies conducted between 2013-2023, employing keywords such as "Prediction, Forecasting, LS, LS-SVM", and featuring statistical data including data volume and accuracy parameters (MAD, MSE, MAPE, or RMSE).Conversely, exclusion criteria targeted quantitative studies and articles available only in full text.All relevant articles were downloaded and stored in a designated folder.In the second phase, each article was reviewed to ascertain (1) the input data size (N) during prediction, training, or testing stages, and (2) the value of Mean Absolute Percentage Error (MAPE).Subsequently, coding and tabulation were conducted, encompassing (1) publication year; (2) author names; (3) type of forecasted data; (4) country; (5) forecasting method (LS); (6) data size (N); and (7) Mean Absolute Percentage Error (MAPE) values.The accuracy criteria for MAPE, as outlined by Yadav and Nath [56], are presented in Table 1.
Table 1 Next, the data is tabulated in Microsoft Excel and saved in CVS (Macintosh) format.The stored data is then uploaded to the JASP software for coefficient determination, rank p-value testing, and forest plot generation.The output from JASP is interpreted to assess the accuracy level based on the MAPE values pf the Least Square method in forecasting, both overall and when employing conventional and combined methods.Finally, conclusions are drawn based on the conducted analysis.

The search result
A total of 140 data points were collected for the study, with 54 meeting the specified criteria.There were 19 articles on the LS method, 10 articles on the LS-SVM method, and 25 articles on the LS method combined with other algorithms.The distribution of data is illustrated in Figure 2 and Figure 3.
Based on the analysis of Figure 2 and Figure 3, the data analysis results reveal the utilization of three classification intervals for data forecasting: LS method, LS-SVM method, and LS modification with other algorithms.The LS method demonstrates commendable performance with 19 data points, exhibiting an average Mean Absolute Percentage Error (MAPE) of 0.13, an average Effect Size (ES) of 0.064, and an average Standard Error (SE) of 0.035, categorizing it as highly proficient.In the LS-SVM method, based on dataset of 10 entries, the average MAPE stands at 0.057, with an average ES of 0.104 and an average SE of 0.034, also falling within the excellent category.Meanwhile, the LS method combine with other algorithms showcases outstanding performance, yielding an average MAPE of 0.053, an ES of 0.084, and an SE of 0.035, leveraging a dataset of 25 entries.Furthermore, the data is categorized into three intervals: economic and industrial (38 data points), social and demographic (9 data points), and agricultural and mining (7 data points).The MAPE values for each data point are delineated in Figure 4.
In Figure 4, the analysis results indicate that the average Mean Absolute Percentage Error (MAPE) value reaches 0.081, signifying a remarkably high level of accuracy in forecasting.Furthermore, the data reveals that the minimum recorded MAPE value is 0.001, while the maximum value reaches 0.412.This information serves as an indicator of the model's reliability, considering that lower MAPE values correspond to more accurate forecasts.The subsequent step involves determining the values of effect size (ES) and standard error (SE) based on MAPE values and the number of data points (N), as illustrated in Figure 5.

Homogeneity test and publication bias test
Based on the ES and SE values of each dataset, the data were processed using JASP software to determine the category of the method employed (Fixed Effect or Random Effect model), coefficients, p-value rank test values, and funnel plot to ascertain the average error rate based on the predetermined model.The resulting JASP output can be found in Tables 2-4.According to Table 2, the obtained information indicates that the input data fall into the heterogeneous category, with a residual heterogeneity test value of 1,290.756and a significant variation among the data, indicating the complexity in the characteristics of the dataset used.Furthermore, from Table 3 an estimation coefficient value of 0.073 with a significant pvalue < 0.001 is obtained for the LS method.This illustrates that the LS method demonstrates a low level of error in predicting or forecasting data, affirming the reliability of this method in predictive analysis.Meanwhile, in Table 4 the pvalue of rank correlation is found to be 0.019 > 0.001, indicating no indication of publication bias within the dataset utilized.Thus, the data from the 54 articles included in the study are deemed sufficiently representative to draw conclusions from this research, without significant influence from publication bias.
In Figure 6, the JASP software output reveals that the funnel plot of the overall data indicates no research bias, marked by all circles enclosed or declared as not indicating publication bias.In other words, the results of the p-Rank test < α = 0.05 suggest no indication of bias, as seen in Table 4.The forest plot value is 0.07 [0.05, 0.10], indicating an error rate of 7.3%, thus demonstrating an overall accuracy level of 92.7% for the LS method.Subsequently, the author divides the data based on the classification of methods and algorithms used.The JASP outputs based on these classifications are presented in Table 5.From the results in Table 5, it is evident that the error rate for the LS method with N = 19 is 13% (0.07-0.18), and the estimated value is 0.125 with a p-value < 0.001, signifying an accuracy level of 87.5%.Furthermore, the error rate for LS-SVM with N = 10 yields a result of 4.6% (0.00-0.09), an estimated value of 0.046, and a p-value of 0.028 < 0.05, indicating that LS-SVM has an accuracy level of 95.4%.This aligns with the research by Sun and Liang [57], where shortterm electricity load forecasting using LS-SVM achieved a low MAPE value of 2.34%.However, in the research conducted by Biswas et al. [58], experimental result indicate that the performance of the MARS model slightly outperforms that of the LS-SVM model.For LS-Combination with N = 25, the error rate is 4.4% (0.02-0.06), the estimated value is 0.044, and the p-value is < 0.001, implying that LS-Combination also has an accuracy level of 95.6% in predicting data.Consistent with the findings of Niu and Dai [59], the proposed short-term load forecasting model with EMD-GRA-MPSO-LS-SVM outperforms other models with a MAPE value of 1.1%.In contrast to the study by Yang et al. [60] utilizing the PLSE-RVM method, where the accuracy rate reached 63.3%.The distribution pattern can be observed in Figure 7.
From the JASP output, it is evident that the plot trajectories of the three algorithms indicate no indication of publication bias, as marked by the absence of open circles, meaning no missing studies.Additionally, all circles are enclosed, signifying that the research sample meets minimal standards.In other words, the p-Rank results for each algorithm are greater than 0.001 (p-value > 0.001), implying no indication of publication bias.Here, the author divides the data based on the publication year, data quantity, and data classification in each study.

Moderator variable test
This study delves into the performance of LS method and its modifications in predicting various types of data.By categorizing the data based on publication year, data quantity, and classification.The data research offers a comprehensive understanding of how the method's performance varies across different contexts.Additionally, supplementary analysis is conducted by considering moderator variables, as illustrated in Table 6.
In the publication year interval, it is divided into two periods: 2013-2018 and 2019-2023.For the years 2013-2018, the coefficient value is 0.044, the p-Rank test value is 0.044, and the forest plot is 0.04 [0.03, 0.06], indicating a very accurate with an accuracy level of 95.6%.For the years 2019-2023, the coefficient value obtained is 0.099, the p-Rank test value is 0.149, and the forest plot is 0.10 [0.06, 0.14], meaning the accuracy level reaches 90.1% in predicting data.This aligns with the research by Yan and Chowdhury [61] with a MAPE value of 0.11 or 11%.The findings of Lin and Pai [62] on solar power output prediction using the ESD-LS-SVR method yield a MAPE value of 0.078 or 7.8%, indicating a very accurate.
Furthermore, the data quantity is divided into two categories: equal to or less than 120 (N≤120) and greater than 120 (N>120).For N≤120, the coefficient value is 0.062, the p-Rank test value is < 0.001, and the forest plot is 0.06 [0.03, 0.09], signifying an error rate of 6.2%.This error rate is higher than the findings of Tian's [63] study, which reported a MAPE value of 3.79%.In the case of N>120, the coefficient value is 0.096, the p-Rank test value is 0.007, and the forest plot is 0.10 [0.03, 0.16], with a minimum value of 3% and a maximum value of 16%.These results are higher than the findings of Yang et al. [64] with a MAPE value of 0.0206 or 2.06% in the worthy category.Electricity load forecasting results with a data quantity of 84 and a MAPE value of 0.006 or 0.6%, as well as a MAPE value of 0.008 or 0.8%, both fall into the very accurate category, according to the studies [65,66], respectively.
In the realm of data classification, three distinct categories emerge: economic and trade data, social and demographic data, and agricultural and mining data.Focusing on economic and trade data, where N = 38, the coefficient stands at 0.044, the p-Rank at 0.056, and the forest plot at 0.04 [0.03, 0.06].With a minimum value of 3% and a maximum of 6%, this attests to an accuracy level reaching 95.6%.This aligns with Mustaffa et al. [67] research on commodity price forecasting, boasting a MAPE value of 5.5%, and stock prediction with a MAPE of 0.8% [68].Tang et al. [69] estimated building material prices with an MSE of 2.44% and MAPE of 2.11%.
Transitioning to social and demographic data, where N = 9, a notable error rate is evident with a coefficient of 0.122, p-Rank of 0.477, and a forest plot of 0.12 [0.05, 0.20], signifying an accuracy level of 87.8%.In Song et al. [70] study on water quality prediction using LS-SVM, a MAPE of 5.2% underscores highly accurate forecasts.Chia et al. [71] achieved a MAPE of 5.6% in water quality index prediction through the SMWOA-LSSVM method.Turning to agricultural and mining data with N = 7, a coefficient of 0.134, p-Rank of 0.562, and forest plot of 0.13 [0.02, 0.24] showcase an accuracy level of 86.6%.Demonstrating very accurate, Yang et al. [72] forecasted short-term electricity load with AS-GCLSSVM, yielding a MAPE of 0.006 or 0.6%.
In this study, the results demonstrate that the Least Square Support Vector Machine (LS-SVM) method and its modifications exhibit a significant level of accuracy in forecasting various types of data.The LS-SVM method proves to achieve a high level of accuracy reaching 95.4% in this research.This finding aligns with previous studies highlighting the superiority of the LS-SVM method in shortterm electricity load forecasting.However, it is noteworthy that the study also finds that the MARS model slightly outperforms LS-SVM in certain contexts.
Furthermore, the findings of this research provide valuable insights for practitioners and researchers in understanding the applicability and limitations of the forecasting methods employed.This study highlights potential avenues for future research, including refining algorithmic modifications, further investigating error rates in specific data categories, and developing more sophisticated forecasting models.It is important to acknowledge the limitations of this study, such as sample size and the scope of data types considered which could be addressed in future research to strengthen findings and broaden our understanding of forecasting methods.

CONCLUSIONS
The utilization of the LS method, combined with algorithmic combinations, has proven instrumental in optimizing regression models for accurate data forecasting.This approach, integrating various methodologies, significantly enhances prediction precision.The analysis reveals an overall accuracy rate of 92.7% for the LS method, categorized as high, with an estimated coefficient of 0.073.Notably, algorithm-specific examination demonstrates an accuracy rate of 87.5% and an estimation value of 0.125 for the LS method.Conversely, the LS-SVM method exhibits a lower accuracy rate of 4.6% (0.00-0.09) with an estimation value of 0.046, indicative of low error rates.
In the case of LS-Combination, an error value of 4.4% is obtained, ranging from 2% to 6%, with an estimation value of 0.044, resulting in an impressive accuracy level of 95.6%.Further accuracy analysis within the data range (N≤120) yields a coefficient of 0.062 and a forest plot of 0.06 [0.03, 0.09], signifying an accuracy level of 93.8%.For datasets exceeding 120 (N>120), a coefficient of 0.096, a p-Rank value of 0.007, and a forest plot of 0.10 [0.03, 0.16] signify an accuracy level of 90.4%, categorized as high.
Moreover, the data classification analysis underscores the highest accuracy in economic and trade data at 95.6%, accompanied by a coefficient of 0.044, a p-Rank value of 0.056, and a forest plot of 0.04 [0.03, 0.06].Conversely, social and demographic data exhibit a considerable error, with a coefficient of 0.122, p-Rank value of 0.477, and an accuracy level of 78%.Meanwhile, agricultural and mining data show a coefficient of 0.134, a p-Rank value of 0.562, and a forest plot of 0.13 [0.02, 0.24], indicating an accuracy level of 86.6%.
The findings underscore the efficacy of the LS method in enhancing prediction accuracy across various datasets, particularly in economic and trade contexts.The study highlights the practical implications of these findings in improving forecasting techniques, with potential future research directions focusing on refining algorithmic combinations and addressing error rates in specific data categories, such as social and demographic datasets.

Figure 2 .Figure 3 .
Figure 2. Data quantity based on method and data classification

Figure 6 .
Figure 6.Funnel plot of overall data

Figure 7 .
Figure 7. Funnel plot of each method

Table 1 .
presents criteria indicating the accuracy level based on Mean Absolute Percentage Error (MAPE) values to classify whether forecast outputs can be accepted or rejected.MAPE values <10% indicate highly accurate forecasting, 10% < MAPE <20% denotes good forecasting, 20% < MAPE <50% suggests acceptable forecasting, and MAPE >50% indicates inaccurate forecasting.Subsequently, Mean Absolute Percentage Error (MAPE) values were transformed into effect size (ES), and finally, ES values were converted into summary effect (SE) using the following formula: Criteria for MAPE value

Table 2 .
Fixed and random effect

Table 4 .
p-value of rank correlation