Development of the Support Vector Regression and Genetic Algorithm Hybrid Models for Forecasting of Madura Rice Yield

2.3 Data preprocessing

Data preprocessing is a critical step in dataset preparation, enabling the production of high-quality data that improves the performance of predictive models. There are several stages in the data preprocessing process, including identifying null or empty values, replacing each column with the column mean (setting missing values to 0), removing outliers, and deleting temporary columns not used for prediction [30]. In this study, data preprocessing is performed not only as a technical step but also as an analytical step to reduce noise, address missing values, and scale the data to suit the model's characteristics [31]. Data cleaning is the process of identifying missing data, invalid zeros, and duplicate entries. When empty or meaningless zero values are found, replacement is performed using the column average value (mean imputation). Analytically, this method was chosen because it preserves the underlying distribution of the data without reducing the sample size, thereby minimising information loss in relatively small datasets, such as rice harvest data with 420 observations. Data cleaning results will affect the information generated by data mining techniques, as the processed data will be reduced in quantity and complexity. Meanwhile, outlier data are identified as extreme values that fall far outside the normal range. The presence of outliers is known to cause significant bias in nonlinear regression models, including SVR. Therefore, outliers are removed or revised using the Interquartile Range method or Z-score analysis. Analytically, removing outliers improves model stability by reducing variance and avoiding the formation of hyperplanes that deviate from the majority data pattern [32]. The presence of outliers will disrupt the overall data analysis, as it can bias the conclusions drawn. The next stage is the data transformation process, in which techniques modify data values and types to conform to the analysis requirements. One data transformation technique is data normalisation. This study used the Min-Max normalisation technique. The selection of Min-Max in this research is based on the following analytical considerations:

1. SVR sensitivity to the range of values, not to the form of distribution. SVR models, especially those with RBF kernels, are more sensitive to feature scaling than to distributional normality. Because RBF operates based on Euclidean distance, equalising the value range is a top priority, and Min-Max provides the most consistent range control (0–1).
2. Parameter stability in GA optimisation. GA explores SVR parameter combinations within a solution space heavily influenced by the range of input values. Min-Max produces fixed, standardised value boundaries, thereby accelerating GA convergence and reducing the risk of overly broad parameter exploration when using the Z-score method, which has an unconstrained scale.
3. Feature distributions are not extremely skewed. Initial descriptive analysis of the six features indicates that skewness remains moderate. Therefore, applying a log or Z-score transformation does not significantly improve distributional evenness, whereas Min-Max preserves comparative information across features without substantially altering the curve's shape.
4. Consistency on a limited dataset size. The dataset of 420 data points is relatively small for a log or Z-score transformation, which is more stable when the sample size is large. Min-Max is safer to use on small datasets because it is less sensitive to fluctuations in the mean or standard deviation. Considering these factors, Min-Max normalisation was chosen as the most appropriate method for this case. The Min-Max normalisation technique can be found using Eq. (1) [33]:

$x^{\prime}=\frac{x-x_{min }}{x_{max }-x_{min }}$     (1)

where, $x^{\prime}$ is the normalized data, $x_{min}$ is the minimum value of the data per column, $x_{max}$ is the maximum value of the data in each column, x is the actual data. This technique scales all data to a predetermined minimum-maximum range, typically with the minimum set to 0 and the maximum to 1. After the data normalisation process, the final stage of preprocessing is to denormalise the data, which is the opposite of normalisation, namely returning the data that has been normalised in the range of zero to one to its original scale. Data denormalisation helps improve query performance or ignore significant storage anomalies by using Eq. (2) [34]. SVR models, especially when optimised with GA, are very sensitive to input quality, so proper preprocessing has a direct impact on the final accuracy of the model.

$x_i=x_{{scaled }}\left(x_{max}-x_{min}\right)+x_{min}$     (2)

The application of Min-Max yields a homogeneous range of values, maintains the stability of the SVR distance calculation, and enables the GA optimisation process to reach the optimal point more efficiently. This combination directly improves the accuracy and consistency of crop-yield predictions.

2.4 Analysis Support Vector Regression model

Prediction is the process of estimating future conditions or values from relevant historical data [35]. With advances in computing technology, modern prediction methods increasingly rely on machine learning approaches, including SVR, LSTM, Random Forest, and hybrid methods such as PSO-SVR and SVR-GA. These methods can model nonlinear relationships commonly observed in agricultural data, which are strongly influenced by weather, soil, seasonality, and cropping patterns. The core concept of SVR is structural risk minimisation, which bounds the generalisation error to prevent overfitting to the training data. SVR maps inputs to a high-dimensional space via kernel functions to better capture nonlinear patterns. SVR seeks to find a linear or nonlinear function that best captures the data pattern while being minimally affected by noise. To do this, SVR defines a tolerance margin (ε) around the prediction line, where minor errors are still considered acceptable. Only data points outside this margin affect the model's shape; such points are called support vectors. This approach helps SVR maintain a balance between accuracy and generalizability, making it suitable for agricultural data that tends to be volatile and nonlinear. SVR models can use various kernel types, such as linear, RBF, and polynomial kernels. Essential parameters in SVR are: C (complexity constant), which controls the model's tolerance for error, ε (epsilon-insensitive loss function), which determines the deviation tolerance margin, and γ, which determines the range of influence of points in the RBF kernel. These parameters significantly impact model performance. Tests were conducted by varying C, ε, and γ to assess how model performance changed with these parameters. These kernel functions are used to select the optimal C parameter for the model. The C parameter is a hyperparameter that controls the extent to which the model tolerates errors in the training data [36]. Testing is performed by varying the C parameter in each kernel to assess how model performance changes with C. In addition, SVR is used to find a function f(x) which has deviation ԑ, the largest of the actual values y, for all training data. An SVR with an ε-sensitive loss function produces a regression model, y = f(x), used to predict outputs for the dataset points. The value of the prediction function f(x_i) is determined by a subset of the training data, referred to as support vectors. In general, the SVR uses Eq. (3) [37]:

$f(x)=w^{\prime} a+b$     (3)

where, the weighting vector is symbolised by $w^{\prime}$. The mapping function x is a and b for preference. In nonlinear regression, a kernel function is applied to map the data to a higher-dimensional space. By adding a vector dimension to the λ variables as the preference value can be reformulated into an Eq. (4). Next, to train the data, use a sequential learning process by forming a Hessian matrix using Eq. (5). Then calculate the error value (E) shown in Eq. (6). And calculate $\delta \alpha_i^*$ and $\delta \alpha_i$ by using Eq. (7). In addition, the value calculation is carried out Lagrange Multipliers by using Eq. (8). Repeating the sequential learning process in Eq. (6) until it reaches the maximum number of iterations or has a stopping condition with Eq. (9). If the data meets the requirements ($x_i^*-\alpha_i$) is not equal to 0. It can be referred to as a support vector.

$f(x)=\sum_{i=1}^i\left(\alpha_j^*-\alpha\right)\left(K\left(x_i, x_j\right)+\lambda^2\right)$     (4)

$R_{i j}=K\left(x_i, x_j\right)+\lambda^2$ untuk $i, j=1,2, \ldots, l$     (5)

$E_i=y_i-\sum_{j=1}^i\left(\alpha_j^*-\alpha\right) R_{i j}$     (6)

$\delta \alpha_i^*=\min \left\{\max \left[\gamma\left(E_i-\varepsilon\right),-\alpha_i^*\right], C-\alpha_i^*\right\}$     (7)

$\begin{gathered}\delta \alpha_i=\min \left\{\max \left[\gamma\left(E_i-\varepsilon\right),-\alpha_i\right], C-\alpha_i\right\} \\ \alpha_i^*=\alpha_i^*+\delta \alpha_i^*\end{gathered}$     (8)

$\begin{gathered}\alpha_i=\alpha_i+\delta \alpha_i \\ \max \left(\left|\delta \alpha_i^*\right|\right)<\varepsilon \text { and } \max \left(\left|\delta \alpha_i\right|\right)<\varepsilon\end{gathered}$     (9)

where, $\alpha_j^*$ and $\alpha$ is a symbol of Lagrange multipliers, $K\left(x_i, x_j\right)$ for the kernel to be used, and $\lambda$ represents data deviation, $R_{i j}$ is the Hessian matrix of the $\mathrm{i}^{\text {th }}$ row of the $\mathrm{j}^{\text {th }}$ column, $y_i$ is the actual value, $\gamma$ is the value of Learning Rate, $\varepsilon$ is the loss value, and $C$ is the complexity value, $\alpha_i^*$ and $\alpha_i$ are Lagrange Multipliers.

In SVR, several kernels can be used in testing, including [38]:

1. A linear kernel is used for datasets that are linear with a straight hyperplane. The distance between two vectors is the sum of the products of each pair of input values. The linear kernel is calculated using Eq. (10):

$K\left(x_i, x_j\right)=\sum x_i x_j$     (10)

2. The polynomial kernel has a curved or nonlinear hyperplane shape so that it can separate data with more complex shapes. The polynomial kernel is calculated using Eq. (11):

$K\left(x_i, x_j\right)=1+\sum\left(x_i x_j\right)^m$      (11)

Shaped kernel similar to a linear hyperplane. This indicates that the degree needs to be determined manually during the training process.

3. Kernel RBF is a kernel function that can map input space into a space with infinite dimensions. RBF can be calculated using Eq. (12):

$\left.K\left(x_i, x_j\right)=e(-\gamma)\left(x_i x_j\right)^2\right)$     (12)

A high γ value in the RBF kernel can cause overfitting. Although SVR is capable of modelling nonlinearity, its performance is very sensitive to the choice of C, γ, and ε. Several studies [19, 20] show that the RMSE of SVR decreases greatly when parameters are optimised. SVR without optimisation often gets stuck in non-ideal local configurations. Prediction accuracy is highly dependent on the optimal parameter combination. Therefore, a global optimisation algorithm, such as the GA or PSO, is needed to select the best fit. In terms of accuracy, this model also exhibits greater consistency in searching the SVR parameter space, making the SVR-GA model combination more effective for nonlinear agricultural patterns. This also occurs in LSTM models that require large datasets; the risk of overfitting is very high in small datasets [39]. Thus, the selection of SVR-GA in this study is analytical, not merely descriptive, because it is supported by evidence of performance and the characteristics of the agricultural dataset used.

2.5 Genetic Algorithm optimization

GA uses a natural analogy based on natural selection [40]. GA consists of several stages: chromosome representation, crossover, mutation, and selection. GA begins by generating alternative solutions (a population). Each solution is represented as an individual or chromosome. Fitness values are used to evaluate prediction results; the fitness can be computed as the minimum or maximum value. The fitness value generated by the SVR in this study was computed after the MAPE was calculated in the GA optimisation. Fitness values are inversely proportional to MAPE values, where the lower the better, while a large fitness value indicates a low MAPE result. The use of SVR with GA optimisation is to minimise MAPE by taking the maximum fitness value as the best result. This is achieved using Eq. (13) not only because GA can explore a broad search space, but also because of several identified relative advantages over other optimisation methods, such as PSO. PSO is relatively fast but more prone to local optimums, especially with nonlinear objective functions such as SVR hyperparameter optimisation. GA is used as an optimisation method to determine the best parameters in SVR, namely C and γ. GA works through the general stages of chromosome representation, selection, crossover, and mutation [10, 41]. Each individual in the population represents a set of SVR parameters, and its fitness is computed from the model's MAPE. Because fitness is inversely proportional to MAPE, optimisation efforts focus on identifying the individual with the highest fitness value. The crossover and mutation processes were described in Eqs. (14) and (15), respectively, process to ensure extensive exploration of the solution space, increasing the chance of finding optimal parameters.

$fitness =\frac{1}{1+ { MAPE }}$     (13)

Crossover is a technique of crossing over, or the process of mating between two parent chromosomes to form a new chromosome (offspring). Crossover produces offspring by combining the values of the two parents. This combination is selected randomly from the population. For example, P₁ and P₂ are parents that have previously been chosen for crossover, so that offspring (C₁ and C₂) can be generated using Eq. (14), where the variable α is randomly selected with a range of [0,1].

$\begin{aligned} & C_1=P_1+\alpha\left(P_2-P_1\right) \\ & C_2=P_2+\alpha\left(P_1-P_2\right)\end{aligned}$     (14)

Mutation is a process that creates new individuals by modifying one or more genes within the same individual. The mutation process aims to change genes lost during selection and replace them with genes not present in the initial population. The mutation used in this study is random: it selects one parent from the population at random, as shown in Eq. (15):

$C_i=x_i+r\left(max _i-min _i\right)$      (15)

The final stage is selection. The goal of selection is to determine which individuals will be selected for reproduction and how many offspring each individual will produce. The selection method used in this study is elitist selection. This process ranks all individuals (parents and offspring) by fitness from highest to lowest and passes the highest-fitness individuals into the next generation.

2.6 Evaluation method

In testing, cross-validation is used to assess the accuracy of a model trained on a specific dataset. Cross-validation is a method for selecting a model's parameters by evaluating error on the test data. K-fold cross-validation is a method for estimating a system's average performance by randomly partitioning the data into k subsets and iteratively assessing the model on each subgroup [42, 43]. K-fold cross-validation is performed by dividing the data into n folds. In this study, K-fold cross-validation will be conducted with K values ranging from 2 to 5. Table 3 shows 5-fold cross-validation, with the test data highlighted in yellow and the training data in white. In this paper, K-Fold cross-validation with K = 5 is used because many experimental results indicate that 5-fold is the best choice for obtaining accurate estimates. Accuracy is evaluated by monitoring the evolution of MAPE, RMSE, MAE, and R². The testing technique will focus on the results of the modelled data mining process. The accuracy of the prediction process will be evaluated using the MAPE, as defined in Eq. (16) [44]. The prediction value evaluation results are shown in Table 4.

$MAPE =\frac{1}{n} \sum_{i=1}^n \frac{y_i-y_i^{\prime}}{y_i} \times 100 \%$      (16)

Table 3. K-fold cross-validation for rice yield prediction evaluation

Table 4. Mean Absolute Percentage Error (MAPE) evaluation results in a prediction

RMSE quantifies the difference between the actual and predicted values. This metric gives greater weight to significant errors (outliers) because of the squaring operation, making it sensitive to outliers. The purpose of RMSE is: The smaller the RMSE value (closer to 0), the more accurate the predictive model is. The RMSE equation is shown in Eq. (17) [45].

$R M S E=\sqrt{\frac{1}{n}} \sum_{i=1}^n\left(y_i-\widehat{y}_l\right)^2$     (17)

MAE is the average absolute error between the actual and predicted values. Unlike RMSE, MAE gives equal weight to all errors and is less sensitive to outliers. The purpose of MAE is that the smaller the MAE value (closer to 0), the better the model's performance. The MAE equation is shown in Eq. (18) [46].

$M A E=\frac{1}{n} \sum_{i=1}^n\left|y_i-\widehat{y}_l\right|$     (18)

Meanwhile, R² measures the proportion of variance in the dependent variable that can be explained by the independent variables in the model. R² provides an understanding of how good the model is overall, not the magnitude of the average error. And the purpose of R² is: The higher the R² value (closer to 1), the better the model is at explaining the data. The R² equation is shown in Eq. (19) [47].

$R^2=1-\frac{\sum_{i=1}^n\left(y_i-\widehat{y}_l\right)^2}{\sum_{i=1}^n\left(y_i-\widehat{y}_l\right)^2}$     (19)

Figure 7. Comparison chart of actual data and predicted data