An Efficient Crop Yield Prediction Framework Using Hybrid Machine Learning Model

Agriculture accounts for 60.45% of land use in India. Its agrarian economy stands as a cornerstone of its societal fabric, with agriculture deeply intertwined in its history and present. The nation's reliance on the annual monsoon and farming practices shapes the trajectory of its economy and the well-being of its populace [1]. As one of the most populous countries globally, India's ability to ensure food security hinges upon the efficacy of its agricultural yields. In this context, the accurate prediction of crop yields assumes paramount importance. On the one hand where farmers want timely guidance to anticipate crop output and establish effective methods to boost agricultural produce thereby earning better return on investments, Governments on the other hand must be able to accurately predict agricultural production to achieve national food security and make knowledgeable decisions regarding imports thereby saving crucial forex. To meet the increased food demands of India's burgeoning population, modern technology practices in agriculture are required.

Previously, farmers relied on their own experiences and accurate historical data to anticipate crop yields and make important production decisions based on the prediction. However, in recent years, new breakthroughs such as crop model simulation, precision agriculture, and machine learning have surfaced to estimate yield more precisely, as can analyze massive amounts of data using high-performance computing [2-5]. However, the task of predicting crop yields accurately is riddled with complexity. It entails the intricate interplay of a multitude of factors, encompassing agricultural methodologies, environmental variables, and technological advancements. The challenge is further compounded by the varied and often unpredictable nature of these variables, making traditional forecasting methods fall short in offering reliable predictions.

Precision agriculture [6], a revolutionary approach in modern farming, underscores the significance of targeted and data-driven cultivation practices. This methodology harnesses advanced technologies, including remote sensing, geographic information systems (GIS), and global positioning systems (GPS) as shown in Figure 1, to meticulously analyze and respond to the variability inherent in agricultural fields. By deploying sensors and data analytics, precision agriculture enables farmers to tailor irrigation, fertilization, and pesticide application precisely to the specific needs of different areas within a field. This approach not only maximizes crop yields but also minimizes resource wastage and environmental impact.

1.png

Figure 1. Components of precision agriculture

Statistical models, in comparison, provide a method that predicts direct connections between predictor variables and crop production within a particular dataset without taking into consideration underlying mechanisms in crop ecology and physiology [7]. When given adequate and trustworthy data, statistical models can make good predictions, but they may be constrained by the limits of the training data. However, widely used performance evaluation metrics are possible to get on statistical models that are helpful for uncertainty studies at regional scales and are less dependent on field calibration data. For forecasting agricultural yield, statistical models like multiple linear regressions (MLR) and simple linear regressions are frequently utilized [8].

There are a wide range of advantages to use machine learning (ML) models to forecast agricultural production. Machine learning places an emphasis on discovering patterns and correlations in data settings to precisely predict yields depending on various characteristics. ML models however must train on datasets that reflect previous experiences and results to create prediction models. Using historical data, the models' parameters are set during training. The performance of the model is evaluated during testing using a portion of the historical data that was not utilized for training [9]. Machine learning techniques have the capacity to adjust and learn from the nonlinear and dynamic processes of crop growth, which is essential for making precise forecasts for agricultural production. Due to learning ability from datasets, machine learning models are ideally suited for predicting agricultural yields at large scale. Further, their adaptability to different parameters like crop varieties, temperature, rainfall, humidity, nutrient content (nitrogen, phosphate, potassium, organic carbon, calcium, magnesium, Sulphur, manganese, and copper), pH levels and geographical area, make ML techniques them suitable to model and forecast crop yields [10].

While the potential of machine learning models to forecast crop yields holds promise, existing approaches have encountered a range of obstacles. Chief among these challenges is the dearth of comprehensive and high-quality data, the inherent variability within agricultural systems, the peril of overfitting intricate models, and the intricacies of temporal dependencies. Additionally, the models' efficacy has been hampered by the complexity of their structures and the insufficiency of thoughtful feature engineering.

Considering these hurdles, there emerges a compelling need for the development of more robust and efficient predictive models that can aid in the realm of sustainable crop production. Modern machine learning techniques such as hybrid or ensemble modeling, driven by their ability to make multiple models work together and handle complex and diverse datasets, present a transformative opportunity. Leveraging these tools could enable more accurate forecasts, assisting farmers, policymakers, and stakeholders alike in making informed decisions. In this paper, we harness the individual power of machine learning algorithms such as Decision Tree, XGBoost, Random Forest, Support Vector Machine, and Linear Regression using the agriculture dataset consisting of top 10 globally consumed crops from Kaggle repository. Choosing the top three algorithms in terms of performance, the study then exemplifies the fusion of these algorithms to propose a hybrid ML framework for efficient crop yield prediction and offer this as a tool for use of all stakeholders.

Therefore, the key contributions from this paper are:

1. A hybrid ML framework leveraging top three individually performing algorithms for an efficient and robust crop yield prediction model.
2. To ensure the accessibility of this model for farmers and policymakers, a user-friendly interface called ‘Crop Yield Predictor’ is developed, offering efficient crop yield predictions. This tool can assist in optimizing resources and promoting sustainable food production.

The manuscript is organized as below: Section 1 provides an overview of crop yield prediction using machine learning and statistical models. Existing methods for agricultural yield prediction are reviewed in Section 2 under related works. Section 3 describes the materials and methods for agricultural yield prediction that have been offered. Section 4 contains the results and discussion. Section 5 concludes with references and draws conclusions.

2.png

Figure 2. Overall architecture and workflow of the proposed framework

Table 1. Feature description in dataset

Feature	Feature ID	Description	Dependent/ Independent
Item	Crop	A total of 10 different crops were used in this dataset like paddy, wheat, maize etc. according to the country and the other factors that influence the crop growth	Independent
Pesticides (NPK)	T (tonne)	Data is collected from Food and Agricultural Organization (FAO) and active ingredients or pesticide usage is measured in tonnes
Rainfall	AAR (average annual rainfall)	The average annual rainfall of different countries is measured in milli meters, and the data used was taken from world data bank
Temperature	AT (average temperature)	Average temperature of particular year in particular region is measured in degree Celsius and world data bank helped in data collection
Yield	Y (tonne)	The data is collected from FAO, and it is measured in tonnes	Dependent

3.1 Machine learning models

The ML models used in this work were chosen after doing a literature review. We use Random Forest, XGboost, Decision Tree, Support Vector Machines, and Linear Regression models, which are briefly described on how they are leveraged.

1. Linear regression: Linear regression is a simple and widely used machine learning technique for predicting a numerical value “(dependent variable) based on one or more input features (independent variables). In the context of crop yield prediction, we can use linear regression to model the relationship between various factors (like weather conditions, soil properties, etc.) and the crop yield.

For a single independent variable (feature), mathematically, the linear regression equation is:

$y=b_0+b_1 * x+\varepsilon$           (1)

where:

- y is the predicted crop yield

- b₀ is the intercept term

- b₁ is the coefficient for the independent variable

- x is the value of the independent variable (e.g., weather

 data)

- ε represents the error term

For multiple independent variables (features), the equation becomes:

$y=b_0+b_1 * x_1+b_2 * x_2+\ldots+b_n * x_n+\varepsilon$                     (2)

where:

- x₁, x₂, ..., x_n are the values of the independent variables (e.g., weather, soil properties, etc.)

- b₁, b₂, ..., b_n are the corresponding coefficients for the independent variables

For crop yield prediction, we collect historical data that includes both input features (e.g., temperature, rainfall, soil nutrients) and the corresponding crop yield values. The goal of training the linear regression model is to find the coefficients (b₀, b₁, ..., b_n) that minimize the difference between the predicted crop yields and the actual crop yields in the training dataset [20]. The training process involves using the least squares method to determine the optimal coefficients that best fit the data. Once the model is trained, we use it to predict the crop yield for new sets of input features by plugging them into the equation [21, 22].

The advantages of Linear regression are that it is a straightforward model that is easy to understand and implement. It provides a clear interpretation of the relationship between the independent variables and the dependent variable (crop yield). The coefficients in the linear regression equation provide insights into the direction and magnitude of the effect of each independent variable on the dependent variable. This can help in identifying which factors are most influential in predicting crop yield. These models are computationally efficient, making them suitable for quick analysis and prediction tasks, especially with a relatively small number of features. Linear regression doesn't assume a particular distribution of the data, which can be advantageous when working with different types of agricultural data. Linear regression at best serves as a baseline model, helping to establish a foundation for more complex modeling techniques if needed.

The drawbacks of Linear regression are that it assumes a linear relationship between independent and dependent variables. If the true relationship is non-linear, linear regression might not capture the complexity of the data accurately. Linear regression may struggle to capture intricate interactions and non-linear patterns present in crop yield data. Other models, like polynomial regression or machine learning techniques, might be better suited for such situations. Linear regression is sensitive to outliers, which can disproportionately influence the model's coefficients and predictions. Outliers are not uncommon in agricultural data due to various factors. When independent variables are correlated with each other, it can lead to multicollinearity issues in linear regression. This can affect the interpretation of coefficients and the model's stability. Without proper regularization techniques, linear regression can easily overfit (capture noise) or underfit (oversimplify) the data, leading to poor predictive performance. Linear regression assumes that the relationship between each independent variable and the dependent variable is independent of other variables. It might not handle complex interactions between features well. In cases where the relationship between variables is highly complex, linear regression might not provide accurate predictions. More advanced techniques may be required.

2. Decision Tree: This is a non-parametric supervised machine learning method that is commonly utilized in classification and regression applications. Figure 3 depicts a hierarchical structure with a root node, branches, internal nodes, and leaf nodes. The Gini impurity and information gain strategies are the two most used as splitting criterion in decision tree models, which aid in evaluating the usefulness of each test condition and its capacity to categorize samples into a certain group. Gini impurity and knowledge gain are provided by:

$H(S)=-\Sigma\left(P(c) * \log _2(P(c))\right)$                   (3)

where, H(S) is the entropy of the dataset S, Σ denotes the sum over all possible classes c and P(c) is the proportion of instances in class c within the dataset S.

$I G(S, A)=H(S)-\Sigma\left(\left(\left|S \_v\right| /|S|\right) * H\left(S \_v\right)\right)$                (4)

where, IG(S, A) is the information gain of the dataset S by splitting on feature A, H(S) is the entropy of the original dataset S, Σ denotes the sum over all possible values v of feature A, |S_v| is the number of instances in the dataset S with feature A equal to v, |S| is the total number of instances in the dataset S and H(S_v) is the entropy of the subset S_v.

$I(S)=1-\Sigma\left(P(c)^2\right)$                     (5)

where, I(S) is the Gini impurity of the dataset S, Σ denotes the sum over all possible classes c, P(c) is the proportion of instances in class c within the dataset S.

3.png

Figure 3. Representation of a decision tree implementation

The advantages of DT are that it provides a visual and intuitive representation of the decision-making process. The tree structure allows us to easily understand the factors and thresholds that contribute to predictions. They can capture non-linear relationships and interactions between variables in the data, making them suitable for complex agricultural systems where linear models might fall short. It can handle both categorical and numerical variables without the need for extensive preprocessing, which can be useful when dealing with diverse agricultural data. DTs are relatively robust to irrelevant features. Features that don't contribute significantly to prediction will tend to be pruned off during the tree-building process. They can handle missing values effectively by placing them in a separate branch during the tree-building process. DTs can be combined into ensemble techniques like Random Forests or Gradient Boosting, which often lead to improved predictive performance by reducing overfitting.

The drawbacks of DT are that they can become overly complex and fit noise in the training data, leading to poor generalization on new, unseen data. Small changes in the data can result in significantly different tree structures, making decision trees sensitive to variations and potentially leading to inconsistent predictions. In datasets with imbalanced classes, decision trees might have a bias towards predicting the majority class, especially if not adjusted appropriately. DTs use a greedy approach for building the tree, which might not lead to the globally optimal solution in some cases. Single decision trees can have high variance, especially on small datasets, which might require ensemble methods to mitigate this issue. While DTs can handle non-linearity well, they might not be the best choice for problems where linear relationships dominate. Preventing DTs from growing too deep or becoming too complex is important to avoid overfitting and maintain interpretability.

3. Random Forest: A RF is an ensemble learning technique that combines multiple decision trees to improve predictive performance and reduce overfitting. In the context of crop yield prediction, a Random Forest model can be used to predict crop yields based on various input features. The working can be explained in below 3 steps:

Step 1. Collect a dataset that includes historical data of crop yields and corresponding input features (e.g., weather data, soil properties, etc.). Randomly sample the dataset multiple times with replacement (bootstrap samples). Each sample is used to train an individual decision tree.

Step 2. For each bootstrap sample, build a decision tree using a subset of the input features. At each split node, randomly select a subset of features to consider for splitting. Split the data based on the selected feature that maximizes a certain criterion (e.g., information gain or Gini impurity) at that node. Repeat this process recursively until the tree is fully grown or a stopping criterion is met (e.g., maximum depth, minimum samples per leaf).

Step 3. Once all decision trees are built, predictions are made by aggregating the predictions of individual trees. For regression tasks like crop yield prediction, this aggregation is usually done by calculating the average of the predictions from all decision trees.

The mathematical representation of the Random Forest model is the ensemble average of individual decision tree predictions:

RandomForest $(x)=(1 / N) * \sum($ DecisionTree_$i(x))$                   (6)

where:

- RandomForest(x) is the predicted crop yield for input features x using the Random Forest model.

- N is the number of decision trees in the ensemble.

- DecisionTree_i(x) represents the prediction of the i-th decision tree.

The advantages of RF for are that it reduces overfitting by combining multiple decision trees, which collectively make more robust predictions on new, unseen data. It can capture complex non-linear relationships between input features and crop yields. The model can provide insights into feature importance, helping to identify which factors contribute most to crop yield predictions and it is less sensitive to outliers and data noise due to the ensemble nature of the model.

The drawbacks of RF is that it can be computationally expensive and requires tuning to optimize performance. Although RF can indicate feature importance, the ensemble structure can make the model less interpretable compared to a single decision tree. While less prone to overfitting than individual decision trees, Random Forests can still overfit if the number of trees is too large relative to the dataset size.

4. Support Vector Machine: SVMs are a powerful machine learning algorithm used for classification and regression tasks. For crop yield prediction, SVMs can be employed to create a regression model that predicts crop yields based on input features. SVM regression works as below:

Step 1. We collect a dataset containing historical data of crop yields and corresponding input features (e.g., climate data, soil characteristics, etc.).

Step 2. Define the SVM regression problem. In SVM regression, the goal is to find a hyperplane that best fits the data while minimizing the margin violations (deviations from the predicted values). Choose a kernel function (linear, polynomial, radial basis function, etc.) that maps the input features into a higher-dimensional space, allowing for more complex relationships between variables. Solve the optimization problem to find the coefficients of the hyperplane (weights) and the bias term that minimize the error while maximizing the margin between the predicted values and the actual crop yields.

The mathematical representation of the SVM regression model can be given as:

$y=w * x+b$                      (7)

where:

- y is the predicted crop yield.

- w represents the weights (coefficients) of the hyperplane.

- x is the vector of input features.

- b is the bias term.

Step 3. Once the SVM regression model is trained, we use it to predict crop yields for new sets of input features.

The SVM regression model's objective is to find a hyperplane that best represents the relationship between the input features and crop yields. The prediction is made based on the distance of a new data point to the hyperplane.

The advantage of SVM Regression is that it can capture non-linear relationships between input features and crop yields using various kernel functions. SVMs generally have good generalization properties and can handle overfitting effectively. SVMs focus on the support vectors, which are the data points that influence the margin of error, making the model less sensitive to irrelevant features.

The drawback SVM Regression is that the training can be computationally intensive, especially with large datasets or complex kernels. SVMs have hyperparameters that require tuning, such as the choice of kernel, regularization parameter, etc. SVMs can be less interpretable compared to simpler models like linear regression and they might face challenges in terms of efficiency and scalability when dealing with massive datasets.

5. XGBoost: XGBoost (Extreme Gradient Boosting) is a popular and powerful machine learning algorithm that has shown excellent performance in various tasks, including regression tasks like crop yield prediction. It is an ensemble learning method that combines the predictive power of multiple weak learners (individual models) to create a robust and accurate predictive model. The working of XGBoost can be explained as below:

Step 1. We collect a dataset that includes historical data of crop yields and the corresponding input features (such as climate data, soil characteristics, etc.).

Step 2. Define the problem as a regression task, where the goal is to predict crop yields (a numerical value) based on input features. Install the XGBoost library and prepare the dataset in the required format.

Step 3. XGBoost has various hyperparameters that control the model's behavior. These include learning rate, maximum depth of trees, number of boosting rounds, regularization terms, and more. We perform hyperparameter tuning to find the optimal combination of these parameters that results in the best predictive performance. Techniques like grid search or random search can be used for this purpose.

Step 4. Train the XGBoost regression model on the dataset. The model works by sequentially adding decision trees to the ensemble, where each new tree tries to correct the errors made by the previous trees.

Step 5. Once the model is trained, we use it to predict crop yields for new sets of input features.

XGBoost is known for its accuracy and robustness, making it suitable for complex prediction tasks like crop yield prediction. It provides insights into feature importance, helping identify which factors are most influential in determining crop yields. It can capture complex non-linear relationships between input features and crop yields. Through hyperparameter tuning and regularization techniques, it can mitigate overfitting.

On the other side, setting up and tuning a model will require more effort compared to simpler algorithms. While it can provide feature importance, the model's ensemble nature can make it less interpretable than linear models.”

3.2 Methodology

The methodology adopted for crop yield prediction involves collecting and loading the data into a .csv file, followed by data pre-processing to handle outliers and selecting relevant features. The performance of the model is evaluated using appropriate metrics, and the predicted crop yields are displayed as the final output. Figure 4 shows the flow chart of the methodology adopted which is briefly explained below.

4.png

Figure 4. Flow-chart of the implementation

Data Collection and Loading. This involves importing all the requisite libraries in Python and loading the dataset from kaggle.com. We combine two datasets taken from http://www.fao.org/home/en/ and https://data.worldbank.org/. Feature description in the dataset is shown in Table 1. The dataset contains information on top 10 crops that are grown in USA. There are features like rainfall, temperature and pesticides that are included to determine the yield of the crops. There are around 28K records in the dataset.

Data Pre-processing. The collected data undergoes pre-processing to ensure data quality and integrity. We carry our exploratory data analysis (EDA) on the dataset to identify and select featured based on their relevance to the prediction task. Furthermore, an outlier removal process is applied to detect and eliminate any anomalous data points that could negatively impact the model's performance.

Model Training and Testing. Here, the dataset is divided into a train set and a test set, as well as into independent and dependent variables, or X and Y. 80% is the training data and 20% is the test data set split. Next, all the five ML models are trained using the train dataset and then the algorithm performance is checked on the remaining test data on all metrics. Based on best of three model performance on coefficient of determination (R²) score, we implement a hybrid ML framework.

Prediction and Evaluation. Once the hybrid ML models have been chosen and trained, they are ready to make predictions on new, unseen data. The input data is passed through all the trained models, and it generates predictions for the corresponding crop yield. Evaluation metrics are used to assess the performance of all the models and compare their predictive accuracy.

Output Display. The final step of the system design involves displaying the outputs, which include the predicted crop yields generated by the hybrid ML model. These predictions can be presented visually, such as in a graphical format, or as a tabular output showcasing the predicted yields for each input instance. Finally, a user-friendly interface is developed using the Gradio library in Python to view the predicted crop yield as per the inputs given by any user.

3.3 Evaluation metrics

The proposed models are evaluated using the following metrics.

1. Coefficient of determination (R²): When forecasting the outcome of a given event, the coefficient of determination is a statistical measurement that assesses how variations in one variable may be explained by differences in another one. In other words, this coefficient measures the strength of the linear relationship between two variables. This metric is represented by a value between 0.0 and 1.0, with 1.0 indicating perfect correlation. As a result, it is a trustworthy model for forecasting the future.
2. Root Mean Square Error (RMSE): It is the residuals' standard deviation (prediction errors). Residuals are a measure of how far away data points are from the regression line; RMSE is a measure of how spread out these residuals are. It indicates how concentrated the data is towards the line of best fit.

$R M S E=\sqrt{\overline{(f-o)^2}}$                (8)

where, f= forecasts (expected values or unknown results), and o=observed values (known results).

3. Mean Average Error (MAE): The MAE measures the average magnitude of the errors in a set of forecasts, without considering their direction. MAE is calculated as the sum of absolute errors divided by the sample size.

$M A E=\frac{\sum_{i=1}^n\left|y_i-x_i\right|}{n}=\frac{\sum_{i=1}^n\left|e_i\right|}{n}$                        (9)

It is thus an arithmetic average of the absolute errors $\left|e_i\right|=\left|y_i-x_i\right|$, where $y_i$ is the prediction and $x_i$ the true value.

Both the MAE and RMSE can range from 0 to ∞. They are negatively oriented scores. Lower values are better.

4. Mean Squared Error (MSE): The MSE is calculated as the average of the squares of the errors, or the average squared difference between the estimated and real values.

If a vector of n predictions is generated from a sample of n data points on all variables, and Y is the vector of observed values of the variable being predicted, with Y being the predicted values (e.g., from a least-squares fit), then the predictor's within-sample MSE is calculated as

$M S E=\frac{1}{n} \sum_{i=1}^n\left(Y_i-\hat{Y}_i\right)^2$                        (10)

Lower the value the better and 0 means the model is perfect.