Leveraging Support Vector Machine for Predictive Analysis of Earned Value Performance Indicators in Iraq's Oil Projects

The project of Karbala Refinery is selected as a case study to achieve the goal of the research being one of the huge projects, project Location is 25 km South of Karbala City, Iraq (100 km South of Baghdad City). The total site area is (10 km²) including the Refinery area is (6 km²). The total cost of this project is about (USD 6,641,089,012), with a working duration of (54 months). More information is summarized in Table 1. Figure 2 displays the units’ diagram for the Karbala Refinery project as well as Figure 3 shows a 3D picture and the site photo of the Crude & Vacuum Distillation Unit.

It is necessary for SVM models to be organized, so as to improve the performance. The main factors to be addressed are developing model input, data divisions, and preprocessing, developing the model architecture, and its optimization (training), stop criteria, and model validation. A structured methodology is applied to develop the model, which involves four major phases:

1. Data division
2. Model architecture
3. SVM model equation
4. SVM model validity

The same variables defined at the data identifying step are used for developing the three mathematical SVM models, using the project characteristics to predict the EV indexes. Through the third version of Weka, the following models were developed:

• SPI: is the schedule performance indicator
• CPI: is the cost performance indicator
• TCPI: is the to-complete cost performance indicator

8.1 Cost performance index (CPI) model

This model's development involves the following five steps:

8.1.1 Data division CPI model

The data is divided into three sets, namely training, testing, and validating sets as shown in Table 3.

Table 3. Impact of data split on performance of CPI model

8.1.2 Selection kernel CPI model

The next step is to choose the kernel, as displayed in Table 4. The poly-kernel was chosen as the optimal kernel for the CPI model because its root mean square error (RMSE) is equal to 0.1, which is the least number found.

Table 4. Effects of the kernel function on CPI model

8.1.3 Selection of the SVM parameters (C and epsilon) for the CPI model

Table 5 illustrates an example of the C effect on the CPI model. When the C value is 10 the greatest (r) value is (96.98%), the mean absolute error (MAE) is (0.0354), and the least RMSE is (0.0786) making this the ideal value. According to the statistics in this table, changes in C, especially those falling between (1 to 10), have little impact on the performance of the CPI models. This supports including it in the study's suggested model.

Table 5. Impact of changing parameter C on CPI model performance

Table 6 displays Epsilon's impact on the CPI model. Considering that the greatest (r) value is (87.06%), the smallest RMSE is (0.0744), and the mean absolute error (MAE) is (0.0528) Epsilon is considered to be at its best when it is valued equal to (0.006). The information in this table demonstrates that variations in Epsilon have little impact on the functionality of CPI models, especially when they fall between (0.001 to 0.05). This is in favor of including it in the study's model.

Table 6. Impact of changing parameter Epsilon on CPI model performance

8.1.4 Equation of CPI model

Table 7 shows the connection weights collected by the Weka software for the optimal CPI model. A scale is not necessary because the program decides whether or if the data should be transformed as well as the method of transformation.

Table 7. Weight estimates for model CPI

By using the connection weights and threshold level stated in Table 6, the CPI value might be predicted in the following method:

$\mathrm{CPI}_{\text {nor }}=0.5589 -(0.2287 \times \mathrm{P} \%) -(0.2287 \times \mathrm{BCWS}) +(1.5477 \times \mathrm{A} \%) +(1.5477 \times \mathrm{BCWP}) -(3.2236 \times \mathrm{ACWP})$                      (2)

$\mathrm{CPI}_{\text {act }}=\mathrm{CPI}_{\text {nor }} \times \mathrm{range}+\mathrm{min}$                    (3)

$\mathrm{CPI}_{\text {act }}=\mathrm{CPI}_{\text {nor }} \times 0.5650+1.0340$                    (4)

The above-mentioned equation's implementation can be clarified by utilizing the data used in the SVM model training for CPI, as shown in report no.66 in Table 8. The predicted value obtained from the above equation equals (1.071), which, when compared to the real value measured by hand (CPI=1.064), is relatively accurate. These value variations are considered as minor. Before utilizing Eq. (2) it should be noted that all input variables must be transformed to values between (0-1) because Eq. (2) was built using Eq. (1). To obtain actual data out from normalized ones, conversions to actual values were made using Eq. (4) and Table 3.

Table 8. Verification of CPI model

8.1.5 Verification of the CPI model

Table 8 summarizes and compares the CPI computation using SVM to validate the estimation model. It comprises the real CPI value received from the Karbala Refinery Project, as well as the estimated CPI value calculated using the SVM equation (as obtained from Weka V.3).

4.jpg

Figure 4. Comparison of predicted and actual for CPI model

Figure 4 illustrates the predicted values against the actual values for the verification data to show the capacity of the SVM model for CPI to assess the model. It is clear from this figure that the (R2=86.1%). The ten spare data that have not yet been used in any subset can have their earned value index predicted using the trained SVM models. The generalization results of the CPI model with (R²=82.03%) are excellent, as illustrated in Figure 5.

5.jpg

Figure 5. Generalization of CPI model

8.2 Schedule performance index (SPI) model

This model's development involves the following five steps:

8.2.1 Data division SPI model

The data is divided into three sets, namely training, testing, and validating sets, as show in Table 9.

Table 9. Impact of data split on performance of SPI model

8.2.2 Selection kernel SPI model

The next step is to choose the kernel, as displayed in Table 10. The poly-kernel was chosen as the optimal kernel for the SPI model because its root mean square error (RMSE) is equal to (0. 0788), which is the least number found.

Table 10. Effects of the kernel function on SPI model

8.2.3 Selection of the SVM parameters (C and epsilon) for the SPI model

Table 11 illustrates an example of the C effect on the SPI model. When the C value is 6 the greatest (r) value is (96.98%), the MAE is (0.0354), and the least RMSE is (0.0786) making this the ideal value. According to the statistics in this table, changes in C, especially those falling between (1 to 10), have little impact on the performance of the SPI models. This supports including it in the study's suggested model.

Table 11. Effect of the parameter C in SVM model performance

Table 12 displays Epsilon's impact on the SPI model. Considering that the greatest (r) value is (97.35%), the smallest RMSE is (0.0717), and MAE is (0.0478), Epsilon is considered to be at its best when it is valued equal to (0.006). The information in this table demonstrates that variations in Epsilon have little impact on the functionality of SPI models, especially when they fall between (0.001 to 0.05). This is in favor of including it in the study's model.

Table 12. Impact of changing parameter Epsilon on SPI model performance

8.2.4 Equation of SPI model

The SPI value might be estimated using the connection weights and threshold level shown in Table 13 as follows:

Table 13. Weight estimates for model SPI

By using the connection weights and threshold level shown in Table 12, the SPI value could be predicted in the following method:

$\mathrm{SPI}_{\mathrm{nor}}=0.2858-(0.3011 \times \mathrm{P} \%)-(0.3011 \times \mathrm{BCWS}) +(0.7557 \times \mathrm{A} \%) +(0.7557 \times \mathrm{BCWP})(0.2038 \mathrm{ACWP})$                      (5)

$\mathrm{SPI}_{\mathrm{act}}=\mathrm{SPI}_{\text {nor }} \times \mathrm{range} +\mathrm{min}$                  (6)

Table 14. Verification of SPI model

The above-mentioned equation's implementation can be clarified by utilizing the data used in the SVM model training for SPI, as illustrated in report no. 56 in Table 14. The predicted value obtained from the above equation equals (0.954), which, when compared to the real value measured by hand (SPI=1.034), is relatively accurate. These value variations are considered as minor. Before utilizing Eq. (5), it should be noted that all input variables must be transformed to values between (0-1) because Eq. (5) was built using Eq. (1). To obtain actual data out from normalized ones, conversions to actual values were made using Eq. (7) and Table 9.

6.jpg

Figure 6. Comparison of predicted and actual for SPI model

8.2.5 Verification of the SPI model

Table 14 summarizes and compares the SPI computation using SVM to validate the estimation model. It comprises the real SPI value received from the Karbala Refinery Project as well as the estimated SPI value calculated using the SVM equation (as obtained from Weka V.3).

Figure 6 illustrates the predicted values against the actual values for the verification data to show the capacity of the SVM model for SPI to assess the model. It is clear from this figure that the (R²=96.5%). Figure 7 shows the generalization results for the SPI model with it can be said are excellent.

7.jpg

Figure 7. Generalization of SPI model

8.3 To complete cost performance indicator (TCPI) model

This model's development involves the following five steps:

8.3.1 Data division TCPI model

The data is divided into three sets, namely training, testing, and validating sets, similar to the previous network models as illustrated in Table 15.

Table 15. Impact of data split on performance of SPI model

8.3.2 Selection kernel TCPI model

The next step is to choose the kernel, as displayed in Table 16. The poly-kernel was chosen as the optimal kernel for the TCPI model because its RMSE is equal to (0.081), which is the least number found.

Table 16. Effects of the kernel function on TCPI model

8.3.3 Selection of the SVM parameters (C and epsilon) for the TCPI model

Table 17 illustrates an example of the C effect on the TCPI model. When the C value is 1 the greatest (r) value is (94.12%), the Mean Absolute Error (MAE) is (0.0422), and the least RMSE is (0.081), making this the ideal value. According to the statistics in this table, changes in C, especially those falling between (1 to 10), have little impact on the performance of the TCPI models. This supports including it in the study's suggested model.

Table 18 displays Epsilon's impact on the TCPI model. Considering that the greatest (r) value is (94.26%), the smallest RMSE is (0.0778), and the Mean Absolute Error (MAE) is (0.0489), Epsilon is considered to be at its best when it is valued equal to (0.04). The information in this table demonstrates that variations in Epsilon have little impact on the functionality of TCPI models, especially when they fall between (0.001 to 0.05). This is in favor of including it in the study's model.

8.3.4 Equation of TCPI model

The TCPI value might be estimated using the connection weights and threshold level shown in Table 19.

By using the connection weights and threshold level stated in Table 18, the value of TCPI might be forecasted as follows:

$\mathrm{TCPI}_{\text {nor }}=0.2858 +(0.0306 \times \mathrm{P} \%) +(0.0306 \times \mathrm{BCWS}) -(0.2616 \times \mathrm{BCWP}) -(0.6319 \times \mathrm{ACWP})$                        (8)

$\mathrm{TCPI}_{\mathrm{act}}=\mathrm{TCPI}_{\text {nor }} \times \mathrm{range} +\mathrm{min}$                    (9)

$\mathrm{TCPI}_{\mathrm{act}}=\mathrm{TCPI}_{\text {nor }} \times 0.8020+0.1960$                    (10)

The above-mentioned equation's implementation can be clarified by utilizing the data used in the SVM model training for TCPI, as shown in report no. 66 in Table 20. The predicted value obtained from the above equation equals (0.440), which, when compared to the real value measured by hand (TCPI=0.591), is relatively accurate. These value variations are considered as minor. Before utilizing Eq. (8), it should be noted that all input variables must be transformed to values between (0-1) because Eq. (8) was built using Eq. (1). To obtain actual data out from normalized ones, conversions to actual values were made using Eq. (10) and Table 15.

8.3.5 Verification of the TCPI model

Table 20 summarizes and compares the TCPI computation using SVM to validate the estimation model. It comprises the real TCPI value received from the Karbala Refinery Project, as well as the estimated TCPI value calculated using the SVM equation (as obtained from Weka V.3).

Table 17. Impact of changing parameter C on TCPI model performance

Table 18. Impact of changing parameter Epsilon on TCPI model performance

Table 19. Weight estimates for model TCPI

Table 20. Verification of TCPI model