Constructing and Optimizing an Evaluation Model for the Implementation of Electronic Voting: An Indonesian Case Study

Electronic Voting (E-Voting) is an electoral process facilitated through electronic means. The numerous benefits of E-Voting, such as enhanced computational speed and accuracy, resource preservation, and cost reduction, have been documented [1-3]. Consequently, progressive nations are contemplating the transition from traditional to electronic voting systems in their general elections. The year 2024 marks a pivotal point in Indonesia's political landscape, with the execution of elections that are crucial for the advancement of democracy in the country. From the inception of the voting process in 1955, up until the thirteenth election in 2024, Indonesia has predominantly relied on traditional voting systems. Although certain aspects, like the counting system, have been technologically augmented, a comprehensive exploration of the influences impacting the successful implementation of E-Voting in Indonesia remains necessary. The anticipated benefits of E-Voting in Indonesia include cost reduction in electoral processes, increased public participation, and improved speed and accuracy in generating election results.

Success stories of E-Voting integration in electoral processes have emerged from countries like Canada, Brazil, India, Estonia, and the Philippines. Out of the many countries that have tried to implement E-Voting, only five of them have achieved consistent success [4]. The failures in other countries can be attributed to various factors such as technological inadequacies [1], security concerns, readiness issues, and lack of public trust [2, 5], despite extensive research on E-Voting over the years [6]. According to the International IDEA's ICTs in Elections Database, out of the 178 studied countries [7], 34 countries have implemented E-Voting systems, either nationally or locally. Nevertheless, numerous countries have struggled with the implementation, primarily due to factors such as lack of voter confidence in technology, as witnessed in Ireland, Germany, and the United States. Data security has also been a significant impediment to the successful application of E-Voting, as observed in the Netherlands.

Given the vast discrepancy between the number of successful and unsuccessful attempts at E-Voting implementation in general elections worldwide, it is reasonable to hypothesize that technology is not the primary determinant of successful E-Voting implementation. This study, therefore, aims to identify the factors contributing to the successful implementation of E-Voting in general elections, how these factors can be adequately prepared by the state to ensure smooth and sustainable E-Voting processes, and how these factors can be leveraged to develop an effective E-Voting framework.

3.1 E-Voting success factors

In previous research show that, at least 17 factors that influenced the successful implementation of E-Voting as shown in Table 1. We conclude that these factors can be grouped into 3 main factors, 1) Public Perceptions, 2). Readiness Aspect and 3) Technological Aspect.

In this study, the public perception and readiness aspect was carried out using quantitative research, while the technological aspect was carried out using a technological approach, which will be carried out in further research, The form of the E-Voting success factor mapping is illustrated in Figure 2.

The factors contained in the quantitative research border include public perception consisting of Trust in Government, Trust in Commissions, and Trust in Technology, as well as readiness consisting of the Readiness of Technology, Readiness of Human Resources, and Readiness of the Constitution to become the proposed E-Voting Success Factors.

Table 1. E-Voting success factors

2.png

Figure 2. E-Voting success factors mapping

3.2 Hypothesis and data collection

From the E-Voting Success Factors mapping as shown in Figure 2, a hypotheses frame was built as follows:

1. There is a significant effect of trust in the government on the successful implementation of E-Voting in general election [21, 22].
2. There is a significant effect of trust in the general election commission on the successful implementation of E-Voting in general election [22-24].
3. There is a significant effect of trust in technology on the successful implementation of E-Voting in general election [6, 7, 25, 26].
4. There is a significant effect of the technology infrastructure readiness on the successful implementation of E-Voting in general election [18, 27].
5. There is a significant effect of the human resource readiness on the successful implementation of E-Voting in general election [18, 21].
6. There is a significant effect of the constitution readiness on the successful implementation of E-Voting in general election [24, 26].
7. There is a significant effect of the public perception aspect and the readiness aspect on the successful implementation of E-Voting in general election.

From this hypothesis, 19 questionnaire questions were made to test the validity of each item and the reliability of each variable. The questionnaire build using Likert Scale consist of 5 available choices, 1) Strongly Disagree, 2) Disagree, 3) Neutral, 4) Agree and 5) Strongly Agree. Based on the results of the validity test using Pearson Moment with a significance level of 5% on the perceptions of 30 initial respondents, there was one question that was invalid with a r value less than 0.463, so it had to be excluded from the variable. Invalid questions come from the Trust in Technology variable, and then the remaining 18 questions in the questionnaire are used in the latent variable reliability test. The results of the reliability test showed that each variable met the Cronbach alpha value more than 0.6, with the lowest value is 0.623 and the highest value of 0.858. When the questionnaire is said to be valid and reliable, the next step is to collect data on the respondents.

In this research, respondents used were a sample of 11,632,816 voters in the province of West Java, Indonesia from the Indonesian General Elections Commission (KPU) in 2020. The minimum sample size was determined with an error margin of 5% using 4 formulas/ Table, 1) Taro Yamane, 2) Slovin, 3) Issac Table, and 4) Kreijcie and Morgan Table as state in [28-31].

Data collection was carried out from November 19 to December 4, 2021, using the google form and generated 415 feedbacks. This exceeded the minimum sample requirements according to Table 1 above, which means that the data from the questionnaire results are usable. From the 415 data points collected, 12 were incomplete, so the remaining 403 data points could be further analyzed.

3.3 Data analysis

Based on data collection, the 403 complete data sets have a distribution of respondent profiles in the form of age, education, and occupation, as presented in Table 2.

The next stage was the evaluation of the SEM-PLS model using SmartPLS 3.3.3, SEM-PLS is a non-parametric statistical method for analyzing complex relationships between latent variables. In SEM_PLS theory, the latent variable is a variable that cannot be measured directly, so it requires a manifest variable. Latent variables consist of endogenous variables (which is influenced) and exogenous variable (which influences). clearly presented in the questionnaire question items.

Table 2. Respondent data distribution

In this study, there are 7 latent variables, namely Trust in Government (TGV), Trust in Commission (TCM) and Trust in Technology (TTG), Readiness of Technology (RTI), Readiness of Human Resources (RHR) and Readiness of Constitution (RCS) and Success E-Voting Implementation (SEI). Each latent variable is explained by 2 or 3 manifest variables; so, the number of manifest variables is 19 items

Based on the research hypotheses, 7 possible SEM-PLS evaluation models were obtained. Each SEM-PLS model was evaluated; the results show that in 1 of 7 SEM-PLS models, there is a new phenomenon that strengthens the Readiness of E-Voting as presented in Figure 3 used to evaluate the successful E-Voting Implementation.

The SEM-PLS models were evaluated for the measurement model and structural model based on the results of the PLS-PLS-Algorithm, Bootstrapping, and Blindfolding procedures. The measurement model describes the relationship between the manifest variables and the latent variables, while the structural model describes the relationship between the latent variables in the model.

3.png

Figure 3. SEM-PLS model

Evaluation of the measurement models was carried out with the following objectives:

1. The evaluation ensures that the measurement model is valid and reliable based on three indicators, namely Internal Consistency, Convergent Validity, and Discriminant Validity. Internal Consistency Indicators are Cronbach Alpha and Composite Reliability values, which are≥0.6 and≤0.95. This is to avoid all variables and indicators measure the same phenomenon. Besides, the Average Variance Extracted (AVE) value should be>0.5. Convergent Validity value is determined based on Outer Loading and AVE. The significant Outer Loading value is>0.7.
2. Based on the measurement results in Table 3, the Cronbach's Alpha, Composite Reliability and AVE values have met the requirements

3. The Discriminant Validity criteria were determined based on the Cross Loading and Fornell-Larcker Criterion. Through the Cross Loading table, the Outer Loading of each indicator on a latent variable must be greater than the value of Cross Loading on other latent variables. Based on the results of the measurement for Cross Loading and Fornell-Larcker Criterion, the Cross Loading value has met the requirements.
4. Values and the Fornell-Larcker Criterion were used by comparing the square root of the AVE that is greater than the correlation value of the latent variables. Based on the measurement results in Table 4, some results have met the requirements.

After evaluating the measurement model, the structural model was evaluated to determine a model's predictive ability and the relationships between latent variables in the model. Therefore, the following steps were taken.

1. Collinearity test. The collinearity test aims to see that there is no collinearity between exogenous latent variables (influence/independent variables) against the same endogenous latent variables (influenced/dependent variables) in the structural model formed. Based on the measurement results in Table 5 below, the value of inner VIF < 5; thus, it can be concluded that there is no collinearity.
2. Evaluation of Path Coefficient. Path coefficient value shows the estimated value of the strength of the relationship between the structural model's latent variables. The t-value and p-value of each path indicate whether the path coefficient is significant. The Readiness of Constitution variable is not significant to Trust in Government based on the path coefficient measurement in Table 6 below since the p-value is 0.586>0.05, while other variables/paths are significant since the p-value is 0.05.
3. Evaluation of the coefficient of determination in Table 7. The coefficient of determination R2 shows the predictive power of a model. This value represents the strength of the exogenous variables that simultaneously affect the endogenous latent variables.
4. Evaluation of f2 and Q2 values. The value of f2 indicates the effect on the endogenous variable if an exogenous variable is removed from the model. The bottom limits of f2 values are 0.02, 0.15, and 0.35. The value of Q2 is used to evaluate the accuracy of the adjusted model. The value of Q2 was determined by using a cross-validated redundancy approach. If Q2>0, it means that the model used to predict endogenous constructs is appropriate. The values of Q2 and f2 were obtained by blindfolding the model.
5. In Table 8, the f2 value indicates that the constitution's readiness does not influence the trust in government variable If it is excluded from the model. In Table 9, the Q2 value>0 for the two endogenous variables shows that the applicable model predicts the two endogenous variables.

Table 3. Internal consistency and AVE

Table 4. Fornell-Larcker criterion

Table 5. Inner VIF

Table 6. Path coefficient

Table 7. Coefficient of determination

Table 8. F2 score

Table 9. Q² score

3.4 Results

Based on the data analysis and measurements, the results are as presented in Table 10, and Figure 4 below shows an illustration of the new success factor of E-Voting.

The results of quantitative research show that there is a new phenomenon, namely, a change in the proposed E-Voting success factors, which strengthens the factors that influence the success of E-Voting.

Table 10. Data analysis result

4.png

Figure 4. Success factors of E-Voting