Optimized Ensemble Learning Based on Genetic Algorithms and R-Shiny App for Early Detection of Preeclampsia

This study used secondary data sourced from the dataverse.harvard.edu website, which is openly licensed for unrestricted use. The dataset chosen for analysis consisted of replication data pertaining to the determinants of preeclampsia in women who delivered at county hospitals in Nairobi, Kenya [19]. A total of 352 postnatal ward mothers participated in the study, providing relevant information through structured interview questionnaires and the review of their medical records. The dataset was divided into two distinct categories: case of preeclampsia and normal, enabling the creation of sub-datasets for training and testing purposes. The proportions of these sub-datasets were adjusted to optimize the model’s performance and yield the most accurate results.

The stages in this research are seen as follows in Figure 1. The initial step is gathering pertinent medical data to provide the basis for further analysis. In the second stage, a Genetic Algorithm is introduced to find the optimized parameter and then apply this best parameter to the Random Forest algorithm. Inspired by biological evolution, Genetic Algorithms adjust the model’s parameters to adapt and perform better continuously. The third step involves predicting and achieving the accuracy of classification. The research culminates in developing an interactive R-Shiny application that offers healthcare practitioners an intuitive interface to input patient data and quickly and accurately assess the patient’s risk for preeclampsia.

1.png

Figure 1. Research stages

2.1 Random Forest

Random Forest represent a significant adaptation of the bagging technique, involving the creation of a diverse ensemble of uncorrelated trees, which are subsequently combined through averaging [20]. A random forest classifier is a meta-estimator that employs decision tree classifiers fitted on various subsets of the dataset. It uses averaging to enhance predictive accuracy and mitigate the risk of overfitting [21]. To mitigate the issue of overfitting, it is imperative to augment the number of decision trees; an elevated number of trees enhances predictive accuracy [22].

Like the name suggests, it is a Random Forest is a tree-based collection of trees, each one dependent on a random collection of variables. More formally, for a p-dimensional random vector $X=\left(X_1, X_2, \ldots, X_p\right)^T$ representing the real-valued input or predictor variables and a random variable $Y$ representing the real-valued response, we assume unknown joint distribution $P_{XY}(X,Y)$. It is the aim of finding the prediction function $f(X)$ for the prediction of that $Y$. The prediction function is derived using a loss function $L(Y,f(X))$ and is defined to reduce the cost of the loss,

$E_{X Y}(L(Y, f(X)))$               (1)

in which the subscripts indicate an expectation in relation to the distribution joint of $X$ and $Y$ [23].

The smallest $E_{X Y}(L(Y, f(X)))$ to minimize squared error loss provides the expectation of a conditional

$f(X)=E(Y \mid X=x)$                (2)

also referred to also regression function.

In the classification situation, if the set of possible values of $Y$ is denoted by $\LARGE y$, then minimizing $E_{X Y}(L(Y, f(X)))$ for zero-one loss gives,

$f(X)=\underset{\operatorname{y} \in {\LARGE y}}{\operatorname{argmax}} P(Y=\operatorname{y} \mid X=x)$               (3)

also referred to by or the Bayes rule.

In which there are $K$ classes that are denoted as $1,2...,K$ typically, a criteria for splitting is the Gini index,

$Q=\sum_{k \neq k^{\prime}}^K \hat{p}_k \hat{p}_k$               (4)

in which $\hat{p}_k$ is the proportion of $k$ class observations in the node:

$\hat{p}_k=\frac{1}{n} \sum_{i=1}^n I\left(y_i=k\right)$                (5)

Random Forest method is an extension of the Classification and Regression Trees (CART) method that incorporates the techniques of bootstrap aggregating (bagging) and random feature selection. In a Random Forest, many trees are grown to create a forest, and then an analysis is conducted on this ensemble of trees [24]. When applied to a dataset with $n$ observations and $p$ explanatory variables, the random forest algorithm operates similarly [25].

1. Bootstrap Sampling: Randomly select $n$ observations from the dataset with replacement, forming a bootstrap sample. This process is known as the bootstrap stage, where each sample is equally likely to be selected.
2. Tree Construction: Using the bootstrap sample, grow a decision tree to its maximum size without pruning. At each tree node, a random subset of m explanatory variables (where $m$ is much smaller than the total number of variables, $p$) is chosen. From this subset, the best variable for splitting is determined. This stage is referred to as the random feature selection stage.
3. Repeat the Process: Repeat steps 1 and 2 for $k$ iterations, creating a forest of $k$ trees. Each tree is built on a different bootstrap sample, resulting in a diverse ensemble of trees.

2.2 Genetic Algorithm

Genetic Algorithms are optimization algorithms that mimic the process of natural evolution [26]. Inspired by the principles of survival of the fittest, selection, and mutation, Genetic Algorithms simulate the evolution of solutions over multiple generations. The objective is to identify an optimal or close-to-optimal solution for a given optimization problem. The inherent flexibility of genetic algorithms, resembling the process of natural selection, enables these algorithms to modify themselves in response to diverse environments to ensure survival and procreation. A generation within the framework of a genetic algorithm constitutes a collective of entities, and each is engaged in the quest for a solution within the solution space. These entities may be depicted as sequences of binary digits, integers, and similar representations, with each sequence symbolizing a chromosome [27].

A key feature of Genetic Algorithms is their capacity to conduct a worldwide search within the hyperparameter space. In contrast to gradient-based optimization approaches that may settle at local optima, Genetic Algorithms employ principles inspired by natural selection, including selection, crossover, and mutation, to investigate a broad spectrum of solutions [28]. This attribute is especially advantageous for intricate models such as Random Forests, where the hyperparameter landscape may exhibit significant non-linearity and several modes. Although Genetic Algorithms can effectively explore the hyperparameter space, they may also be computationally demanding, particularly with large populations and multiple generations. This computing demand may restrict its usefulness in resource-constrained circumstances [29].

The initial population of potential solutions is determined randomly or through heuristic methods. Each solution is then assessed based on a predetermined fitness function. The individuals with the highest fitness are selected as the best candidates. These top individuals evolve and generate a new population through genetic operators like crossover and mutation. This process continues, evaluating and evolving the population until a termination condition based on the fitness function or the maximum number of generations is reached. Finally, the best individuals from the population are identified and presented as the solution to the optimization problem [30].

In this article the process of selection occurs following mutation and crossover. The fitness function $F$ can be utilized to determine the best individuals This is defined by the Eq. (6) [31].

$F_{i, j}=a c c_{i, j}-\gamma\left(L_e-L_s\right)$                  (6)

where, $a c c_{i, j}$ is the accuracy for the j-th individual of the generation of i-th obtained through test using the CNN model. The weight is $\gamma$, which represents the layers.

The fitness quality is the determining factor in determining whether one who has the highest fitness is chosen as the winner. This study follows the Eq. (7).

$P_{i, j}=\frac{F_{i, j}}{\sum_{j=1}^M F_{i, j}}$                (7)

where, $P_{i, j}$ is the likelihood that j-th person will survive.

The following outlines the procedure followed by the Genetic Algorithm (Figure 2).

2.png

Figure 2. The process used during Genetic Algorithm [32]

The selection of Genetic Algorithms parameters, including population size, number of generations, mutation rate, and crossover rate, is essential for the optimization process’s effectiveness. Here are a few factors to consider when selecting these parameters: 

1. Population Size: An increased population size can augment genetic variety and elevate the likelihood of identifying optimal solutions. Nonetheless, it also escalates computational expenses. It is customary to initiate with a moderate population size (e.g., 50-100 persons) and modify based on initial findings [33].
2. Number of Generations: The quantity of generations dictates the duration of the algorithm's execution. This parameter may be adjusted according to the convergence behavior noted in the first runs. Should the population exhibit considerable enhancement across generations, it could be advantageous to permit more generations [34].
3. Mutation and Crossover Rates: The mutation rate regulates the frequency of random alterations, while the crossover rate dictates the probability of merging solutions. The rates may be adjusted according to empirical findings, with typical initial values being a mutation rate of 0.01-0.1 and a crossover rate of 0.6-0.9 [35].

This research highlights an important aspect that contribute to improving the Random Forest model: parameter optimization using Genetic Algorithm. The application of Genetic Algorithm for parameter optimization in the Random Forest model has shown promising results. Genetic Algorithms offer a robust and efficient approach to searching the parameter space, identifying the optimal combination of parameters that maximize the model’s performance. The Random Forest model can be significantly improved in terms of predictive power and generalization capability by fine-tuning the parameters.

Furthermore, the development and utilization of an R-Shiny App for early detection of preeclampsia offer several significant benefits in maternal healthcare. This interactive and user-friendly application provides a streamlined and accessible platform for healthcare professionals, making identifying and assessing preeclampsia risk more efficient and accurate. With its intuitive interface, medical practitioners can input patient data and receive real-time predictions, enabling earlier detection and intervention, which is crucial for maternal and fetal well-being.

This research emphasizes accuracy as a performance indicator; however, we acknowledge that future endeavors could enhance the model and its application by integrating additional metrics to yield a more nuanced comprehension of the model’s efficacy in identifying preeclampsia risk.

Future studies should examine and contrast the model’s performance across various demographic groups to mitigate any biases and improve its applicability to diverse populations. This approach would facilitate an assessment of the model’s generalizability and guarantee uniform performance across diverse subpopulations. Incorporating these concerns in future studies will bolster the rigor of this research and augment the findings’ contribution to the field.