Leveraging Neighbourhood Component Analysis for Optimizing Multilayer Feed-Forward Neural Networks in Heart Disease Prediction

The heart, a pivotal organ in the human body, is responsible for pumping blood through an extensive network of vessels spanning approximately 60,000 miles. With an astounding 100,000 beats per day, it achieves forty million beats annually, culminating in nearly three billion beats over an average lifetime. The heart's functions are not limited to the oxygenation of the body and nutrient delivery to cells, but it also plays a crucial role in waste removal. Despite its significance, cardiovascular disease remains the leading cause of mortality globally, with the World Health Organisation (WHO) reporting that approximately 17.9 million people succumb to heart disease annually. As such, early prediction and intervention are critical in curtailing the mortality rate associated with heart disease.

Irregular heartbeats serve as a primary symptom of heart disease, necessitating the maintenance of various heart metrics, ranging from blood pressure to cholesterol levels. Although some of these metrics require professional evaluation, others can be monitored at home. A healthy heart offers resistance against major health issues and facilitates early detection of anomalies. Myocardial infarction, commonly known as a heart attack, is triggered when a region of the heart muscle is deprived of adequate blood supply. This is primarily attributed to coronary heart disease, although severe spasms or sudden contractions can also disrupt blood flow to the heart muscle.

Risk factors for heart disease often stem from detrimental behaviours, resulting in conditions such as obesity, hypertension, hyperglycaemia, and high cholesterol. Noteworthy symptoms of a heart attack in both men and women include chest discomfort, weakness, lightheadedness, fainting, pain or discomfort in one or both arms or shoulders, and shortness of breath. Additional symptoms may encompass unusual fatigue, nausea, or vomiting.

The exponential increase in medical data collection presents a formidable opportunity for medical practitioners to enhance their diagnostic capabilities. The remainder of this paper is structured as follows: Section 2 reviews related works, Section 3 details the proposed methodology, Section 4 engages in discussion, Section 5 presents an analysis of the results, and Section 6 offers a conclusion and outlines potential avenues for future research.

In this section, we emphasize the significance of widely used machine learning technique called Multilayer Feed Forward Neural Network. Also, all its variants and derivatives are discussed in brief.

3.1 Artificial neural network

Artificial Neural Network is a machine learning paradigm which is a “representation” and “reflection” of functioning of human brain. Neural Network is an art of constructing a computer program which learns from observational data. In 1958, Franklin Rosenblatt invented a simple binary classifier model called Perceptron. Perceptron is a building block of an artificial neural network. Perceptron is the most basic form of neural network comprised of single neuron. Multilayer Perceptron (MLP) is a generalized neural network model consists of four main parameters namely input values, weight and bias and activation function. Perceptron takes real values as its inputs.

3.2 Multilayer feed forward neural network

The fully connected artificial neural network is called multilayer feed forward neural network (MLFFN) [9]. Generalization of a single layer perceptron is known as multilayer perceptron (MLP) or multilayer feed forward neural network. MLFFN has multiple hidden nodes. In this type of network, all the input vector passes to the hidden layer and then to output layer. Each node in a single layer is connected to every other node in the next layer. All the internal calculation and extraction take place in the hidden layer.

The MLFFNN takes the input vector along with some random weights and bias and that representation is called as linear combination. Then, the linear combination or the weighed sum is passed to the transfer function or the sigmoid activation function. Sigmoid activation function has the ability to transfer the real valued data to lie between “0” and “1”. Sigmoid function has an advantage of predicting the probability of a binary data. After processing the linear combination, the output from the hidden layer passes to the output layer. Hence, MLFFN reconstruct each input vector into desired output vector with appropriate weight components.

3.3 Mathematical formulation MLFFNN

Let $X \subset[0,1]^m$ is a real valued input vectors whose components are defined as $\bar{x}=\left(x_1, x_2, \ldots x_m\right)$. Let $Y \subset$ $[0,1]^m$ is an output vectors whose components are defined as $\overline{o o}=\left(o o_1, o o_2, \ldots o o_m\right)$.

Mathematical formation of MLFFN [10] with single hidden layer is defined by function $\mathrm{N}: \mathrm{X} \rightarrow \mathrm{Y}$ such that it converts input vector into desired output vector over the subsequent composition.

Input to the $j^{\text {th }}$ hidden neuron $i h_j$ is given by the linear combination input vector $\bar{x}_l$ and weight vector.

i.e.,$i h_j=\sum_{i=1}^{m+1} w i h_{j i} x_i$        (1)

where, $\left(w{i h_{j 1}}, w i h_{j 2} \ldots w i h_{j m}, w i h_{j(m+1)}\right) \in R^{m+1}$ defines the weight vector between the input layer and $j^{\text {th }}$ hidden neuron for all $j$ and $x_{m+l}$ is bias term and is equal to one.

Output to j^th hidden neuron oh_j is computed by the activated function for studying non-linearity on the input vectors.

i.e., $\quad o h_j=\frac{1}{1+e^{-i h_j}}$         (2)

Input to the k_th hidden neuron io_k is represented by the linear combination of vector oh_j and weight vector.

i.e., $\quad i o_k=\sum_{j=1}^{h+1} w h o_{k j} o h_j$        (3)

where, $\left(w h o_{k 1}, w h o_{k 2}, \ldots w h o_{k h}, w h o_{k(h+1)}\right) \in R^{h+1}$  defines the weight vector the hidden layer and k^th output neuron for all k, h is number of hidden output neuron for all k, h is number of hidden neurons and oh_h+1 is bias term and is equal to one. Output to k^th output neuron oo_k is computed by the activated function for studying non-linearity on the input vectors.

i.e., $\quad o o_k=\frac{1}{1+e^{-i o_k}}$           (4)

It can be expressed as $N\left(\mathrm{x}^{\mathrm{P}}\right)=o o_k$, where, $o o_k$ represents the final output $k^{\text {th }}$ component.

1.png

Figure 1. Multilayer feed forward neural

Figure 1 shows the typical multilayer Feedforward neural network. Generally, in feedforward propagation all the input vector directly moves to hidden layers and then to the output layer without forming any loops or cycles inside the network. Finally, the output which comes out of the output layer is considered as a predicted value. Mostly, all predicted value may not be equal to the actual value. If the predicted value does not match the actual value, there is possibility of misclassification or loss function. In order to retrieve the error, we employ a most prominent machine learning algorithm called backpropagation algorithm. BPN is leading algorithm in reducing the error rate. It reduces the error rate by computing the derivate of the loss function with respect to weight components in the input layer. weights are adjusted after one cycle. This process is Feedback propagation.

3.4 Back propagation

In 1974, Werbos and in 1985, Parker introduced an algorithm called backpropagation (BPN) algorithm. And in 1986, it was further developed by Rumelhart [11]. BPN is a machine learning algorithm used to minimize the loss function while training a multilayer feed forward neural network. It is a process of updating all the weight components with respect to the error rate. It is an algorithm which backpropagates all the error from output node to hidden node and then to input node.

Backpropagation works under these following steps.

• Each input vector passes through per connected path.
• Initially weights are chosen randomly so probably error occurs.
• To minimize the error rate in the predicted output, we apply BPN mechanism.
• BPN calculates the gradient of the loss function via chain rule.
• It backpropagates the loss function from output node to input node.
• BPN is a supervised machine learning algorithm so its aims to get the predicted value closer to the desired value by adjusting the weight components.
• Weight components are then adjusted iteratively until the model reach the global minimum loss.

3.5 Gradient descent and its variants

In 1847 “Augustin -Louis Cauchy” discovered an algorithm called Gradient descent algorithm. It is popularly known as steepest descent. It is an iterative optimization method which minimize the cost function for finding the local minimum of a function. It is the first order optimization algorithm used to train neural network model. We use batch gradient descent algorithm train our MLFFFNN. This algorithm takes each training data into account in every iteration. It takes the average of gradient of overall training samples and the mean gradient is used to update. The model updates its weight parameter after every epoch.

3.5.1 Learning rate

Learning rate is the hyper parameter which controls the weight with respect to the loss function. A smaller learning rate needs additional training epoch. Therefore, learning rate basically plays a significant role in controlling the weight parameter. Any smaller changes made to the weight in each iteration which makes the model converge slowly to the global minimum loss.

3.5.2 Error correction

Error is the measure of loss function between the actual value and predicted value. It basically affects the performance of the model and it can be calculated by the following formula:

$E(\bar{W})=\frac{1}{2} \sum_{p=1}^p \sum_{k=1}^n\left(e_k^p\right)^2$       (5)

where, $e_k^p=d_k^p-o o_k^p, d_k^p$=actual output, $o o_k^p$=predicted output.

The performance of ANN was constructed by Multilayer Feed Forward neural network. In MLFFN, any number of hidden layers can be added as per requirement. In the proposed model, only one hidden layer with nine neurons is used in MLFFN for which the MLFFN contributes the best results. Based on the proposed method, 75% of 303 data are used for training and the remaining 25% are used for testing. This model was trained under the combination of nearest neighbour algorithm-based multilayer perceptron. To determine the efficiency of the classification model with the best features selection we split the dataset into two distinct divisions. Firstly, the training set comprised 227 instances with 13 featured samples and one output label and the testing dataset consist of 76 instances. The decomposition of a 303×13 matrix is not determined randomly. Preferably, pretty proper data are collected for training the model by the NCA method. Ultimately, the quality and efficiency of the model will be higher. Each point in a training set is considered as its neighbour but with contrasting probabilities. The intersection of two point is treated to be zero.

The main goal of proposing a clustering algorithm is considering its optimal neighbour is to collect all possible numbers of data points to put in a training set. Instead of taking all the training data points in a random ratio here, our proposed methodology takes the data points which has the greatest impact as a result it improves the efficiency of the model.

Figure 5 and Figure 6 show the individual contribution of the component such as age to predict the presence of heart disease through the proposed NCA-MLFFN technique.

In Figure 5, the attribute age is labelled on the x-axis and predicted values are labelled on y-axis. The proposed model plotted the healthy patient at the bottom and the heart disease patients at the top of the figure. Also, Figure 6 shows the performance of the developed NCA-MLFFN model classifying heart disease with its minimum loss function is shown.

Cleveland heart disease dataset is highly non-linear in nature as shown in Figure 2. Many researchers [4] have tried to classify Cleveland heart disease dataset. Even though the paper [4] had find the important risk factors on the p-value which gives the necessary codes for every attribute, accuracy produced by is 84% whereas proposed model produces the accuracy 97%. In the study [5], although measures the distance metric between the data point, the proposed model learning vector quantization produced its best accuracy of about 80% in the Learning Vector Quantization Algorithm, proposed model nearest neighbourhood is used to make cluster to find the groups of data based on the similarity of the features which produce the accuracy of 97%. In the study [6], the number of neurons in the hidden layer can be increased or decreased based on the error obtained. The performance of the system was improved by varying the epochs and the number of neurons. Eventually, the model in the study [6] obtained an accuracy of about 85.55% and Florence et al. [14], initially used three neurons in the hidden layer and it can be increased up to thirteen neurons whereas the proposed model uses nine fixed hidden neurons based on the method of the density of the learning data.

5.png

Figure 5. Distribution of age and prediction

6.png

Figure 6. Distribution of age and error

The proposed model, which employs a nearest neighborhood search for training a multilayer feed forward neural network, demonstrates superior accuracy compared to other methods. Initial efforts were undertaken to pre-process the dataset, mitigating the risk of overfitting during the model's training phase. The proposed approach leverages a Neighbourhood selection technique to select optimal features for learning, thereby enhancing the model's generalization. The selected features were subsequently trained using a multilayer feedforward neural network (MLFFN).

The MLFFN was designed with a single hidden layer equipped with nine neurons. The model's efficiency was determined by the error rate obtained using the backpropagation algorithm. It is noteworthy that approximately 75% of the data was allocated for learning, while the remaining 25% was utilized for testing the model.

The model's performance was evaluated using various metrics such as Sensitivity, Specificity, and Precision, validated using a confusion matrix. The proposed algorithm ultimately achieved an impressive accuracy of approximately 96.03%.

In future works, the proposed model's performance may be enhanced through precise calibration of parameters for the same Cleveland heart disease data or by employing various distinct methodologies. The implementation of these modifications could potentially bolster the predictive power and overall performance of the model. Hence, the research opens avenues for further exploration and refinement of this model.