A Novel Approach for the Detection of Cardiovascular Abnormalities from Electrocardiogram and Phonocardiogram Signals Using Combined CNN-LSTM Techniques

The primary objective of this study is to develop a robust methodology for detecting cardiovascular abnormalities by analyzing ECG and PCG signals. Integrating CNN with LSTM networks aims to enhance the accuracy and reliability of abnormality detection, thus contributing to early diagnosis and intervention for cardiovascular diseases.

Methodology:

Data collection involves acquiring high-quality ECG and PCG datasets from patients presenting a variety of cardiovascular conditions, encompassing arrhythmias, heart murmurs, and valve disorders. These datasets undergo preprocessing steps to ensure data integrity, including noise removal, baseline wander correction, and normalization. Following preprocessing, CNNs are employed to automatically extract spatial features from the ECG and PCG signals. These features encapsulate crucial patterns and characteristics indicative of cardiovascular abnormalities. Subsequently, LSTM networks are utilized to model the temporal dependencies within the extracted features. LSTM cells are adept at capturing sequential patterns and long-term dependencies in time-series data. By fusing the output representations from the CNN and LSTM models, we aim to leverage the complementary strengths of spatial and temporal modeling techniques for more accurate detection of cardiovascular abnormalities.

This combined CNN-LSTM approach offers a novel methodology for analyzing ECG and PCG signals, providing insights into both spatial and temporal aspects of cardiovascular dynamics. Through rigorous evaluation and validation of diverse datasets, we anticipate that this approach will contribute to advancements in the early diagnosis and management of cardiovascular diseases, ultimately improving patient outcomes and healthcare delivery.

Comparison with Existing Methods:

Previous studies in cardiovascular abnormality detection have often utilized either CNNs or LSTMs individually for signal analysis. While CNNs excel at capturing spatial features from signals, LSTMs are effective in modeling temporal dependencies. However, few studies have explored the combined use of these architectures for cardiovascular abnormality detection from ECG and PCG signals.

Existing Approaches:

Some existing methods rely solely on CNNs for feature extraction from ECG and PCG signals. While these methods can effectively capture spatial patterns, they may overlook important temporal dynamics crucial for accurate abnormality detection [22]. On the other hand, approaches utilizing only LSTMs focus primarily on modeling temporal dependencies but may not fully exploit the spatial information present in the signals.

Proposed Model Advancements:

The proposed model represents a significant advancement over existing methods by synergistically integrating both CNNs and LSTMs. By combining the strengths of these two architectures, our model can effectively capture both spatial and temporal aspects of cardiovascular signals, leading to enhanced detection performance.

(1). Comprehensive Feature Extraction: Unlike previous methods that rely solely on CNNs or LSTMs, our model leverages CNNs for spatial feature extraction and LSTMs for temporal modeling. This comprehensive approach ensures that both spatial and temporal aspects of the signals are adequately captured, leading to a more robust representation of cardiovascular dynamics.

(2). Optimized Fusion Strategy: The proposed model incorporates an optimized fusion strategy to combine the output representations from the CNN and LSTM layers. By carefully integrating spatial and temporal information at an appropriate level, our model avoids redundancy and ensures that complementary information from both architectures is effectively utilized for abnormality detection.

(3). Enhanced Performance: Through rigorous evaluation of diverse datasets, our model demonstrates superior performance compared to existing methods. By effectively capturing both spatial and temporal features, our model achieves higher accuracy, sensitivity, and specificity in detecting cardiovascular abnormalities, thus improving early diagnosis and patient outcomes.

In summary, the proposed model represents a significant advancement in cardiovascular abnormality detection by synergistically combining CNNs and LSTMs. Through comprehensive feature extraction and optimized fusion strategies, our model outperforms existing methods and offers promising prospects for improving the diagnosis and management of cardiovascular diseases. The research design includes the process of collecting dataset, processing, model development, training, testing, and evaluation of the performance of the developed model and comparing the performance of the developed model with that of the existing models in terms of the accuracy of the model in classification.

3.1 Datasets

The datasets used here for the research are the heart sound recordings and ECG signals of individuals adapted from Liu et al. [23] and Clifford et al. [24] respectively. The datasets are popularly termed Physionet 2016 and Physionet 2017 respectively.

Python was used as the programming language for developing the model. It is mainly used by researchers in projects and works who use it for web development, data analysis, regression techniques, and more. For the feature extraction purposes in this research the "Librosa" as the library is used since current research is based on audio-analysis. Librosa extracts audio features from files by focusing on spectral contrast, tempo, Mel-frequency cepstral-coefficients, spectral roll-off, spectral centroid, and zero-crossing rate [25]. By focusing on these features, the samples or targets gained by the researchers from audio file extraction in a study are used and analyzed further. The samples in the Librosa use the standard sampling rate of 22050 which also can be overridden according to the investigators' desired outcome or sampling rate.

In this research, the sampling rate is set at 22050 and not overridden. The samples are calculated every second and the rates are recorded (sampling rate). In the medical field research and investigation to plot the data analyzed, the researchers use the ‘spectrogram’ and ‘scalogram’. In this research, the ECG and PCG wavelet transformations of the audio process are measured using the "scalogram". It returns a signal's continuous wavelet-transform (CWT) coefficient absolute value. The noise reduction and signal transformation are much more efficient in scalograms for analyzing ECG and PCGs [26].

Scalograms derived from the CWT offer a powerful means of extracting informative features from cardiovascular signals like the ECG and PCG. By applying the CWT to the raw signals, scalograms provide a comprehensive representation of how signal energy is distributed across both time and frequency domains. This representation allows for a detailed examination of the temporal evolution of frequency components within the signals.

One advantage of using scalograms for feature extraction is their ability to capture transient phenomena and dynamic changes in signal characteristics. The time-frequency localization provided by scalograms enables the identification of specific frequency components that may correspond to important cardiovascular events, such as heart murmurs or arrhythmias. This temporal resolution is crucial for accurately characterizing the dynamic nature of cardiovascular signals and detecting abnormalities that may manifest as transient deviations from normal patterns.

Furthermore, scalograms offer a rich source of features that can be extracted to quantify various aspects of the signal's time-frequency characteristics. Features such as energy distribution across different frequency bands, dominant frequency components, and spectral entropy can provide valuable insights into the underlying physiological processes. Additionally, statistical measures derived from scalograms, such as mean, variance, and skewness of energy distribution, can capture the overall shape and complexity of the time-frequency representation.

By leveraging these features extracted from scalograms, researchers can develop robust classifiers and diagnostic algorithms for detecting cardiovascular abnormalities. The combination of time and frequency information provided by scalograms enhances the discriminative power of the extracted features, enabling more accurate and reliable detection of abnormalities in ECG and PCG signals. Overall, scalograms offer a versatile and effective approach for feature extraction in cardiovascular signal analysis, contributing to advancements in early diagnosis and treatment of cardiovascular diseases.

3.2 Data acquisition

The acquisition of datasets in medical research is generally done either by collecting primary datasets through direct face-to-face interactions with the patients or participants or as secondary datasets relevant to the study through existing resources like journals, data or libraries, articles, studies, research, and other resources.

In this research ‘Physio Net’ which is a medical research-data repository has been utilized, where, the data are in its raw form. The datasets from 2016 and 2017 (i.e., 2 consecutive years) had been utilized for consistent analysis and results to be obtained from the model. The dataset is obtained from resources conducted by the investigators [23, 24] where the PCG signals and ECG signals are obtained and classified accordingly. The 2016-based Physio-net dataset i.e., ‘Training b-f’ contains the raw PCG sound waves, and ‘Training a contains the synchronized PCG and ECG signals. The 2017 Physio-net-based dataset includes the ECG signal. Thus, the datasets are obtained and used. The datasets acquired are balanced where the PCG and ECG waveforms are in a total of 544 samples.

3.3 Pre-processing

Pre-processing is a technique where the datasets are modified according to the researchers' necessity majorly based on their size, contrast, brightness, pixelization, and shape. There are various pre-processing techniques namely Short-time Fourier-transform (STFT), Syncrosqueezed-wavelet transform (SWT), empirical-mode decomposition (EMD), extended EMD (EEMD), and more [27, 28].

Pre-processing steps are crucial for ensuring the quality and integrity of the data before feeding it into the neural network model. Here's a detailed outline of the pre-processing steps for the ECG and PCG signals: ECG and PCG signals are often contaminated with various types of noise, including baseline wander, electrode artifacts, and environmental interference. Signal denoising techniques such as median filtering, wavelet denoising, or adaptive filtering can be applied to remove noise while preserving the underlying signal of interest.

Baseline wander refers to low-frequency variations in the signal caused by patient movement or electrode placement. Techniques such as high-pass filtering or polynomial fitting can be used to remove baseline wander and restore the baseline to its original position. Normalization is essential for ensuring that the signals are on a consistent scale and have zero mean and unit variance. Normalization techniques such as min-max scaling or z-score normalization can be applied to scale the signals to a predefined range. ECG and PCG signals are typically segmented into individual heartbeats or cardiac cycles for analysis. Segmentation algorithms such as peak detection or template matching can be used to identify the start and end points of each heartbeat. Resampling may be necessary to ensure that the signals have a consistent sampling rate, especially if they were recorded at different rates. Techniques such as interpolation or decimation can be used to resample the signals to a desired sampling rate.

Any remaining artifacts or outliers in the signal can be rejected or interpolated to prevent them from affecting the analysis. Automatic artifact detection algorithms or manual inspection may be employed to identify and remove artifacts. After pre-processing, relevant features are extracted from the ECG and PCG signals to characterize various aspects of cardiac activity. Features such as amplitude, duration, frequency content, and morphological characteristics of the signals can be computed to represent different physiological phenomena. By applying these pre-processing steps, the ECG and PCG signals are cleaned, standardized, and prepared for further analysis by the CNN-LSTM model. This ensures that the model receives high-quality input data, leading to more accurate and reliable detection of cardiovascular abnormalities.

3.4 EMD

The EMD technique is pretty useful in data pre-processing. To represent the transformed audio data into signal data, the scalogram is used as the plot graph technique. Scalogram is generated, especially for the acquired and pre-processed datasets, namely, 'ECG', 'PCG', and finally the 'synchronized ECG&PCG'. The data acquired (audio) as a resource is transformed into digital through conversion (data transformation) where the noise is turned into a signal (noise-to-signal) through the data composition/decomposition method. This technique (EMD) allows the user to effectively use the transformed data with less error rate.

EMD offers a powerful pre-processing technique for enhancing the analysis of cardiovascular signals, such as ECG and PCG data. By decomposing these complex signals into a finite set of Intrinsic Mode Functions (IMFs), EMD enables a granular examination of their inherent oscillatory components. Through this decomposition, noise reduction becomes more targeted, as high-frequency noise components can be discerned and attenuated while preserving essential signal characteristics in smoother IMFs. Moreover, IMFs serve as rich sources of information for feature extraction, capturing nuances in amplitude, frequency, and energy distribution across different oscillatory scales. Additionally, the temporal analysis of IMFs enables the detection of transient phenomena and dynamic changes within the signals, offering insights into sudden irregularities or temporal patterns associated with cardiovascular abnormalities. By integrating EMD into the pre-processing pipeline, researchers can enhance the robustness and sensitivity of subsequent analysis techniques, ultimately improving the accuracy of cardiovascular abnormality detection and diagnosis.

3.5 System flow and architecture

3.5.1 Proposed model flow

The system flow of the proposed research includes several steps. According to the purpose of predicting/ identifying cardiac abnormalities through ECG and PCG signals by using the machine learning process, the following steps are focused (refer to Figure 1):

1.png

Figure 1. System flow of predicting cardiac abnormalities through ECG and PCG signals

3.5.2 Architecture diagram

The research adopts the hybrid algorithm that uses LSTM with CNN layers where the pre-processed datasets are processed and classified through the machine learning algorithm (deep learning) (refer to Figure 2).

2.png

Figure 2. Architecture of the developed cardiac-abnormalities prediction (CAP) model

The proposed hybrid model (refer to Figure 2) for predicting cardiac abnormalities is constructed with CNN architecture with LSTM which utilizes transfer learning and allows the researcher to access the pre-existing datasets (Physio-net) in PCG and ECG waveforms (heart sound recording files), which are acquired from two consecutive years and individual dataset namely, 2016 and 2017. Initially, two datasets are trained as separate modules with two distinct CNNs to predict and identify the abnormalities in the PCG and ECG as separate processes. Later, they are compiled and synchronized as one single integrated CNN with a converted dataset file (i.e., noise-to-signal=audio-to-digital). The abnormalities in the synchronized data are further examined and the errors are reduced through decomposition, where EMD is used as an error reduction-based reconstructive technique that thresholds or filters the IMFs. Thus, the outcome with minimal errors post identifying the abnormalities is classified as two datasets, namely, normal and abnormal where the performance of the model will be evaluated through metric evaluation technique using these two datasets to validate the reliability and consistency (accuracy).

3.6 Algorithm

The algorithm is the base for a model that designs and drives the model towards achievement. When a researcher finds more errors or bugs, fine-tuning the algorithm would provide them with the expected outcome when appropriate techniques and methods are approached. However, if a technique or a method is adopted based on similar data but not on the purpose, the adoption of the algorithm might result in producing more errors thus less accuracy. Thus, the selection of an algorithm is necessary in prediction-based machine learning models.

3.6.1 Algorithm adopted

To detect and classify the noise and signal from the datasets acquired in the research many existing researchers had used the SVM (support vector machine) algorithm. SVM provides the users with reduced error samples for training and testing the models. Contrarily, the LSTM (long-short-term memory) allows the users to predict and remember data accurately through selective memory and the machine learning model is also allowed to forget (cancel or delete) historical data. This process of the LSTM technique results in rapid processing and more space for the user in data analysis. Thus, in this research, the LSTM algorithm has been used to evaluate the model to predict the cardiac abnormalities in PCG and ECG signals. The EMD algorithm is used along with IMFs (intrinsic mode functions) here to identify the noise-to-signal frequencies and filter the noise (de-noise) the same. The de-noised data is represented through time-series-based data analysis. Thus, the composite signals are broken down into decomposed signals, where the features are extracted stage-by-stage through IMF.

The hybrid algorithm is far more effective than a single algorithm in machine learning models, especially for video and sound recording-based files where the processing and transformation of signals take more time than digital data analysis [29]. Hence, the research being an audio-dataset based prediction it adopts the hybrid algorithm of CNN with LSTM algorithm with EMD technique.

3.6.2 Training and testing

The datasets are split into training, testing, and validation (70:20:10 ratios) where the total dataset is split into three sections for analyzing the model. The raw data acquired (Physio-net 2016 and 2017) are transformed from audio into digital form (scalogram graph plots) as data transformation. Data transformed post-processing through EMD and IMF are then passed through the algorithm for the classification of classes "normal" and "abnormal" (refer to Figure 3).

3.png

Figure 3. Processing data through EMD

The split between training and testing data

Determining the split between training and testing data is a critical step in developing and evaluating machine learning models, including those based on CNN-LSTM architectures for cardiovascular abnormality detection. Before splitting the data, ensure that the entire dataset, comprising ECG and PCG signals along with their corresponding labels (indicating the presence or absence of abnormalities), is properly organized. Shuffle the dataset to randomize the order of samples, which helps prevent any bias in the data-splitting process.

Determine the ratio or percentage of data to allocate for training and testing purposes. Common split ratios include 70/30, 80/20, or 90/10, where the first number indicates the percentage of data allocated for training and the second number indicates the percentage allocated for testing. The choice of split ratio depends on factors such as the size of the dataset, the complexity of the model, and the availability of data for training and testing.

In scenarios where the dataset is imbalanced, meaning one class (e.g., abnormal samples) is underrepresented compared to others, consider using stratified splitting. Stratified splitting ensures that the distribution of classes remains consistent in both the training and testing sets, helping prevent biases in model evaluation. Once the split ratio is determined, partition the dataset into separate subsets for training and testing. Ensure that the split is random and that each subset contains a representative sample of the overall dataset. It's also common practice to reserve an additional subset, called a validation set, for hyperparameter tuning and model selection.

In situations where the dataset is limited in size, or to obtain more reliable performance estimates, consider using cross-validation techniques such as k-fold cross-validation. Cross-validation involves splitting the dataset into multiple subsets (folds), training the model on different combinations of training and validation sets, and averaging the performance metrics across folds. After splitting the data, perform a final check to ensure that there are no data leakage issues, where information from the testing set inadvertently influences the training process. By following these steps, ensure a fair and unbiased split between training and testing data, enabling robust evaluation of the CNN-LSTM model's performance in detecting cardiovascular abnormalities.

Once the error is identified by the decomposition algorithm (noise reduction), post noise reduction when the result obtained (MSE) is ‘0’ then the model returns the value that there is no noise and hence PSNR value has no significance. Based on the noise reduction algorithm (refer Algorithm) the signals are evaluated and the outcome is obtained. Noise reduction is applied to the datasets to acquire efficient and reliable outcomes.

Algorithm: Signal to Noise Ratio Calculation

def signal-to-noise (a, axis=0, ddof=0):

mse=np.mean((a)**2)

if (mse==0): # MSE is zero means no noise is

                             present in the signal

            # Therefore, PSNR has no importance.

        return 100

max_pixel=255.0

psnr = 20*log10(max_pixel/sqrt(mse))

return per

img=cv2.imread("pcg.png")

value=signal-to-noise(img)

print (f"PSNR value is {value} dB")

The model is thus trained to identify the anomalies in the input and they are initially stored as separate files where PCG and ECG are evaluated. Finally, the synchronization is done where the ECG and PCG outcomes are integrated as one outcome. Once the desired outcome is achieved the model is saved and tested. During the testing process, synchronized datasets are examined and evaluated to validate the model's performance.