Physiology-Guided Transformer for Electrocardiogram Classification Via R-Peak-Informed Positional Encoding

Electrocardiography is one of the most widely used non-invasive diagnostic tools for assessing cardiac health. It provides crucial insights into the electrical activity of the heart and plays a central role in detecting conditions such as Myocardial Infarction (MI), Hypertrophy (HYP), arrhythmias, and other cardiac abnormalities [1]. Accurate interpretation of electrocardiogram (ECG) signals is essential for timely and effective clinical decision-making. However, manual analysis of ECGs can be time-consuming, error-prone, and subject to inter-observer variability, especially in high-volume clinical environments. These challenges have driven increasing interest in the development of automated, intelligent systems capable of interpreting ECG data with high accuracy and reliability [2].

In recent years, deep learning techniques have significantly advanced the field of automated ECG analysis. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been used to model spatial and temporal features of ECG signals, achieving strong performance across various classification tasks [3-5]. Despite their success, these models often struggle to capture long-range dependencies within the signal and may require extensive architectural complexity to compensate for their limited receptive fields. Moreover, they often lack interpretability, which is a critical requirement for clinical adoption [6-8].

The Transformer architecture, originally developed for natural language processing [9], has emerged as a powerful alternative for modeling long-range sequential data. Its self-attention mechanism allows the model to weigh the importance of different time steps, enabling it to learn complex temporal relationships without relying on recurrent structures. Although Transformers have been increasingly adopted in biomedical signal processing [10, 11], their direct application to ECG analysis presents several challenges. In particular, ECG signals are characterized by periodic patterns linked to the cardiac cycle, and traditional positional encodings used in Transformers may fail to capture this domain-specific temporal structure effectively.

To address this limitation, this study proposes an enhanced Transformer-based model that incorporates physiological knowledge directly into the attention mechanism. The core innovation lies in the use of a custom R-peak-informed positional encoding scheme that aligns the model's temporal representation with key cardiac events such as QRS complexes and ST segments. By encoding the temporal distance from each time point to the nearest R-peak, the model gains a contextually meaningful sense of timing, which improves its ability to focus on diagnostically relevant waveform components. This biologically grounded enhancement allows the Transformer to more effectively model the structure of ECG signals and improves both its predictive performance and interpretability.

The proposed model is evaluated on a clinically annotated benchmark dataset and compared with a variety of baseline architectures, encompassing both traditional deep learning approaches and more recent state-of-the-art models specifically tailored for ECG classification. Quantitative and qualitative analyses are performed to assess classification accuracy, computational efficiency, and interpretability. Ablation studies further isolate the contribution of R-peak-informed encoding and other architectural components. The results demonstrate that integrating physiological priors into attention-based models not only improves diagnostic performance but also enhances model transparency, making the approach suitable for real-world medical applications.

This paper is organized as follows: Section 2 reviews related work on ECG classification and deep learning approaches. Section 3 presents the proposed methodology, including the architectural design and the novel R-peak-informed encoding. Section 4 describes the experimental setup and results. Section 5 discusses ablation studies, benchmarking, and deployment aspects. Finally, Section 6 concludes the paper and outlines the directions for future research.

3.1 Preprocessing

Raw 12-lead ECGs are preprocessed to remove noise, including baseline wander below 0.5 Hz from respiration and electrode shifts. High-frequency noise from muscle electromyogram (EMG), powerline interference, and motion artifacts typically resides above 40 Hz. To isolate the clinically relevant frequency band, a fourth-order zero-phase Butterworth band-pass filter with cutoff frequencies of 0.5 Hz and 40 Hz was applied. The Butterworth filter was selected for its maximally flat passband response, minimizing signal distortion within the band of interest. The zero-phase implementation, achieved by forward and backward filtering, was critical to prevent phase shift, which represents a distortion that can alter the temporal alignment of key waveform components such as the QRS complex and T-wave.

Mathematically, if x_i(t) represents the raw signal from lead i, the filtered signal $\widetilde{x_i}(t)$ is obtained. To quantify the impact, we conducted an ablation study where the model was trained on data preprocessed with alternative filter settings. This analysis confirmed that the chosen band (0.5–40 Hz) optimized the signal-to-noise ratio (SNR), leading to a ~3% improvement in macro F1-score compared to using unfiltered data or suboptimal bands. The former preserved excessive baseline drift, while the latter admitted too much high-frequency noise, both of which degraded feature extraction. Following filtering, the signals were resampled from 1000 Hz to 100 Hz using polyphase anti-aliasing filtering to reduce computational overhead while preserving the Nyquist frequency for all morphologically significant waves.

3.2 Segmentation

After preprocessing, the ECG signals are divided into smaller segments for localized analysis. This step aims to extract fixed-length windows centered around critical cardiac events, specifically the R-peaks. Each segment spans 1000 samples, corresponding to a 10-second duration at a sampling rate of 100 Hz. The segmentation process uses detected R-peaks to center each window, which helps ensure that at least one complete cardiac cycle is included in each segment. Let $R=\left\{r_1, r_2, \ldots, r_{\mathrm{n}}\right\}$ represent the sequence of detected R-peaks. Each segment $x_k \in R^{T \times 12}$ is constructed as: $s_k=\left[\widetilde{x_1}\left(r_k-T / 2: r_k+T / 2\right), \ldots, \widetilde{x_{12}}\left(r_k-T / 2: r_k+T / 2\right)\right]$ , where $\widetilde{x_i}$ denotes the filtered signal from lead i, and T = 1000 is the window length. This R-peak-centered segmentation improves the temporal alignment of cardiac cycles across segments and reduces inter-class ambiguity during training and inference.

3.3 R-Peak extraction

R-peak extraction is a critical step that serves both the segmentation and positional encoding components of the model. R-peaks represent the most prominent feature of the ECG waveform, corresponding to ventricular depolarization during the QRS complex. Various signal processing techniques can be used for R-peak detection, including the well-known Pan–Tompkins’s algorithm and simpler energy-based methods. One such approach involves calculating the local energy envelope of the signal using derivatives. The energy function $E(t)$ is defined as $E(t)=\sum_{\tau=t-w}^{t+w}(d x(\tau) / d \tau)^2$, where w is the window size. Peaks in the energy signal that exceed a dynamic threshold are identified as R-peaks. The resulting set $\left\{r_k\right\}$ is used not only to guide the segmentation of ECG windows but also to enrich the temporal representation during the encoding phase of the model.

3.4 R-peak-informed positional encoding

A major innovation of this work lies in the incorporation of R-peak-informed positional encoding, which enhances the model’s ability to interpret temporal structure in ECG signals. Traditional transformer models use fixed positional encodings, such as sinusoidal functions, to provide information about the relative positions of elements in a sequence. These are defined as $P E_{(t, 2 i)}=\sin \left(t / 10000^{2 i / d}\right)$ and $P E_{(t, 2 i+1)}=\cos \left(t / 10000^{2 i / d}\right)$, where t is the position and d is the model's dimension. In our model, we extend this scheme by incorporating a cardiac-aware proximity weighting function $\rho(t)$, which captures the relative distance between each time step and the nearest R-peak. This weighting is defined as $\rho(t)=\exp \left(-\left(t-r_k\right)^2 / 2 \sigma^2\right)$, where $r_k$ is the nearest R-peak and $\sigma$ controls the spread. The modified positional encoding is given by $P E_{(t, i)}^{\prime}=P E_{(t, i)} \cdot(1+\alpha \cdot \rho(t))$, where $\alpha$ modulates the strength of the R-peak influence. This enhancement allows the transformer model to assign greater importance to time points near cardiac events, such as QRS complexes, thereby improving diagnostic relevance and attention alignment.

3.5 Transformer-based classification model

The core component of the system is a transformer-based deep learning model that is capable of learning both local morphological features and long-range temporal dependencies in multichannel ECG signals. The model input is a segmented window $X \in R^{T \times \mathbb{1} \mathbb{2}}$, where T = 1000. First, a TimeDistributed 1D convolutional layer is applied to each ECG lead independently to extract low-level temporal features, resulting in a transformed input $X^{\prime}=\operatorname{Conv} 1 D_{\text {Leads }}(X)$. The R-peak-informed positional encoding $P E^{\prime} \in R^{T \times d}$ is then added to $X^{\prime}$, combining morphological and positional information. The result is fed into a series of transformer encoder blocks; each composed of a multi-head self-attention layer and a feed-forward neural network. The self-attention mechanism computes the weighted representation of the sequence using the formula Attention $(Q, K, V)=\operatorname{softmax}\left(Q K^T / \sqrt{d_k}\right) V$, where Q, K, V are the query, key, and value matrices derived from the input, and $d_k$ is the key dimension. The feed-forward network (FFN) applies a nonlinear transformation using $\operatorname{FFN}(x)=\operatorname{LeakyReLU}\left(x W_1+b_1\right) W_2+b_2$, enabling the model to learn more abstract representations. Residual connections and layer normalization are employed to stabilize training and maintain gradient flow. A global average pooling operation is used to reduce the temporal dimension, followed by two dense layers that culminate in a softmax output. The final output $\hat{y}=\operatorname{softmax}(W x+b)$ is a probability vector over the diagnostic classes: MI, HYP, and Normal. This architecture integrates clinical priors into the attention mechanism, making the model both accurate and interpretable in its decision-making.

5.1 Ablation study

To assess the importance of each architectural component, we conducted several ablation experiments. These included removing or modifying the positional encoding, as well as varying the number of attention heads and encoder layers. Table 4 presents the results of the ablation study in terms of the macro-averaged F1-score.

Table 4. Ablation study results

The full model, which includes both R-peak-informed temporal encoding and standard multi-head self-attention, achieved a macro F1-score of 94.9%, which serves as the performance upper bound. When the R-peak-informed encoding was removed, the macro F1-score dropped to 91.2%, highlighting the importance of incorporating physiologically aligned temporal priors into the attention mechanism. This component alone contributes to a relative performance gain of nearly 4%. Furthermore, replacing the R-peak-informed scheme with a standard sinusoidal positional encoding resulted in an even lower F1-score of 90.8%, suggesting that the model benefits from explicitly learning timing patterns that align with cardiac physiology rather than relying solely on uniform time-step encodings.

Even more pronounced was the performance degradation observed when positional encoding was entirely excluded from the model. In this configuration, the Transformer performed with a macro F1-score of only 88.6%, which demonstrates that temporal order information is essential for meaningful sequence representation in physiological signal classification.

To visually present the impact of these design decisions, we provide a bar plot in Figure 5. This figure compares the macro F1-score achieved by each model variant. It is evident that removing or simplifying temporal encodings causes a consistent decline in classification performance, whereas adding complexity in terms of more attention heads and deeper encoder blocks improves the performance, but with diminishing returns. For example, the variant using two heads and only one encoder block achieved an F1-score of 91.4%, which is significantly lower than the full model. In contrast, using eight heads and four encoder blocks achieved 93.5%, suggesting that although deeper and wider configurations improve representation, marginal gains do not outweigh the added computational cost when compared to the full model’s more balanced configuration.

Figure 5. Ablation study: Impact of model components on performance. Each bar represents a different variant of the model, showing how the removal or simplification of components degrades the macro F1-score

The ablation study confirms the critical importance of incorporating physiological domain knowledge, particularly the R-peak-informed positional encoding mechanism. It also demonstrates that while increasing model capacity by adding more attention heads and encoder layers does provide measurable improvements, the proposed configuration represents a well-optimized compromise between performance and computational cost. The study further validates the architectural choices of the model and affirms the design rationale for incorporating cardiac-specific inductive biases into the transformer framework for ECG analysis.

5.2 R-Peak detection errors analysis

The effectiveness of physiology-guided models depends on the reliability of biological feature extraction. Since our R-peak-informed encoding requires precise identification of R-peaks, we evaluated its resilience to detection errors. From the test set, 200 segments with perfect annotations were selected, and controlled perturbations were introduced. False Negatives (FNs) were simulated by removing a percentage of true peaks, while False Positives (FPs) were generated by inserting spurious peaks between them. Error rates were varied between 5% and 20%, and the macro F1-score was measured at each level.

Figure 6 illustrates that performance degrades gradually as error rates increase. At 10% error, the F1-score declined from 94.9% to 92.5% with FNs and to 91.8% with FPs, showing that spurious peaks have a stronger negative effect because they misdirect the attention mechanism, whereas missing peaks primarily create localized information gaps. Importantly, even at 20% error, the model retained high overall performance, which highlights the resilience provided by global self-attention and its ability to smooth over imperfect annotations.

5.3 Comparative analysis

To validate the model's effectiveness relative to existing approaches, we compared it with both classical and state-of-the-art models. These included convolutional, recurrent, hybrid, and attention-based architectures. The results, presented in Table 5, clearly demonstrate the superiority of our model across all key evaluation metrics, including accuracy, macro-averaged F1-score, and ROC-AUC.

Figure 6. Model robustness to synthetic R-peak detection errors. Classification performance (Macro F1-Score) under increasing false negatives (FNs, dashed) and false positives (FPs, solid). Performance degrades gradually up to ~15% error, with FPs causing slightly greater loss. Shaded areas indicate ±1 standard deviation over 5 runs

Table 5. Comparison with baseline and state-of-the-art models

Compared to traditional deep learning models such as the 5-layer CNN and the bidirectional LSTM, which achieved macro F1-scores of 87.9% and 89.2%, respectively, the proposed model achieved a significantly higher F1-score of 94.9%. This represents an improvement of more than five percentage points in F1-score, reflecting a substantial enhancement in classification balance and reliability. The ROC-AUC metric also increased, improving from 0.945 for CNN and 0.953 for the Bi-LSTM to 0.984 for the proposed model. In terms of overall accuracy, our model reached 94.8%, compared to 88.6% for CNN and 89.7% for Bi-LSTM.

The model was also evaluated against more advanced deep architectures, including GRU with attention, one-dimensional ResNet, and a ResNet-GRU hybrid. These models achieved macro F1-scores ranging from 90.7% to 91.1%. In contrast, the proposed model outperformed them by margins between 3.8 and 4.2 percentage points in F1-score. These results emphasize the effectiveness of incorporating R-peak-informed temporal priors, which guide attention to clinically relevant waveform regions and improve feature discrimination.

In addition, the proposed model demonstrated superior performance compared to recent Transformer-based approaches. Models such as ViT-ECG and ECG-BERT achieved F1-scores of 92.6% and 93.7%, respectively.

A deeper architectural analysis clarifies these differences. ViT-ECG adapts the Vision Transformer paradigm by treating the ECG as a two-dimensional image. This requires fixed-length input segments and relies on standard, learnable positional encodings that are agnostic to the periodicity of cardiac signals. While effective, this approach introduces architectural rigidity and necessitates a larger parameter footprint (5.2 M) to compensate for the absence of physiological priors. ECG-BERT, in contrast, employs a large pre-trained Transformer with 12.5 M parameters, leveraging transfer learning from a corpus of unlabeled ECG signals. Although this enables the extraction of general-purpose representations, it also incurs high computational complexity, memory requirements, and inference latency, which limit its suitability for real-time or embedded applications. The proposed model addresses these limitations by embedding a cardiac-specific inductive bias through R-peak-informed positional encoding. This mechanism aligns attention with clinically meaningful temporal landmarks, enabling the model to achieve superior accuracy with a compact architecture of only 3.9 M parameters.

Although these models performed well, they required significantly higher model complexity and computational resources. ECG-BERT, for example, contains more than 12 million parameters, whereas our model achieves a higher accuracy and F1-score with only 3.9 million parameters. This highlights the efficiency of our architecture in achieving strong diagnostic performance with a reduced computational footprint.

To confirm statistical reliability, we performed two-tailed paired t-tests and Wilcoxon signed-rank tests over 10 random subject-wise splits. The results showed significant improvements over CNN (p = 0.003) and Bi-LSTM (p = 0.008), and remained consistent with other comparisons, validating the robustness of the performance gain.

Figure 7. Inference time vs. model size for ECG classification models. The proposed transformer achieves an optimal trade-off, balancing compactness and high performance

To evaluate real-world deployment ability, Figure 7 visualizes the trade-off between model size and inference speed across all models. Although CNN is compact, its performance is poorer. ViT-ECG and ECG-BERT offer high accuracy but suffer from larger size and latency. Our model achieves strong performance with a balanced resource footprint, making it ideal for point-of-care and mobile diagnostic settings.

These results demonstrate that the proposed R-peak-guided Transformer achieves superior classification accuracy while maintaining a well-balanced trade-off between predictive performance, interpretability, and computational efficiency.

5.4 Deployment efficiency analysis

To further ensure deployment feasibility, we benchmarked our model across three hardware platforms. On an NVIDIA RTX 3090 GPU, inference was completed in ~2.1 ms per ECG segment, supporting real-time deployment in hospital systems. On a Snapdragon 888 mobile processor, the model required ~148 ms, enabling seamless integration into point-of-care diagnostic tools and mobile health applications. On a Cortex-M4 microcontroller, direct deployment of the full model is challenging due to memory constraints, but with aggressive 8-bit quantization and pruning, inference could be achieved in ~2.1 seconds, demonstrating feasibility for wearable monitoring in resource-limited contexts. These results underline the adaptability of the model across diverse deployment settings, while optimization techniques such as post-training quantization, structured pruning, and knowledge distillation remain promising avenues for further reducing computational demands. Table 6 presents inference latency, power use, and deployment contexts across platforms, highlighting the model’s efficiency in varied scenarios.

Table 6. Cross-platform inference latency and deployment viability

5.5 Attention interpretability analysis

Beyond qualitative visualization, we established a quantitative framework to assess the clinical alignment of the model's attention mechanism. This evaluation measures the degree to which the model's focus coincides with clinically established regions of interest (ROIs) for each cardiac condition. For a stratified subset of 50 test samples per class, clinical experts manually annotated key segment boundaries anchored to detected R-peaks: the ST segment (from J-point to the end of the T-wave) for MI, the QRS complex (from onset to offset) for HYP, and the entire cardiac cycle for Normal (NORM) rhythms.

We then computed an Attention Focus Ratio, defined as the proportion of the total attention weight (from the final transformer layer's [CLS] token attention head) that falls within the expert-annotated ROI. A higher ratio indicates a stronger concentration of attention within clinically meaningful regions.

The results, summarized in Table 7, provide robust quantitative validation of the model's interpretability. For MI samples, a mean of 78.4% of the model's attention was allocated to the ST segment, directly reflecting the clinical primacy of ST-deviation in infarction diagnosis. For HYP, 72.1% of attention was concentrated within the QRS complex, which encodes voltage criteria essential for HYP assessment. In contrast, attention for NORM was significantly more distributed, with a mean ratio of 41.3%, reflecting the absence of a localized pathological feature.

This structured analysis reinforces the qualitative patterns observed in attention maps and demonstrates that the R-peak-informed encoding effectively biases the Transformer toward physiologically salient waveform regions. As a result, the model’s decision-making is not only accurate but also transparent and clinically interpretable.

Table 7. Quantitative analysis of attention-clinical alignment

5.6 Discussion

The experimental results confirm that embedding domain-specific temporal priors into the Transformer, through R-peak-informed positional encoding, enhances its ability to capture the temporal and morphological structure of cardiac cycles. This hybrid attention–physiology design improves ECG classification performance, robustness, and interpretability.

Ablation studies show that R-peak-guided encoding is key to improving model accuracy. Unlike traditional sinusoidal encodings, which provide uniform positional cues, the R-peak-informed method incorporates temporal distance to the nearest R-wave, allowing attention to better target clinically relevant regions such as QRS complexes and ST intervals. The consistent gains in macro F1-score and ROC-AUC highlight the value of embedding physiological priors in time-series modeling.

Quantitative evaluation shows that the model achieves consistently high accuracy across diagnostic categories, with F1-scores above 95% for MI detection and Normal rhythm identification. Some confusion occurs between HYP and Normal classes, which reflects their clinical similarity. The model’s ability to capture subtle temporal and spatial patterns across all 12 leads highlights its strong generalization and sensitivity to minor abnormalities.

In comparative evaluations, the proposed model outperforms conventional architectures like CNN and Bi-LSTM, as well as advanced methods such as GRU-Attention, 1D-ResNet, ViT-ECG, and ECG-BERT. It achieves superior macro F1-scores and ROC-AUC with fewer parameters and faster inference, offering an optimal trade-off between diagnostic accuracy and efficiency. This balance makes it well-suited for resource-limited environments, including wearable devices, portable ECG systems, and time-critical clinical applications.

Beyond technical performance, the translation of AI tools into clinical practice necessitates addressing key deployment challenges. The model's efficiency makes it suitable for on-device inference, enhancing data privacy by minimizing cloud transmission. However, maintaining model efficacy over time requires secure strategies for updates, such as federated learning, which preserves patient confidentiality. Ultimately, adoption by healthcare professionals depends on integrating transparent, interpretable outputs into clinical workflows to build trust and ensure that AI augments rather than disrupts diagnostic reasoning. Proactively addressing these technical and ethical considerations is crucial for successful real-world implementation.

The model's interpretability, quantitatively and visually demonstrated, is a key asset in addressing these deployment challenges. Temporal attention visualizations, which showed alignment with clinical reasoning. For MI, attention concentrated on early depolarization and ST segments in anterior leads, consistent with infarction markers. In HYP, focus shifted to late depolarization in lateral leads, reflecting ventricular enlargement. Normal ECGs displayed diffuse attention, mirroring the absence of focal abnormalities. These physiologically meaningful patterns enhance transparency and support clinical acceptance.

While the PTB Diagnostic ECG Database provides high-quality, well-annotated signals suitable for initial validation, its limitations must be acknowledged. The dataset’s participant pool, though clinically valuable, does not fully capture the diversity of anatomical variations, age groups, ethnicities, and comorbidities observed in global populations. In addition, the signals were collected under relatively controlled conditions, which do not reflect the noise, artifacts, and quality fluctuations that typically occur in ambulatory, wearable, or emergency settings. As a result, the strong performance reported here should be interpreted within the context of this dataset. Validating the model on external cohorts recorded in different hospitals, with varied ECG hardware and a broader range of pathologies, remains an essential step to establish robustness and real-world applicability of the proposed R-peak-informed encoding scheme.

Another important limitation lies in the model’s reliance on accurate R-peak detection. Errors in this preprocessing stage may produce incorrect positional encodings, ultimately degrading performance. Future work should therefore focus on extending evaluation to more diverse datasets and enhancing the model’s resilience to noise and potential inaccuracies in peak detection.

Prospective enhancements could include extending the model to support multi-label classification in order to identify co-occurring cardiac abnormalities. Incorporating hierarchical attention mechanisms or hybrid architectures that combine convolutional and transformer components may also improve robustness and denoising capabilities.