Multi-Channel Fidelity Modeling of Transformer-Based Language Systems for Biomedical Translation under a Signal Processing Framework

From the perspective of signal processing theory, machine translation (MT) can be formally conceived as a cross-linguistic signal transformation problem: a source-language text constitutes a structured information signal, and the translation system operates as a signal transducer that encodes, transforms, and decodes this signal into a target-language representation [1, 2]. Under this framing, translation quality is fundamentally a fidelity problem—the degree to which the output signal preserves the informational content, structural patterns, and domain-specific features of the input signal. Shannon's foundational work on information theory [3] established that any transmission channel incurs potential information loss; in MT, this loss manifests as lexical substitution errors, syntactic distortions, and—most critically in specialized domains—failure to preserve high-frequency domain-specific terminology components.

Biomedical texts represent a particularly demanding signal transformation scenario: they exhibit high-density technical terminology (e.g., DNA, MRI, PCR) functioning as high-frequency spectral components that carry disproportionate semantic load, complex passive-voice constructions with deeply nested clausal structures, and strict numerical fidelity requirements analogous to amplitude preservation in signal reconstruction [4, 5]. These properties make biomedical translation a rigorous testbed for evaluating the multi-channel fidelity characteristics of translation systems [6].

The emergence of large language models (LLMs)—themselves built on attention-based signal processing architectures [7]—has substantially altered the MT landscape. Recent studies report that LLMs approach or surpass commercial NMT performance on high-resource language pairs [8, 9], while others note persistent domain mismatch and rare word prediction challenges in specialized settings [10]. This divergence reflects a critical methodological gap: most comparative studies rely on single-channel evaluation (BLEU alone), which captures only global waveform overlap while remaining orthogonal to domain-critical quality dimensions [11].

This study addresses these gaps through four research questions: (RQ1) Do LLMs and NMT systems differ significantly in biomedical text signal transformation fidelity? (RQ2) Can few-shot input conditioning with domain glossaries improve fidelity across all evaluation channels? (RQ3) How stable is transformation fidelity across medical sub-domain signal distributions? (RQ4) What is the inter-channel correlation structure, and can global waveform metrics adequately capture domain-spectral component preservation?

The contributions are: (1) a systematic six-tool comparison grounded in signal processing theory, using the latest model versions with real API/web outputs; (2) a multi-channel signal fidelity framework integrating global waveform, domain-spectral, amplitude, and composite evaluation channels; (3) empirical demonstration of input-conditioning asymmetry—a novel finding wherein increased signal conditioning degrades global fidelity in GPT-4o while uniformly improving spectral-component preservation across all LLMs.

3.1 Signal fidelity metrics in machine translation

MT evaluation has evolved from human judgment through string-matching to neural semantic assessment—a trajectory closely paralleling the development of signal quality metrics from subjective listening tests to objective perceptual models [11]. Papineni et al. [13] introduced BLEU, computing n-gram precision as a global waveform similarity measure. Popović [14] proposed chrF++, combining character-level and word-level n-gram F-scores, offering finer spectral resolution than BLEU. However, these surface metrics reward only exact pattern matches, assigning equal penalty to low-information function-word errors and critical domain-term mistranslations [12]. Zhang et al. [14] proposed BERTScore using contextual embeddings; Rei et al. [15] introduced COMET, demonstrating the highest correlation with human judgments in WMT evaluations [16]. These advances establish that translation signal quality assessment requires both surface and semantic measurement channels, supplemented by domain-specific indicators.

3.2 Large language model-based signal transformation capabilities

LLM translation research has grown rapidly since 2023. Jiao et al. [8] found ChatGPT approaching commercial NMT performance; Hendy et al. [9] confirmed GPT-4's advantages across 18 translation directions. Regarding input signal conditioning (prompt engineering), Peng et al. [17] and Moslem et al. [18] examined conditioning strategy effects on global fidelity. Domain-specific evaluations by Manakhimova et al. [19] demonstrated that LLM translation quality advantages over dedicated NMT systems are not uniform across specialized text types, with performance varying by domain and linguistic feature. Gao et al. [20] similarly found domain-dependent performance patterns when comparing ChatGPT, Google Translate, and DeepL in a specialized Chinese-to-English translation task, underscoring the importance of sub-domain analysis.

5.1 Overall signal fidelity comparison

Table 2 presents overall channel scores under zero-shot (LLM) and direct (NMT) conditions. On the global waveform channel (BLEU), Claude Sonnet-4 achieves the highest fidelity (38.17), followed by Gemini-2.5-Pro (38.05), Qwen3-Max (37.48), GPT-4o (37.35), DeepL (36.51), and Google Translate (35.86). The four LLMs form a compact fidelity band (37.35–38.17), all outperforming both NMT baselines in absolute terms.

Table 2. Multi-channel signal fidelity scores (zero-shot / direct translation)

Notably, all systems achieve high domain-spectral fidelity (Term. Accuracy: 0.937–0.975) and amplitude fidelity (Num. Consistency: 0.980–0.990), indicating that current-generation transducers—both LLM and NMT—reliably preserve high-frequency abbreviation components and numerical amplitude values in biomedical signal transformation. The inter-system differentiation on domain-spectral channels is substantially smaller than on the global waveform channel. Figure 1 visualizes these multi-channel fidelity patterns, confirming the compact LLM fidelity band on global waveform metrics (left panel) and the uniformly high domain-level performance across all transducers (right panel).

Figure 1. Overall multi-channel signal fidelity: global waveform metrics (left) and domain-spectral metrics (right)

Table 3 presents pairwise statistical comparisons with DeepL as baseline. No LLM achieves a statistically significant global fidelity advantage over DeepL (all p > 0.05), with Cliff's delta values ranging from +0.017 to +0.058 (all negligible). This indicates that while LLMs consistently outperform DeepL in absolute BLEU, the signal differences do not reach statistical significance with the current corpus size. Figure 2 presents a forest plot of these effect sizes. Gemini-2.5-Pro exhibits the largest positive effect (δ = +0.058, p = 0.072), followed by Claude Sonnet-4 (δ = +0.052, p = 0.114), while GPT-4o shows the smallest positive shift (δ = +0.017, p = 0.151). Google Translate is the only system with a negative effect relative to DeepL (δ = −0.021, p = 0.790). All effect size bars cluster near zero, visually confirming the negligible magnitude of pairwise fidelity differences and supporting the convergence interpretation.

Table 3. Statistical significance tests (BLEU channel, pairwise vs. DeepL)

Figure 2. Effect size forest plot (Cliff's delta vs. DeepL): LLM-NMT fidelity gap characterization

5.2 Input conditioning effects on signal fidelity

Table 4 presents signal fidelity changes under few-shot + glossary conditioning relative to the unconditioned (zero-shot) baseline. A striking system-dependent conditioning response emerges: GPT-4o exhibits a statistically significant global waveform fidelity decrease (ΔBLEU = −0.71, p = 0.004), while the other three LLMs show positive but non-significant gains. Claude Sonnet-4 achieves the largest positive response (+1.32), followed by Gemini-2.5-Pro (+0.70) and Qwen3-Max (+0.66). Critically, all four LLMs improve to near-perfect domain-spectral fidelity under conditioning (0.990–1.000 vs. 0.940–0.975 unconditioned). Figure 3 visualizes the conditioning-induced ΔBLEU for each LLM, highlighting GPT-4o's anomalous negative response. Figure 4 directly compares the absolute zero-shot and few-shot+glossary BLEU scores for each LLM. The visual contrast is striking: GPT-4o's conditioned output (36.6) falls below its unconditioned baseline (37.4), whereas Claude Sonnet-4 shows the most pronounced upward shift (38.2 → 39.5), with Gemini-2.5-Pro (38.0 → 38.7) and Qwen3-Max (37.5 → 38.1) exhibiting moderate gains. This asymmetric pattern underscores that few-shot conditioning does not uniformly enhance global waveform fidelity across LLM architectures.

Table 4. Input conditioning effects on multi-channel signal fidelity

Figure 3. Input conditioning effect on global waveform fidelity (ΔBLEU). Red bar indicates statistically significant fidelity decrease (p = 0.004)

=

Figure 4. Zero-shot vs. few-shot+glossary BLEU: per-system conditioning response

5.3 Multi-channel radar profile

Figure 5 provides a normalized multi-channel fidelity visualization across all six systems. The radar profile reveals that while LLM transducers cluster tightly on global waveform channels (BLEU, chrF++), substantial inter-system divergence emerges on the Rule-Based Accuracy and Rule-Based Completeness channels, where Gemini-2.5-Pro consistently leads. The visualization also captures Claude Sonnet-4's asymmetric channel profile: the highest global waveform fidelity (BLEU) yet a proportionally smaller normalized chrF++ advantage, suggesting distributional differences in how its output signal overlaps with reference translations at the character versus n-gram level.

Figure 5. Multi-channel radar profile of normalized signal fidelity metrics across all six transducers

5.4 Sub-domain signal distribution robustness

Figure 6 presents the global waveform fidelity heatmap across sub-domain signal distributions. Cross-domain fidelity variance differs substantially: Claude Sonnet-4 shows the highest variance (BLEU range: 28.93–52.57, σ = 7.0), driven by exceptionally high fidelity in Forensic Medicine (52.57) and Statistical Methods (46.36) channels but substantially lower fidelity in the Pharmacy channel (28.93). GPT-4o demonstrates the most uniform cross-domain response (σ = 3.1), suggesting more consistent signal transformation behavior across sub-domain spectral distributions. The Forensic Medicine channel (n = 2) should be interpreted with caution due to insufficient sample size for robust statistical estimation.

Figure 6. Global waveform fidelity (BLEU) heatmap across sub-domain signal distributions and transducers

5.5 Sentence-length signal sensitivity

Figure 7 shows global waveform fidelity across three signal-length categories. Most transducers peak at medium-length signals (20–40 words), with fidelity degradation for both short and long signals—consistent with the well-known length sensitivity of n-gram fidelity metrics [13]. Claude Sonnet-4 demonstrates the most stable length-domain response (3.9-point range across categories), while Google Translate shows the most severe long-signal fidelity penalty (4.0-point drop from medium to long). The asymmetric degradation pattern observed for DeepL (nearly flat from short to medium, then steep decline) suggests architecture-specific handling of signal complexity.

Figure 7. Signal-length sensitivity of global waveform fidelity (BLEU) across three length categories

5.6 Inter-channel correlation analysis

Figure 8 presents the Spearman inter-channel correlation matrix, computed on sentence-level scores from GPT-4o’s zero-shot output (N = 100). The central finding is the near-zero correlation between the global waveform channel (BLEU) and the domain-spectral channel (Terminology Accuracy, r = 0.12), confirming the theoretical prediction that these channels measure orthogonal quality dimensions. BLEU and chrF++ exhibit strong correlation (r = 0.84), indicating partial channel redundancy—both primarily capture the same global waveform construct. Numerical Consistency (amplitude channel) shows near-zero correlation with all other channels (|r| ≤ 0.03), confirming full orthogonality and establishing it as an independent fidelity dimension. This inter-channel orthogonality structure empirically validates the multi-channel framework design. Table 5 reports the complete set of pairwise correlation coefficients alongside their signal processing interpretations, further confirming the independence of evaluation channels.

Figure 8. Spearman inter-channel correlation matrix confirming near-orthogonal quality dimensions

Table 5. Spearman inter-channel correlation coefficients with signal processing interpretation

Figure 2 presents the effect size forest plot (Cliff's delta) for each system compared pairwise against DeepL. Gemini-2.5-Pro exhibits the largest positive effect (δ = +0.058, p = 0.072), followed by Claude Sonnet-4 (δ = +0.052, p = 0.114) and Qwen3-Max (δ = +0.031, p = 0.055), while GPT-4o shows the smallest positive shift (δ = +0.017, p = 0.151). Google Translate is the only system with a negative effect relative to DeepL (δ = −0.021, p = 0.790). All confidence intervals overlap zero and all effect sizes fall within the negligible range, visually reinforcing the convergence finding from Section 5.1 that current-generation transducers cluster within a narrow fidelity band.

Figure 4 provides a side-by-side comparison of absolute BLEU scores under zero-shot and few-shot+glossary conditions for each LLM. The visual contrast highlights the conditioning asymmetry discussed in Section 5.2: GPT-4o's conditioned output (36.6) falls below its unconditioned baseline (37.4), whereas Claude Sonnet-4 exhibits the most pronounced upward shift (38.2 → 39.5). Gemini-2.5-Pro (38.0 → 38.7) and Qwen3-Max (37.5 → 38.1) show moderate gains. This pattern confirms that few-shot conditioning with domain glossaries does not uniformly enhance global waveform fidelity across LLM architectures.