Risk-Calibrated Mitigation of XSS and SQL Injection in JSON API Traffic Using Deep Learning and Class-Specific Thresholds

Modern web applications rely on Application Programming Interface (API)-based communication, in which user-controlled data arrives through query parameters, headers, cookies, form fields and structured request bodies. JavaScript Object Notation (JSON) dominates this traffic because it supports lightweight, machine-readable exchange between clients, mobile applications, backend services and cloud platforms. The design improves interoperability but widens the input-validation attack surface, because malicious payloads can hide inside nested objects, arrays, serialized values or encoded strings rather than appearing as simple parameters [1, 2]. Cross-Site Scripting (XSS) and SQL Injection (SQLi) remain the most persistent input-validation attacks in this setting. They target different execution contexts, a browser and a database respectively, but share one weakness: untrusted input reaches a sensitive sink without adequate validation, sanitization, canonicalization, or contextual encoding [2, 3]. In JSON API traffic, the same malicious intent appears under many surface forms, including URL encoding, HTML entities, Unicode confusables, whitespace variation, comment padding, case variation and JSON wrapping, and that variability is what makes reliable detection difficult.

Deep-learning detectors perform well on this task. Sequence models, including Convolutional Neural Network (CNN), Long Short-Term Memory network (LSTM), and Bidirectional Long Short-Term Memory network (BiLSTM), attention architectures and multi-feature fusion models, learn attack-related patterns such as SQL operators, delimiters, HTML tags, JavaScript fragments and encoded characters [4, 5]. High clean-test accuracy is not sufficient for deployment, however. A detector may classify benchmark samples correctly yet fail once an attacker applies semantics-preserving transformations, and black-box and reinforcement-learning mutation methods are known to produce evasive XSS and SQLi variants against learned detectors and WAF-like systems [6, 7]. Deployment also demands more than a predicted class: an API gateway must allow, block, challenge, or escalate every request, and a maximum-probability classifier cannot express that policy. What is needed is risk-calibrated mitigation, in which confidence is interpreted through class-specific thresholds and translated into auditable actions. Calibration is the prerequisite, because neural networks produce confidence scores that need not reflect the true likelihood of correctness; temperature scaling corrects this without retraining [8]. It must be tied to the decision thresholds, because the cost of a missed SQLi is not the cost of a blocked benign request [9, 10].

This paper presents a risk-calibrated mitigation framework for XSS and SQLi detection in JSON-oriented API traffic. The framework uses a trained deep-learning detector as a scoring component, converts its outputs into calibrated probabilities by temperature scaling, and applies class-specific thresholds with an explicit review margin to produce Allow, Block and Review actions. The mitigation layer applies the threshold vector 0.40/0.40/0.60 for Normal, SQLi and XSS respectively, together with a review margin of 0.30 that withholds near-miss requests from the Allow action. The selected constants are fitted on one half of the validation split and every reported figure is computed on data disjoint from that fit.

The contributions are as follows. The paper formulates XSS and SQLi defense as a mitigation decision problem rather than a three-class classification problem; applies class-specific thresholds with an explicit review margin so that the policy reflects different operational risks; defines an auditable Allow/Block/Review layer; evaluates the calibrated system on a corpus deduplicated at template level, against an adversary re-optimized against the deployed pipeline, and under a latency budget; and reports how much the evaluation protocol itself, rather than the model, determines the headline numbers.

The framework has four stages: a request is canonicalized, scored by a trained BiLSTM-Attention detector, converted into calibrated probabilities by temperature scaling, and mapped to an action by class-specific thresholds and a review margin. The first three estimate how likely each class is; only the fourth decides what to do, and keeping them separate is what makes the policy auditable and adjustable without retraining.

3.1 Input and canonicalization stage

Let x represent a raw web payload or API request content. The input may be a raw string, URL parameter, header value, form field, or JSON request body. Since the target deployment setting is JSON-oriented API traffic, the input may contain nested objects, arrays, escaped strings, encoded characters, or attack-bearing values embedded inside structured fields.

Before inference, the input is processed by a canonicalization function:

x_c = C(x)          (1)

where, C(·) denotes the JSON-aware canonicalization layer and x_c is the canonicalized input. The purpose of this stage is to reduce representation-level variance before the detector sees the payload. The canonicalization layer handles supported forms of URL encoding, HTML entity encoding, selected Unicode/confusable forms, whitespace and comment variation, and deterministic JSON serialization.

3.2 Deep learning scoring layer

After canonicalization, the input is encoded into a model-ready sequence representation and passed to the trained detector. Let f_θ denote the selected BiLSTM-Attention model with parameters θ. The detector produces a vector of raw logits:

z = f_θ(x_c)         (2)

z = (z_Normal, z_SQLi, z_XSS)          (3)

The three target classes are:

K = {Normal, SQLi, XSS}            (4)

The raw logits are not used directly as deployment decisions; they are passed to the calibration stage, because neural-network scores may be poorly calibrated and a high raw score need not correspond to reliable operational confidence.

3.3 Temperature scaling

The framework applies temperature scaling to transform raw logits into calibrated probabilities. Temperature scaling is a post-processing method that adjusts the sharpness of the predicted probability distribution without retraining the model [8].

Given logits z and temperature parameter T > 0, the calibrated probability for class k is:

p_k = exp(z_k/T) / Σ_j∈K exp(z_j/T)            (5)

If T > 1, the probability distribution becomes softer. If T < 1, the probability distribution becomes sharper. The temperature value is fitted using the validation set and then frozen before the final test and adversarial evaluation. The detector parameters are not changed during this step.

For the selected robust configuration, the temperature is fitted on the val_fit half of the validation split by minimising the negative log-likelihood with the L-BFGS optimizer, following Guo et al. [8], and is then frozen. The fitted value is T = 0.4889. Because T < 1, the softmax distribution is sharpened, and because the transformation is strictly monotone, it preserves the ranking of classes within a request, so the predicted label is unchanged and only the confidence that the thresholds act on is recalibrated.

The calibrated probability vector is:

p = (p_Normal, p_SQLi, p_XSS)           (6)

3.4 Class-specific thresholds

Instead of applying a single global threshold or relying on the maximum-probability class, the framework uses separate thresholds for each class:

τ = (τ_Normal, τ_SQLi, τ_XSS)           (7)

Instead of one global threshold, each class carries its own confidence requirement, because the operational cost of an error differs by class: a missed SQLi may expose the database, while a blocked benign request denies service to a legitimate user. A fourth constant, the review margin, is a floor on the malicious score above which a request is treated as uncertain rather than allowed; because it applies to whichever malicious class scores higher, the resulting band runs from the margin up to that class's own blocking threshold. For the selected robust configuration, the constants fitted in Section 5.2 are:

τ_Normal = 0.40, τ_SQLi = 0.40, τ_XSS = 0.60           (8)

τ_review = 0.30     (9)

The thresholds are selected on validation data by grid search over the temperature-scaled probabilities, maximising macro-F1, and are then frozen for all clean-test, adversarial and latency evaluation. Section 4.4 gives the protocol and Section 5.2 the resulting operating point.

The thresholds are selected on val_fit alone and are not adjusted after seeing val_report, clean-test or adversarial results. Freezing them before any reporting split is scored avoids optimistic bias and makes the operating point auditable: the constants that govern deployment are fixed artifacts, versioned alongside the model.

The candidate threshold values form a grid over {0.40, 0.45, ..., 0.90} for each class (step 0.05), giving 11³ = 1,331 combinations. Each combination is scored on the calibrated validation probabilities, and the vector that maximizes validation macro-F1 is retained. The search space, the resulting operating-point comparison, and the sensitivity of macro-F1 and the Review rate to the thresholds are reported in Section 5.2.

3.5 Mitigation decision logic

The calibrated mitigation layer converts calibrated probabilities into one of three operational actions: Allow, Block, or Review. The decision rule prioritizes malicious classes before allowing traffic.

Malicious classes are evaluated before benign ones, and a request whose strongest malicious probability falls inside the review margin is withheld from Allow even when the Normal probability would otherwise clear its threshold. Using the selected constants, the decision logic is:

(1) if p_SQLi ≥ 0.40 → Block as SQLi

(2) else if p_XSS ≥ 0.60 → Block as XSS

(3) else if max(p_SQLi, p_XSS) ≥ 0.30 → Review

(4) else if p_Normal ≥ 0.40 → Allow

(5) else → Review

The rule differs from argmax classification in two ways. Malicious thresholds are checked first, so a request with a sufficiently high SQLi or XSS probability is blocked even when another class scores similarly. The review margin in (3) then captures the near-miss region that neither blocks nor confidently allows. Without (3), the Review action is effectively unreachable at this operating point: on the clean test split it fires on 0 of 27,109 requests, because a request whose malicious probabilities are both below their thresholds almost always retains a Normal probability above 0.40. The margin is what makes the third action operative, and Section 5.3 quantifies the trade-off it buys.

3.6 Algorithm and security interpretation

Allow lets the request continue to the protected application, and Block stops it. Review marks it uncertain: not confidently malicious enough to block, not confidently benign enough to allow. In deployment, it may trigger logging, secondary WAF inspection, analyst review, rate limiting or a challenge, and its purpose is to stop low-confidence traffic from being silently treated as safe.

The following examples illustrate the calibrated rule using tau = (0.40, 0.40, 0.60) and a review margin of 0.30.

Example 1: Clear SQLi. If p_Normal = 0.12, p_SQLi = 0.81 and p_XSS = 0.07, then p_SQLi ≥ 0.40 and the decision is Block SQLi.

Example 2: Clear XSS. If p_Normal = 0.18, p_SQLi = 0.10, and p_XSS = 0.72, then p_XSS ≥ 0.6 and the decision is Block XSS.

Example 3: Benign request. If p_Normal = 0.86, p_SQLi = 0.08, and p_XSS = 0.06, then neither malicious threshold is exceeded and p_Normal ≥ 0.4. The decision is Allow.

Example 4: Uncertain request. If p_Normal = 0.42, p_SQLi = 0.39 and p_XSS = 0.19, no class clears its blocking threshold, but max(p_SQLi, p_XSS) = 0.39 reaches the review margin of 0.30 without clearing the SQLi threshold of 0.40, so (3) applies and the decision is Review. This is the case raised in Section 2.2: under blocking thresholds alone the request would have been allowed.

The framework improves deployment suitability in four ways. It makes the decision policy explicit, so thresholds can be audited, versioned and changed under operational control. It separates scoring from enforcement. It routes low-confidence requests to Review rather than allowing them. And it supports different risk tolerances, since a deployment that fears false negatives can lower the malicious thresholds or treat Review as a temporary block, while one that fears false positives can do the opposite. The novelty is the explicit treatment of threshold calibration as a mitigation mechanism rather than as a reporting convention.

5.1 Threshold calibration results and calibration quality

The first evaluation objective is whether the selected robust detector can be converted into a stable mitigation operating point using temperature scaling and class-specific thresholds. The temperature, the thresholds and the review margin are all fitted on val_fit; val_report and the clean test split are scored with those frozen constants and are used for reporting only.

The selected robust configuration is BiLSTM-Attention + JSON-aware canonicalization + FSAT + calibrated thresholds. The fitted temperature is T = 0.4889 and the selected threshold vector is τ = (0.40, 0.40, 0.60) for [Normal, SQLi, XSS], with a review margin of 0.30. On val_fit, this operating point attains macro-F1 = 0.9881; on the held-out half of the validation split it attains macro-F1 = 0.9859 and accuracy = 0.9848. The difference between the two, 0.0023, is the direct measure of how much the threshold search exploited the split it was tuned on.

Repeating the whole fit on two further validation partitions gives the same threshold vector in all three cases, temperatures of 0.4889, 0.4706, 0.4800 and val_report macro-F1 of 0.9859, 0.9866, 0.9871; the fit-to-report gap has mean 0.0009 and never exceeds 0.0023. The operating point is therefore a property of the model rather than of one partition (Table 7).

Table 7. Nested calibration protocol and performance by split

Temperature scaling improves the reliability of the detector's confidence without changing the ranking of classes within a request.

On the clean test split, ECE falls from 0.0382 to 0.0109, NLL from 0.0809 to 0.0625 and the Brier score from 0.0398 to 0.0287; val_report follows the same pattern (ECE 0.0402 to 0.0102 (Table 8)). Figure 1 shows the corresponding reliability diagrams: before scaling the accuracy exceeds the mean confidence in every populated bin, and after scaling the bin accuracies track the diagonal more closely. Because temperature scaling is strictly monotone, it preserves each sample's argmax label, so the classification metrics are essentially unchanged; what changes is the confidence that the thresholds act on. The maximum calibration error is reduced far less than the average one on the test split (0.6231 to 0.5863), so a small number of sparsely populated high-confidence bins remain miscalibrated after scaling.

Table 8. Calibration metrics before and after temperature scaling

Note: Expected Calibration Error (ECE); negative log-likelihood (NLL)

tu_pian_1.png

5.2 Threshold operating-point analysis

The class-specific thresholds are chosen by grid search over {0.40, 0.45, ..., 0.90} per class (1,331 combinations) on the temperature-scaled val_fit probabilities, maximising macro-F1. The maximum, 0.9881, is attained by 21 grid points, all sharing τ_Normal = 0.40 and τ_SQLi ≤ 0.50; the tie is broken toward the smallest vector in grid order, giving (0.40, 0.40, 0.60). Compared with the operating point reported in the previous version, the SQLi threshold moves from 0.60 to 0.40. The change is a consequence of deduplication rather than of a different search: with template-level duplicates removed the Normal-SQLi boundary is harder, and macro-F1 now prefers the more sensitive SQLi threshold.

Macro-F1 is nearly flat across a broad plateau of malicious thresholds (Figure 2), so the operating point is chosen for its decision behaviour rather than for a macro-F1 difference that the plateau cannot resolve. The Normal threshold acts as a safety dial (Figure 3): macro-F1 is stable up to about 0.40 and declines beyond it. The review margin is selected separately, on val_fit, as the widest band whose benign review burden stays within a fixed 1% budget; that yields τ_review = 0.30, which on val_fit routes 0.39% of requests to Review. Table 9 reports the effect of this margin on the clean test split.

tu_pian_2.png

tu_pian_3.png

Figure 3. Effect of the Normal threshold on val_fit macro-F1 and on the Review rate

Table 9. Effect of the review margin on the clean test split

Note: SQL Injection (SQLi)

5.3 Per-class, clean-test and diagnostic performance

Per-class performance on the held-out half of the validation split is strong across all three classes (Table 10). XSS reaches the highest recall and the Normal class the highest support, so benign traffic is handled reliably. SQLi has the lowest recall, confirming that the Normal-SQLi boundary is the difficult region: benign inputs frequently contain SQL-like terms, quotes and identifiers without being malicious, and after deduplication the easy repeated templates that previously inflated this class are gone. SQLi mitigation therefore requires careful thresholding to balance unsafe false negatives against excessive false positives.

Table 10. Per-class performance on val_report

Note: Cross-Site Scripting (XSS); SQL Injection (SQLi)

With the temperature, thresholds and margin frozen, the selected configuration is evaluated on the held-out clean test split of 27,109 requests. It reaches macro precision 0.9867, macro recall 0.9857, macro-F1 0.9862 and accuracy 0.9851, closely matching val_report and confirming that the operating point transfers to unseen traffic.

These figures are lower than those reported in the previous version because of the corpus, not the model. Retraining the identical architecture on the deduplicated corpus moves clean-test macro-F1 from 0.9889 ± 0.0006 to 0.9858 ± 0.0006 over three seeds, a drop of 0.0031. The previously reported performance was therefore only mildly inflated by cross-split duplication. Still, the deduplicated figure is the one that generalises, and it is the figure reported throughout this section.

The clean-test confusion matrix (Table 11) locates the residual errors. The model maps 250 SQLi payloads to Normal and 95 Normal requests to SQLi, while XSS is almost perfectly separated. The Normal-SQLi boundary therefore accounts for the overwhelming majority of errors, which is what limits SQLi recall while XSS recall stays near unity.

Table 11. Clean-test confusion matrix

Note: Cross-Site Scripting (XSS); SQL Injection (SQLi)

Table 12 translates these predictions into the deployed actions. Under the blocking thresholds alone, the split resolves into 12,745 Allow, 14,364 Block and 0 Review, so the Review action never fires and 245 SQLi and 10 XSS payloads are allowed. This is the concrete cost of an inert third action, and it motivates the review margin introduced in Section 3.5: at τ_review = 0.30 the Review rate becomes 0.240%, 6 of the allowed SQLi payloads are withheld for inspection and 59 benign requests are diverted; at τ_review = 0.20 the figures are 1.808%, 43 and 444. Neither band is free, and the choice is an explicit operational trade-off rather than a modelling result.

Table 12. Allow/Block/Review distribution by true class

Note: Cross-Site Scripting (XSS); SQL Injection (SQLi)

Because the framework is positioned for JSON-oriented API traffic, Table 13 reports performance on the JSON-like slice of the clean test split. On the 3,917 JSON-like requests (14.4% of the split), macro-F1 is 0.9852, against 0.9863 on the non-JSON remainder, so detection does not degrade on JSON-structured inputs. Precision and recall are macro-averaged over the three classes.

Table 13. Clean-test performance on the JavaScript Object Notation (JSON)-like slice

5.4 Adversarial robustness and per-mutator ablation

The selected configuration is then attacked directly. The adversary of Section 4.5 is re-optimized against the deployed pipeline with its temperature and thresholds frozen, using 2,000 malicious seed payloads drawn from the deduplicated test split. A payload counts as evaded if any of the nine campaigns drives it to Allow.

Under this protocol, the selected configuration allows 80 of 2,000 seed payloads, an evasion rate of 4.00% with an exact 95% Clopper-Pearson upper bound of 4.95%. Under the weaker not-Blocked criterion, the rate is 4.10%. SQLi remains the more malleable class under mutation (Table 14). This replaces the zero-evasion result reported in the previous version (Table 15), which was obtained by re-scoring a frozen holdout generated against a weaker earlier model and is not evidence about an attacker who targets the deployed system.

Table 14. Adaptive evasion rate by configuration

Note: Cross-Site Scripting (XSS); SQL Injection (SQLi)

Table 15. The selected configuration under three evaluation protocols

One control guards against the result being an artifact of the seed pool: 92.4% of the 2,000 seeds contain attack syntax that can be verified independently of the corpus labels after repeated URL-decoding and HTML unescaping, and restricting the measurement to that subset gives an evasion rate of 4.17%, statistically indistinguishable from the pooled figure. The result therefore does not rest on the noisy records described in Section 4.1.

Each configuration is attacked through its own operating point, fitted on val_fit by the same protocol, so no configuration is handicapped by another's thresholds. The comparison isolates where the robustness comes from. Without JSON-aware canonicalization, the same architecture is evaded on 94.10% of seeds before fine-tuning and 80.30% after; with canonicalization the figures are 5.30% and 4.00%. Canonicalization removes roughly nineteen twentieths of the attack surface and fine-tuning removes about a quarter of what remains, so canonicalization is the dominant contributor and few-shot fine-tuning is a secondary refinement. Fine-tuning also helps far more when canonicalization is absent, which is consistent with it learning to absorb encoding variance that canonicalization otherwise removes outright.

The decision margins show that the successful evasions are a thin tail rather than a general weakness. Taking, for each seed payload, the highest P(Normal) reached anywhere in the campaign, the median is below 0.0001 and the 95th percentile is 0.3379, both far below the 0.40 Allow threshold, while the maximum reaches 0.9994. The distribution is therefore sharply bimodal: the attacker cannot move most payloads at all, but on the small minority it does defeat, it drives them well past the threshold rather than just over it (Figure 4). The aggregate rate alone conceals both halves of that behaviour, which is why the margin distribution is reported beside it.

The per-mutator ablation explains why chained search is needed at all. Each operator is applied singly to 2,000 malicious test payloads and compared against an unmutated control, so the control measures the model's plain false-negative rate on the same payloads (Table 16).

Table 16. Per-mutator ablation with an unmutated control

For the selected canonicalization-enabled configuration, the control allows 2.10% of the payloads, and no single mutator raises this materially: the strongest, SQL-comment insertion, reaches 2.35%, and several operators leave the rate unchanged. Single-operator mutation is therefore ineffective once canonicalization is applied, and the 4% reached in Section 5.4 is produced by chained mutation rather than by any individual transformation.

Without canonicalization, the picture is different: the same control allows 1.95% while random casing alone reaches 5.80% and double URL-encoding 2.55%. Representation-level operators are exactly what canonicalization neutralises, which is consistent with the configuration comparison in Table 14.

tu_pian_1.png

5.5 Latency and deployment feasibility

Latency is measured on the deployed path (canonicalization, inference, temperature scaling and the threshold decision) at batch size 1, since an API gateway scores one request at a time. Measurements are taken in-process rather than through an HTTP wrapper, so they exclude network and serialisation overhead and represent the model-side cost only.

At 0.5 KB the selected configuration reaches P50 = 4.0 ms and P95 = 6.8 ms on GPU, rising to P50 = 9.4 ms at 8 KB, with CPU figures of 6.5 ms and 12.9 ms at the same points (Table 17). Canonicalization takes 6.2 ms on average at 8 KB, so it rather than inference dominates at large payload sizes, and batched throughput reaches 1,426 requests per second. Both are well inside the budget of an inline gateway component.

Table 17. End-to-end latency by payload size

5.6 Tradeoffs, security interpretation and discussion

The clean-best and the robust-best configurations are not the same model, and the adaptive evaluation shows the size of the gap. Disabling canonicalization gains roughly two thousandths of clean macro-F1 and pays for it with an evasion rate of 80.30% against 4.00%, a factor of twenty. A model chosen on clean accuracy alone would therefore have been the wrong deployment choice by a wide margin, which is why the selected configuration is fixed on clean performance, adversarial behaviour, calibration and latency together rather than on macro-F1 alone.

The Review decision is the part of the framework that separates a classifier from a mitigation policy, but it only functions if it is reachable. Section 5.3 showed that under blocking thresholds alone it is not: it fires on 0 of 27,109 clean test requests, and every uncertain request is therefore silently allowed.

The review margin repairs this. At τ_review = 0.30, the deployed system routes 0.240% of clean traffic to Review, withholding 6 of the 245 SQLi payloads that the blocking thresholds would have allowed, at the cost of 59 benign requests. Widening the margin to 0.20 raises the recovery to 43 payloads for 444 benign diversions. The margin is thus a tunable security dial with an explicit and measurable price, rather than a mechanism that is nominally present but never active.

The results support four conclusions. First, class-specific thresholding with an explicit review margin converts a detector into an auditable mitigation policy, and the margin is necessary: without it the Review action is unreachable at the selected operating point.

Second, calibration belongs to the security pipeline rather than to post-processing, and it must be fitted and reported on disjoint data. Fitting the temperature and thresholds on one half of the validation split and reporting on the other costs 0.0023 macro-F1. That is a small quantity, but it is the part of the previously reported figure that was not independent evidence.

Third, the evaluation protocol dominates both conclusions. The same system reports no evasions under the transfer holdout used previously, 6.40% once that holdout is regenerated without the sampling defect described in Section 4.5, and 4.00% under an attacker re-optimized against the deployed pipeline; and more than half of the test split shared a payload template with training before deduplication, so any comparison drawn on such a split measures memorisation as much as generalisation.

Overall, the framework converts detector outputs into deployable security decisions, and the revised evaluation shows what that conversion is and is not worth: a well-calibrated, auditable policy with a measurable residual attack surface, rather than an impermeable defence.