A Leakage-Audited Prefix-Only Protocol for Session-Based Consumer Navigation Prediction in E-Commerce Clickstream Data

E-commerce platforms record consumer browsing as event logs of products, categories, and pages. These logs expose short-term intent, local browsing persistence, session progression and abandonment, and they support recommender engines, search ranking, adaptive interfaces, and marketing systems. Clickstream logs also allow consumer behavior to be studied without costly proprietary user studies or privileged access to corporate data. Recent surveys confirm that artificial-intelligence recommenders and interactive analytics remain central research themes in e-commerce personalization, user cognition and decision support [1-3].

Historical interaction structure has long been the principal resource for such systems. Matrix factorization established collaborative filtering at scale [4], and subsequent work moved towards sequence and session modeling through recurrent, attention-based and graph-based architectures [5-11]. The study [12] indicated that sequential recommendation remains an active field. Section 2 discusses these developments in detail.

A recurring difficulty in applied e-commerce analytics is that ambitious modeling and trustworthy evaluation are not equally easy to achieve. Studies conducted on proprietary production environments, on multimedia product representations or through live experimentation cannot usually be re-examined by third parties. Benchmark studies on public logs are therefore valuable, because reviewers and independent researchers can inspect the preprocessing steps, repeat the experiments and verify the reported results. Recent work argues along the same lines for transparent data, metrics, and evaluation environments in recommender research [13, 14].

Transparency alone, however, does not guarantee validity. A separate body of work shows that leakage, understood as the use of information that would not be available at prediction time, is a widespread and frequently undetected source of over-optimistic results in machine-learning-based science [15] and in offline recommender evaluation specifically [16]. The choice of data-splitting strategy alone can reorder the apparent ranking of competing models [17]. The risk is acute in clickstream settings, because several natural session descriptors are defined only once the session has ended. The completed length of a session, and the position of a click expressed as a fraction of that length, are the clearest examples. A benchmark that admits such descriptors measures hindsight rather than prediction.

This study therefore addresses one prediction target on public data. The target is not purchase but the next step in a consumer-navigation session. Given the current click and the clicks preceding it within the same session, the model predicts either the product category of the next click or the termination of the session.

This formulation preserves the realism of consumer navigation while ensuring that every conclusion can be verified on public data using ordinary academic computing resources. It also places the work within information-system design rather than model development alone. The primary artifact is a data-engineering protocol: a specification of which derived variables a session-based feature pipeline may legitimately compute, together with an audit procedure for verifying compliance. Such a specification bears directly on analytics governance, because the feature pipelines that serve offline benchmarks commonly also serve production scoring services, where a hindsight variable produces silent degradation rather than a visible error.

The contributions are ordered by weight. The primary contribution is a leakage audit for session-based clickstream benchmarks. The audit identifies the future-dependent variables that inflate next-step and exit prediction, contrasts performance obtained with and without those variables, and specifies a prefix-only protocol that excludes them. It is formalized as a reusable checklist and as an explicit algorithm, so that it can be applied to feature pipelines other than the one studied here. The second contribution is the packaging of this protocol as an open-data workflow covering preprocessing, leakage-safe target construction, temporal splitting, and reporting. The remaining contributions are empirical and subordinate to the protocol. The study reports hold-out results under a temporal split with class-specific metrics, ranking-aware metrics, a sequential baseline, and paired significance testing. It establishes that the prefix-safe session block produces a statistically reliable but practically small class-balanced improvement on this benchmark, of approximately one tenth the magnitude of the gain attributable to boosted tabular learners. It further verifies, across six random seeds and four rolling temporal splits, that the ratio between the two effects is a stable property of the benchmark rather than a feature of one particular run.

A pipeline-validation run on the YOOCHOOSE dataset from the Recommender Systems (RecSys) Challenge 2015 is also reported [18]. Its purpose is to confirm that the audit procedure transfers unchanged to a second public clickstream source, not to provide independent confirmation of the numerical results obtained on the primary dataset. Section 4.8 states the limits of that run explicitly.

3.1 Primary dataset

The primary dataset is a public clickstream benchmark recording anonymized browsing of an online clothing retailer. Each row represents one click event and carries calendar attributes, a session identifier, a country code, the current main category, the clothing model, the color, the position of the product photograph on the page, the photography type, the price, and the page indicator. The tabular release contains no personal identifiers, no product images, and no confidential customer records. These properties make the benchmark suitable for modeling sessions, categories, and short-term navigation.

Figure 1 supports the use of a temporal rather than a random split. The hold-out period remains large while being disjoint in time from the training period. This arrangement reproduces the temporal drift that a deployed model would encounter, and it avoids the optimistic estimates that follow when clicks from the same period are distributed at random across training and test sets [16, 17].

Table 1 summarizes the dataset. It contains 165,474 clicks drawn from 24,026 sessions recorded between April and August 2008, covering 47 country codes, four main product categories and 217 clothing models. The average is 6.89 clicks with a median of 4, so the distribution is dominated by short sessions with a long tail of longer, exploratory ones. Prices are retained in their original unit and average 43.80. The monthly record distribution is shown in Figure 1.

8-1.png

Figure 1. Monthly record distribution in the public clickstream dataset

Table 1. Public dataset characteristics

3.1.1 Quantifying temporal drift between the training and test periods

A temporal split is informative only if the two periods differ, so the magnitude of the shift is quantified rather than assumed. Three diagnostics are computed. The first is the population stability index of the main-category distribution between the training period and the test period. The second is the Jensen-Shannon divergence between the first-order category-transition matrices estimated separately on the two periods. The third is a calibration check: the expected calibration error of the best-performing model on the test period, compared with the same quantity measured on a held-out slice of the training period. Table 2 reports these values.

The distribution shift between the two periods is negligible by the conventional threshold: the population stability index is 0.0149, an order of magnitude below the 0.10 boundary. The Jensen-Shannon divergence between the transition matrices is 0.0575, likewise small. Calibration degrades slightly, with expected calibration error rising from 0.0419 within the training period to 0.0519 in the test period, but the increase is minor. The temporal split is therefore a conservative design choice rather than a stress test on this dataset: it eliminates the optimism that a random split would introduce, without imposing a substantial distribution shift. This has a bearing on how the hold-out results should be read. Because the two periods are distributionally similar, the reported figures approximate performance under stationarity, and they should not be presented as evidence of robustness under drift. Establishing that would require a benchmark whose periods actually differ, which this one does not supply.

Table 2. Temporal drift and calibration diagnostics between the training and test periods

3.1.2 Secondary dataset for pipeline validation: YOOCHOOSE / RecSys Challenge 2015

A second public benchmark is used to establish that the protocol is not specific to a single source. The YOOCHOOSE dataset from the RecSys Challenge 2015 contains anonymized click sessions from a large European e-commerce retailer, together with buying events for a subset of those sessions. The click file records a session identifier, a timestamp, an item identifier, and an item category; the buy file records a session identifier, a timestamp, an item identifier, a price, and a quantity [18].

The YOOCHOOSE events were transformed into the same prefix-only format used for the primary experiment. Events were ordered by session and by timestamp, and the predictors for each observed prefix were computed only from the current event and from events preceding it within the same session. Future-dependent information was excluded, including any indication of whether the current event is the final click of its session, the completed session length, and any post-click purchase record. The label was defined as the next item category or, where the category field proved degenerate, as continuation versus exit.

This second dataset is not introduced in order to raise the reported scores. Its function is to establish that the audit steps execute without modification on a larger and more heterogeneous session log.

3.2 Prediction task and leakage-safe target construction

The target is the main-category value of the next click within the session. The final click of each session is labeled Exit and assigned class 0, and every other click takes the main-category value of the click that follows it. The task is therefore a five-class classification problem with four continuation classes and one termination class. This formulation is operationally relevant, because continuation and termination are acted upon by the same downstream system.

Labels are produced by shifting the category values one step forward within each session. Leakage is prevented by a single constraint: a predictor may be computed only from the current row or from clicks already observed at the moment of prediction. No predictor may encode the next click, the final click, or the completed session length. Three variables are excluded for this reason. The first is a final-click marker, which reveals the exit label directly. The second is completed session length, which is unknown while a session is in progress. The third is relative click position expressed as a fraction of completed session length, which encodes the same information indirectly. Prefix variables remain admissible because they are known before the next click occurs. These include the current click, the observed prefix length, the previous category, the previous page, the count of earlier clicks on the same category, and the count of earlier clicks on the same model. This distinction is the substantive content of the protocol. Where it is not enforced, clickstream results are inflated by information transferred from the target step or from later clicks.

3.3 Feature engineering strategy

The feature set comprises two blocks. The base block describes the current click through month, day, click order, country, current main category, current clothing model, color, photograph location, photography flag, price, secondary price indicator, and page indicator. These variables characterize the current click without summarizing the observed history.

The session block summarizes the visit up to and including the current click. It contains the observed prefix length, an entry-click flag, the count of prior clicks on the same model, the count of prior clicks on the same category, the current-category run length, the previous page indicator, the previous main category, the category two steps back, the previous price, the price delta, the prefix mean price, and the prefix minimum price. Where a normalized position is required, it is defined against a quantity observable in real time, such as a training-set maximum, and never against the length of the current session. The representation is deliberately compact and interpretable. In place of a sequence encoder or multimodal input, it supplies classical learners with a structured summary of progression, repetition, and local state, while preserving the online prediction setting.

8-2.png

Figure 2. End-to-end methodology pipeline from raw data to evaluation artifacts

Figure 2 depicts the pipeline. It reads the public Comma-Separated Values (CSV) file, orders clicks within sessions, constructs the next-step target, extracts leakage-safe features, splits the chronologically ordered data into training and test sets, trains the baseline and proposed models, and reports hold-out metrics together with interpretation artifacts.

The same stages were applied to YOOCHOOSE. Events were ordered within sessions, the next-step or exit label was constructed, prefix-only features were extracted, a chronological split was applied, and classification, ranking-aware, and leakage-audit metrics were reported. The second dataset is therefore a direct application of the protocol rather than a separate experiment.

3.3.1 Group-wise ablation design for the session block

Because the session block as a whole produces a gain an order of magnitude smaller than the learning algorithm does, the aggregate figure is decomposed rather than reported as a single figure. The block contains three functionally distinct groups. Progression variables record how far the session has advanced and comprise the observed prefix length and the entry-click flag. Repetition variables record returning behavior and comprise the prior same-model count, the prior same-category count and the current-category run length. Transition variables record the immediately preceding state and comprise the previous page, the previous main category, the category two steps back, the previous price, the price delta, the prefix mean price and the prefix minimum price. A group-wise ablation was conducted in which each group was first added to the base block in isolation and then removed from the complete session block, so that both the marginal and the unique contribution of each group is visible. Table 3 reports the resulting macro-F1 values.

Table 3. Group-wise ablation of the prefix-safe session block (random forest, macro-F1)

The decomposition is more informative than the aggregate. Added in isolation, repetition variables raise macro-F1 from 0.6149 to 0.6223, a gain of +0.0074, whereas progression contributes +0.0015 and transition +0.0013. The complete block, however, reaches only 0.6170. The isolated gains therefore do not accumulate, and the block as a whole performs worse than its strongest component. The removal columns identify the cause: dropping transition variables from the complete block raises macro-F1 to 0.6224, the best figure in the table, while dropping repetition lowers it to 0.6152, close to the base block alone. Repetition carries the signal; transition variables dilute it. Progression is redundant with the base block by construction, since the source click-order field and the observed prefix length are the same quantity, as Section 6 notes. The practical implication is that a reduced session representation consisting of the repetition group alone would outperform the full session block reported here. That configuration is not adopted as the headline result, because selecting it on the basis of hold-out performance would convert the test period into a selection set; it is recorded as the specification a subsequent study should evaluate on fresh data.

3.4 Learning models and experimental protocol

Seven model configurations were evaluated. The first is a majority-class predictor that always returns the most frequent next class observed in training. The second is a first-order Markov transition baseline that estimates the next-category distribution from training-period transitions. The third and fourth are a decision tree and a random forest trained on the base block alone. The fifth is a random forest trained on the base block together with the prefix-safe session block. The sixth and seventh are gradient-boosted learners, Light Gradient Boosting Machine (LightGBM) and Categorical Boosting (CatBoost), trained on the same prefix-safe representation. Top-k accuracy and threshold-free metrics were computed for all of these configurations; they are evaluation criteria rather than additional models. This design separates the contribution of the session representation from the contribution of the learning algorithm.

Table 4. Hyperparameters of the learning models

Table 5. Feature groups used in the experiments

The decision tree used a maximum depth of 8 and a minimum leaf size of 50. The random forest used 200 trees, a maximum depth of 12, a minimum leaf size of 5, and balanced subsampling to offset class skew. The two boosted learners were added as stronger tabular models under the identical prefix-only protocol. LightGBM tests histogram-based gradient boosting on the encoded clickstream features, and CatBoost provides native handling of mixed categorical and numerical structure. Table 4 lists the exact hyperparameters used to produce the reported hold-out results, and Table 5 lists the feature groups.

The rolling-validation experiments in Section 4.7 use smaller ensembles than the main experiment: 120 trees for the forests and 300 for LightGBM. Two considerations motivate the difference. First, the rolling design fits twelve models rather than three, and ensemble size was reduced in order to keep the total training cost within the same computational budget. Second, the rolling analysis is a directional stability check on the sign and approximate magnitude of the gains, not the source of the headline figures. The base random forest reaches macro-F1 of 0.6135 on the month-7 test slice with 120 trees, against 0.6149 on the months 7–8 hold-out with 200 trees; the two test periods are not identical, so the comparison is indicative rather than exact, but it is consistent with limited sensitivity to ensemble size in this range. The absolute values obtained under the rolling design should nevertheless not be compared directly with those of the main hold-out. Only the within-split differences are comparable, and those are what the section reports.

Ordinal encoding was applied to the categorical fields so that training and test inputs share a consistent representation. Three properties of this choice affect comparability between models. First, the codes are arbitrary labels and carry no ordinal meaning; country code 12 does not stand in any ordered relation to country code 7. Second, axis-aligned tree learners are invariant to strictly monotone relabeling of an ordinal code, because a split of the form x ≤ k partitions the set of category values regardless of the numeric labels assigned to them. The encoding therefore cannot alter the hypothesis space available to the decision tree, the random forest, or LightGBM. It can affect which partitions are reachable within a limited depth, since an arbitrary ordering may require several splits to isolate a category group that a favorable ordering would isolate in one; this is one reason the trees are grown to depth 12 rather than to a shallower limit. Third, and most directly relevant to comparability with the sequential baseline, the Markov model does not consume the encoded matrix at all. It is estimated from the raw main-category identifiers as transition counts over an unordered state space, so no ordering is imposed on it, and the encoding cannot affect its estimates. CatBoost likewise uses its native categorical handling rather than the ordinal codes. The encoding step is thus confined to the tree learners whose hypothesis space it provably cannot alter.

Training used months 4–6, comprising 116,095 clicks, and testing used months 7–8, comprising 49,379 clicks. Classification performance is reported as accuracy, weighted F1, macro-F1 and balanced accuracy. Ranking-aware performance is reported as top-2 accuracy, top-3 accuracy, per-class average precision, macro PR-AUC and macro one-versus-rest area under the receiver operating characteristic curve (ROC-AUC). The ranking metrics matter because the operational task is not confined to a single hard prediction; it also covers prefetching, menu reordering and candidate ranking, all of which consume a ranked list rather than an arg-max label.

3.4.1 Choice of primary metric

Accuracy and macro-F1 rank the models differently in Table 6, so the choice of headline metric requires justification rather than convention. Macro-F1 is adopted here for three reasons. The first concerns the class distribution. The majority-class baseline reaches accuracy of 0.2453, so a model can appear competitive on accuracy while performing poorly on the least frequent classes; the same baseline reaches macro-F1 of only 0.0788, which exposes that behavior immediately. The second concerns the asymmetry of operational cost. The intended applications, namely next-category prefetching, menu reordering, and adaptive widget placement, act on a predicted class. A system that never predicts Exit incurs only a small accuracy penalty, because Exit is a minority class, yet it forfeits every intervention that depends on detecting a departing user. Macro-F1 weights each class equally and therefore tracks this cost structure more closely than accuracy does. The third concerns comparability between models. Accuracy on this task is largely determined by the two dominant transition patterns, which a first-order transition table already captures, so reporting accuracy alone would obscure precisely the differences this benchmark is designed to measure. Balanced accuracy and macro-ROC-AUC are reported alongside macro-F1 for the same reason. Accuracy is retained in every table so that readers who prefer it may apply it, and Section 4.1 states explicitly where the two metrics disagree and why.

Table 6. Hold-out performance on next-category prediction (months 7–8)

3.5 Sequential baseline and statistical validation

A first-order Markov baseline is evaluated on the same temporal split, in order to establish whether the tree learners capture regularities beyond simple category-to-category transition frequencies. The baseline estimates the probability of the next target class from the current main category, using transition counts observed in the training period alone. Let c_t denote the current category at click t, and let $y_{t+1} \in \mathrm{Y}$ denote the next target class, where Y contains the four continuation categories and the exit class. The Laplace-smoothed transition probability is computed as follows:

$P\left(y_{t+1} \mid c_t\right)=\left[N\left(c_t \rightarrow y_{t+1}\right)+\alpha\right] /\left[N\left(c_t\right)+\alpha \cdot|Y|\right.$         (1)

where, N(c_t→ y_t+1) is the number of training transitions from current category c_t to target class y_t+1, N(c_t) is the total number of transitions observed from c_t, |Y| = 5 is the number of target classes, and α = 1 is the Laplace smoothing parameter used in the reported experiments. For each test instance, the Markov baseline predicts the target class with the highest smoothed transition probability. The baseline is non-neural, interpretable, and fits with the context of a single-step prediction.

Paired statistical validation was performed on the same hold-out instances. McNemar's test with continuity correction was applied to paired accuracy differences, and paired bootstrap resampling was applied to macro-F1 and balanced accuracy, because these metrics are nonlinear and sensitive to class imbalance. The bootstrap used 2,000 resamples drawn with replacement at the level of the test instance, with the random seed fixed at 42. Reported intervals are percentile intervals at the 95% level. Because clicks belonging to one session are not independent observations, every macro-F1 interval was recomputed with a session-level clustered bootstrap that resamples complete sessions rather than individual clicks, and both sets of intervals are reported in Table 7. They agree closely: for the session block, the click-level interval is (0.0001, 0.0042) against a clustered interval of (0.0002, 0.0042), and across the three comparisons the clustered intervals are between two and eight percent narrower rather than wider, a difference within Monte Carlo error. The likely reason is that sessions in this benchmark are short, with a median of four clicks, so the effective number of independent units is close to the number of test instances. Every conclusion drawn below holds under either resampling unit. The discordant-pair counts underlying each McNemar test are given in Table 7, so that every reported statistic can be recomputed from the published tables. The comparisons of primary interest are not confined to the session block against the base random forest; they include each boosted learner against the same reference, in order to test whether the learning algorithm rather than the representation delivers a reliable gain.

Table 7. Paired significance tests against the base random forest (months 7-8 hold-out)

3.6 Leakage-audit protocol and reproducibility checklist

Algorithm 1 states the protocol formally, so that it can be applied to feature pipelines other than the one used here. The input is an event log carrying a session identifier, an ordering key, and an event attribute set; the output is a design matrix in which every column is verifiable as prefix-computable. Steps 1 to 3 construct the target. Steps 4 to 8 build the feature matrix under the prefix constraint. Step 9 is the audit itself: each candidate column is tested against an admissibility condition, and any column whose value would change if the session were truncated at the current click is rejected. Step 11 fixes the temporal split. The condition in Step 9 is the operative definition of a leakage-safe clickstream feature, and it is stated independently of the particular variables used in this study.

The complete set of hold-out metrics produced by this protocol is reported in Section 4 and is not restated here.

Section 4.2 reports the quantitative contrast between a diagnostic run that admits the rejected variables and the audited run that excludes them. That contrast is the empirical justification for the protocol.

The application scope is narrow, but it is real. A next-step category model can support low-cost interventions such as category prefetching, navigation-menu reordering, adaptive widget placement, and history-aware exploration prompts. Because the model consumes clickstream fields alone, it is cheaper to implement and to monitor than multimodal alternatives that require image pipelines and large-scale ranking infrastructure. Evidence that users respond to personalized recommendations while browsing supports the value of such interventions, although that literature addresses perception and intention rather than next-step prediction, and it is cited here as motivation rather than as method [29, 30].

The protocol also has a role at the level of system design, which is where its contribution to information-system practice lies. In a production analytics stack, session features are typically computed once and served both to offline evaluation and to online scoring. The admissibility condition in step 9 of Algorithm 1 can be implemented directly as a validation rule within the feature store: any derived column whose value changes when the session is truncated at the current event is rejected before it reaches either consumer. This converts the audit from a manual review step into an automated contract between the feature pipeline and the models that depend on it, which is the form in which it is most likely to be enforced in practice. The same rule doubles as a documentation artifact, since the list of rejected columns records why an audited benchmark reports lower figures than an unaudited pipeline would.

The workflow also has value for teaching and for peer review. Public data allow preprocessing choices, evaluation procedures, and reported metrics to be re-examined independently. The manuscript therefore contributes an experimental workflow for transparent work with navigation data, and not only a set of results.

The YOOCHOOSE component connects the protocol to a larger public session log carrying both click and purchase signals, which allows subsequent work to examine whether prefix-only modeling behaves comparably under category-navigation and purchase-oriented objectives [18].

More broadly, the study argues for a particular standard of evidence in applied machine learning. Claims resting on production A/B tests, conversion optimization, enterprise privacy management, or multimodal customer profiling are attractive but rarely replicable outside the originating organization. Benchmark evidence is less striking, but it is more persuasive when the claims are confined to what is observable and reproducible.

This study specifies a prefix-only protocol for session-based clickstream benchmarking, formalizes it as an auditable algorithm, and applies it to next-category and exit prediction on public e-commerce data. The evaluation covers a majority baseline, a first-order transition model, tree ensembles, and two gradient-boosted learners, and reports class-balanced, ranking-aware, and threshold-free metrics under a strict temporal split. Every figure derives from a single documented pipeline under one declared seed, with stability verified across six seeds.

The audit is the principal contribution. Admitting variables that are defined only once a session has ended recovers the termination class perfectly, a result unattainable from information available before the next click, while moving the aggregate scores by a margin that a genuinely better model might plausibly produce. That asymmetry is what makes the audit useful in practice: it tells reviewers and practitioners which figure to distrust first, and the admissibility condition stated in Algorithm 1 gives them a means of preventing the problem, whether applied to a benchmark or embedded as a validation rule in a production feature pipeline.

The empirical findings are secondary to the protocol and are correspondingly modest. Boosted tabular learners improve class-balanced and ranking quality over bagged trees by a margin that is small in absolute terms, statistically reliable, and stable across seeds and across rolling temporal splits. The prefix-safe session block also contributes, but by roughly one tenth as much, and its contribution is concentrated in variables recording repetition rather than progression or transition. The transition baseline achieves the highest raw accuracy while ranking last among the non-trivial models on class-balanced measures, which illustrates why the choice of headline metric requires justification in tasks with skewed class distributions.

One methodological observation deserves emphasis beyond this dataset. Accuracy and macro-F1 responded to different interventions here, and the paired tests appropriate to each disagreed accordingly: the configuration with the largest macro-F1 gain was statistically indistinguishable from the baseline in accuracy, while the configuration with the clearest accuracy advantage barely moved macro-F1. Session termination likewise remains the hardest element of the task under honest features, and the manuscript reports it as such rather than concealing it behind an aggregate score. A pipeline-validation run on a second public source confirms that the audit executes unchanged on a different schema, though a full run with a non-degenerate category field is still required before cross-dataset claims are made. Taken together, the protocol, the audit, and the stated limits form a reusable basis for clickstream benchmarking in which the reported figures correspond to what a deployed system could actually know at the moment of prediction.