Trajectory Analysis of Learning Processes for Personalized Education: Deep Spatiotemporal Embedding and Discovery of Learning Behavior Patterns

2.1 Problem definition

This chapter aims to address the problem of trajectory analysis and behavior pattern discovery for dual-phase learning process images, providing clear mathematical definitions and semantic meanings for the input, intermediate representations, and output, setting clear objectives for the subsequent technical framework design. The input to this problem consists of two learning process image pairs captured at different moments $T_1$ and $T_2: I_1 \in \mathrm{R}^{H \times \pi \times 3}$ and $I_2 \in \mathrm{R}^{H \times W \times 3}$, where $H$ and $W$ represent the image height and width, respectively, and 3 is the number of RGB channels. The image contents are visualized encoded results of learning trajectories, such as eye movement focus and handwritten operations. Based on this input, the model first generates two key intermediate outputs: First, the deep spatiotemporal embedding vector of the learning trajectory $E \in \mathrm{R}^{N \times D}$, where $N$ is the total number of trajectory sampling points, and D is the embedding dimension. The vector must encode both the spatiotemporal position information and semantic association features of the trajectory; Second, the trajectory segment set S={S₁,S₂,...,S_M}, where M is the number of segments, and each segment corresponds to a continuous trajectory segment with unified semantics. The final output of the problem includes two types of results: First, the learning behavior change map $C M \in \mathrm{R}^{H \times W \times 1}$, which represents the spatial distribution of learning behavior changes between the two moments at the pixel level; second, the learning behavior pattern label set L={l₁,l₂,...,l_K}, where K is the number of pattern categories, and the labels must accurately correspond to typical learning behavior patterns with educational semantics. Figure 1 shows the complete model architecture for learning trajectory analysis based on parameter-sharing dual-branch STv2 + multi-scale differential fusion.

1.png

Figure 1. Model architecture for learning trajectory analysis based on parameter-sharing dual-branch STv2 + multi-scale differential fusion

2.2 Parameter-sharing dual-branch Swin Transformer V2 encoder

The core function of the parameter-sharing dual-branch STv2 encoder is to perform multi-scale feature extraction and trajectory-oriented representation for dual-phase learning process images. Its design centers on ensuring feature consistency across the dual-phase and encoding multi-granularity trajectory information. To address the trajectory discontinuity issue caused by cross-phase feature space shifts in traditional dual-branch structures, the encoder adopts a connected network architecture, where both branches share the complete STv2 backbone network weights. This design enforces feature extraction of dual-phase images within a unified semantic space by binding the weights, thus providing the basis for subsequent temporal continuity modeling of trajectories, while also reducing model parameters, laying the foundation for lightweight deployment.

The encoder employs a four-stage progressive feature extraction process, which uses downsampling and module stacking to achieve multi-granularity encoding of trajectories from micro to macro levels. The input images are first converted into a sequence of image patches using the 4×4 Patch Partition operation, then processed in the first stage: Three STv2 modules are applied to the image patch sequence for feature encoding, generating scale 1 features with a resolution of H/4×W/4. These features focus on the spatial positions and morphological information of micro trajectories, such as eye movement points and pen tip landing points. After the first stage, subsequent stages perform 2x downsampling via Patch Merging operations, while adjusting the number of stacked STv2 modules based on the granularity of trajectory information: The second stage stacks three STv2 modules, generating scale 2 features with a resolution of H/8×W/8, encoding the associations of local continuous trajectory segments; The third stage stacks four STv2 modules, generating scale 3 features with a resolution of H/16×W/16, primarily modeling the logical associations of trajectories, such as from the problem stem to the options, or from the formula to the calculation region. The increased number of modules is due to the complexity of trajectory logical associations requiring deeper feature interactions; The fourth stage stacks three STv2 modules, generating scale 4 features with a resolution of H/32×W/32, encoding global trajectory distribution patterns, such as the spatial transition features between focused and scattered regions. The multi-scale feature pyramid formed after the four stages fully covers the microscopic morphology, local associations, and global distribution information of the trajectory.

2.2.1 Architecture design

The architecture design of the parameter-sharing dual-branch STv2 encoder is based on the core principles of "trajectory continuity modeling" and "multi-granularity information adaptation," achieving a balance between the precision and efficiency of feature extraction through structural constraints and process optimization. The design logic of the dual-branch connected structure stems from the core requirement of dual-phase trajectory analysis—the comparability of features across phases: If the two branches use independent weights, the feature expressions of the same trajectory pattern at different moments would be misaligned, leading to misjudgments of key trajectory turning points. The weight-sharing mechanism ensures that dual-phase images generate features under the same feature extraction rules, maintaining feature consistency for the same trajectory pattern and providing a reliable basis for subsequent trajectory temporal evolution analysis. The selection of the STv2 backbone is based on the advantages of its window-based self-attention mechanism, which can efficiently capture the local continuity of trajectories within a window through attention calculations and, simultaneously, achieve global information interaction via window shifting operations, thus adapting to the local clustering and global migration characteristics of trajectories.

The core window self-attention calculation of the STv2 module is shown in formula (1). This calculation divides the feature map into non-overlapping windows and performs attention interaction within the windows, ensuring the efficient capture of local trajectory continuity while reducing computational complexity.

$W-M S A(X)=\operatorname{Softmax}\left(\frac{Q K^T}{\sqrt{d_k}}+M\right) V$                    (1)

where, X is the input feature of the STv2 module, Q, K, and V are the query, key, and value matrices, respectively, dk is the dimension of the query matrix, and M is the mask matrix used to avoid attention interference from invalid pixels within the window. Through this calculation, the module can efficiently aggregate the local trajectory features within the window, such as the associations between adjacent eye movement points and continuous handwritten trajectories.

The module configuration and resolution design of the four-stage feature extraction process are optimized to adapt to the characteristics of trajectory information. The downsampling strategy of Patch Partition and Patch Merging uses fixed ratios of 4×4 and 2×2, ensuring reasonable compression of feature dimensions while avoiding excessive loss of trajectory spatial position information. The differentiated configuration of the number of STv2 modules in each stage follows the principle of "trajectory information complexity matching": The information structure of micro and global trajectories is relatively simple, and three modules are sufficient for full encoding, while the logical associations in trajectories, which involve multiple regions and steps, require four layers of modules for deep iteration to achieve feature fusion, ensuring the completeness of feature representation for logical chains such as "problem analysis - computation - answering." This differentiated configuration ensures feature quality while avoiding redundant computation and improving the overall efficiency of the encoder.

2.2.2 Spatiotemporal embedding mechanism

The core goal of the spatiotemporal embedding mechanism is to convert the multi-scale features output by the encoder into trajectory embedding representations that incorporate both temporal continuity and spatial significance, realizing the semantic transformation from "image features" to "trajectory features." This mechanism strengthens the spatiotemporal correlation representation of the features through three progressive steps: temporal embedding, spatial embedding, and spatiotemporal fusion, providing core inputs for subsequent trajectory reconstruction and pattern discovery.

The temporal embedding step focuses on encoding the temporal evolution information of dual-phase trajectories, capturing dynamic changes between the two moments' trajectories through differential operations. For dual-phase features F₁^s and F₂^s at each scale s=1,2,3,4, the temporal difference feature ΔF^s=F₂^s−F₁^s is computed element-wise. The physical meaning of this differential operation is to quantify the feature change at the same spatial location between different moments, directly corresponding to the temporal migration of the trajectory, such as the change in the eye movement focus from the question stem to the options or the movement of the pen tip from the blank area to the calculation area. By generating multi-scale differential features in parallel, the temporal evolution of trajectories at different granularities is synchronized and encoded, preserving fine motion information for micro-trajectories and capturing global migration patterns for macro-trajectories.

$\Delta F^s=F_2^\delta-F_1^\delta$                    (2)

where, $\Delta F^s \in \mathrm{R}^{H s \times W s \times C s}$ is the temporal difference feature at scale s, and H_s, W_s, and C_s represent the height, width, and number of channels of the feature at scale s, respectively.

The spatial embedding step employs a spatial attention mechanism to enhance the features of key trajectory areas by quantifying the importance of spatial positions, increasing the feature weights of densely distributed trajectory areas. The calculation of spatial attention weights is shown in formula (3). The feature map is globally pooled and non-linearly transformed to generate an attention weight map of the same size as the feature map, which is then multiplied element-wise with the original features to obtain spatially enhanced features.

$A^s=\operatorname{Sigmoid}\left(F C\left(G l o b a l A v g P o o l\left(F^6\right)\right)\right.$                    (3)

$F_{s p a}^s=F^s \odot\left(A^s\right)$                     (4)

where, F^s is the original feature at scale s, GlobalAvgPool( ) is the global average pooling operation, FC( ) is the fully connected layer, Sigmoid( ) is the activation function, A^s is the spatial attention weight map, and ⊙ represents element-wise multiplication, F_spa^s is the feature at the s-th scale after spatial enhancement. The process assigns higher weights to key regions such as the formula region and the question stem area, while suppressing background noise interference.

The spatiotemporal fusion step introduces a temporal attention gating unit to achieve collaborative integration of temporal and spatial information, focusing on reinforcing the feature representation of key trajectory turning points. The gating unit dynamically adjusts the weights of the differential features at different moments, allowing the model to focus on key events such as attention shifts and strategy changes. The calculation of the temporal attention gating is shown in formula (5):

$G^s=\operatorname{Sigmoid}\left(\operatorname{Conv}\left(\Delta F^s \oplus F_{s p a}^s\right)\right)$                (5)

$F_{s t}^s=G^s \odot \Delta F^s+\left(1-G^s\right) \odot F_{s p a}^s$                     (6)

where, ⊕ is the channel-wise concatenation operation, Conv( ) is a 1×1 convolution layer for dimension adjustment, G^s is the temporal attention gating coefficient, and F_st^s is the spatiotemporal fused feature at scale s. Through this gating mechanism, the temporal evolution information from the temporal embedding and the key area information from the spatial embedding are deeply fused. The resulting multi-scale spatiotemporal embedding features contain both the temporal information of "when it changes" and the spatial information of "where it changes," providing accurate and semantically rich feature support for subsequent trajectory reconstruction and behavior pattern discovery.

2.3 Multi-Scale Difference Fusion Module (MDFM)

MDFM is the core hub connecting the parameter-sharing dual-branch STv2 encoder and the subsequent trajectory analysis modules. Its primary goal is to aggregate the four-stage dual-phase features output by the encoder, enhance trajectory change information through differential operations, and then use multi-scale fusion and cross-scale associations to achieve deep enhancement of trajectory semantics. The design logic of this module is based on the multi-granularity nature of learning trajectories—fine shifts in micro-trajectories, continuous associations in local trajectories, and distribution changes in global trajectories must be captured simultaneously. Traditional single-scale fusion or simple concatenation cannot achieve the collaborative representation of multi-granularity information. MDFM fills this gap with a two-stage design of "differential initialization - trajectory-aware fusion." Figure 2 shows the schematic diagram of the complete MDFM architecture.

2.png

Figure 2. Schematic diagram of the MDFM architecture

2.3.1 Differential feature initialization

The core task of differential feature initialization is to transform the dual-phase multi-scale features into an initial representation of trajectory changes by quantifying the spatial differences between the features at two moments, directly mapping the position shifts and morphological changes of the trajectory. This process provides "change-directed" base features for subsequent fusion. The design of this process is based on the dynamic nature of trajectories—core information in learning trajectories lies in the "change from T₁ to T₂," such as the movement of the eye focus from the question stem to the options, or the movement of the pen tip from the blank area to the calculation area. These changes can be accurately captured by the differential operation on dual-phase features.

Specifically, for the dual-phase features F₁^s and F₂^s (at time T2) output by the encoder at each scale s=1,2,3,4, the element-wise absolute difference is calculated to obtain the differential features F_d^s, as shown in formula (7). Here, $F_d^s \in \mathrm{R}^{H s \times W s \times C s}$, where Hs, Ws, and Cs are the height, width, and channel number of the feature at scale s. The element-wise absolute difference operation effectively retains the change magnitude information and suppresses background noise shared by both moments. The F_d^s at different scales corresponds to trajectory changes at different granularities: F_d¹ at scale 1 focuses on pixel-level shifts in micro-trajectories, such as the eye movement points or pen tip locations; F_d⁴ at scale 4 captures the macroscopic changes in global trajectory distributions, such as the transition from focused regions to scattered regions. The parallel generation of multi-scale differential features ensures complete coverage of trajectory change information from micro to macro, providing comprehensive initial input for subsequent multi-granularity fusion.

$F_d^s=F_1^s-\left(F_2^s\right)$                      (7)

The advantage of differential feature initialization lies in the "unbiased retention of change information"—compared to attention-based weighted differences or thresholded differences, the element-wise absolute difference avoids distortion of the change magnitude by arbitrary parameters, fully retaining the original intensity information of trajectory changes. For example, the differential responses to small adjustments or large migrations of the eye focus will naturally show intensity differences, which serve as a natural basis for "focusing on significant changes" in the subsequent multi-scale enhancement and also lay the foundation for identifying key turning points in trajectories.

2.3.2 Trajectory-aware multi-scale fusion

Trajectory-aware multi-scale fusion is the core process of MDFM. Through "multi-scale convolution enhancement - cross-scale trajectory association - residual fusion," the initial differential features are transformed into trajectory-enhanced features F_fusion that incorporate both multi-granularity information and semantic associations. The innovation of this step lies in introducing a "trajectory-guided" fusion logic, breaking the limitations of traditional multi-scale fusion, where "scales are processed independently," and enabling semantic association across different granularities of trajectory information.

Multi-scale convolution enhancement adapts feature extraction to the different needs of trajectory change at various granularities by applying parallel heterogeneous convolution kernels. It creates "micro-local-global" three-level feature enhancement channels for each scale's differential feature F_d^s. Specifically, three convolution operations with kernel sizes of 3×3, 5×5, and 7×7 are applied to F_d^s, as shown in formula (8). Here, Conv(k×k) represents the convolution operation with a kernel size of k×k, and F_d1^s, F_d2^s, and F_d3^s are the enhanced features output by these three convolutions. The 3×3 convolution focuses on the fine details of micro-trajectories, such as the small movement of the pen tip; the 5×5 convolution captures the continuous associations of local trajectories, such as the continuous movement of the eye focus in the question stem area; and the 7×7 convolution extracts the distribution pattern of global trajectories, such as the transition from the "question analysis - computation - answering" regions. After parallel convolution, multi-scale enhanced features F_enh^s=[F_d1^s,F_d2^s,F_d3^s] are obtained by concatenating, achieving aggregation of multi-granularity information within each scale.

$\begin{gathered}F_{d 1}^s=\operatorname{Conv}\left(3 \times 3, F_d^s\right), F_{d 2}^s =\operatorname{Conv}\left(5 \times 5, F_d^s\right), F_{d 3}^s=\operatorname{Conv}\left(7 \times 7, F_d^s\right)\end{gathered}$                    (8)

Cross-scale trajectory association is implemented through a cross-scale attention mechanism that achieves semantic binding between features from different scales, solving the problem of "decoupling micro-trajectory details from macro-trajectory semantics." The semantic interpretation of learning trajectories requires multi-scale association, such as the "pen tip's stay in the formula area" corresponding to "focusing on the solving strategy." This association needs to be realized through cross-scale feature interaction. Specifically, the enhanced feature F_enh⁴ at the global scale serves as the "semantic-guided feature" for computing attention weights for the features at other scales F_enh^s. First, F_enh⁴ is converted into a global semantic vector G through global average pooling and a 1×1 convolution. Then, G is used to calculate attention weights for each scale's F_enh^s, resulting in the weight vector A^s. Finally, the features at each scale are semantically calibrated by weighted summation, as shown in formula (10), where GAP( ) is the global average pooling operation, and Sigmoid( ) is the activation function, and ⊙ represents the element-wise multiplication:

$G=\operatorname{Conv}\left(1 \times 1, \operatorname{GAP}\left(F_{e n\mathrm{~h}}^4\right)\right)$                   (9)

$A^{\mathrm{s}}=\operatorname{Sigmoid}\left(\operatorname{Conv}\left(1 \times 1, G \oplus F_{e n\mathrm{~h}}^s\right)\right), F_{a l t}^{\S}=A^{ \pm} \odot F_{e n\mathrm{~h}}^s$                  (10)

Residual fusion retains the original change information from the initial differential features by using residual connections, avoiding information loss during multi-scale enhancement and cross-scale association processes. The features F_att^s, weighted by attention, are upsampled to the resolution of scale 1, then concatenated to form the fusion feature F_concat. Subsequently, F_concat is residual added to the initial differential feature F_d¹ at scale 1 to obtain the final trajectory semantic-enhanced feature F_fusion, as shown in formula (11). The introduction of residual connections can effectively alleviate the vanishing gradient problem caused by deep fusion while ensuring the collaborative retention of both original change information and enhanced semantic information, providing high-quality feature input with both "change precision" and "semantic depth" for subsequent trajectory embedding decoding.

$F_{\text {fusion }}=\operatorname{Conv}\left(1 \times 1, F_{\text {concat}}\right)+F_d^1$                    (11)

2.4 Learning trajectory analysis and behavior pattern discovery module

The Learning Trajectory Analysis and Behavior Pattern Discovery Module is the core link between feature extraction and educational semantic interpretation. Its main goal is to transform the multi-scale fused features into interpretable learning trajectory representations and to infer behavior patterns with educational practical value from the trajectories. This module employs a two-stage design of "trajectory spatiotemporal embedding decoding - behavior pattern discovery," first achieving the semantic transformation from features to trajectories, and then completing the induction from trajectories to patterns through segmentation, clustering, and labeling processes, ultimately establishing an accurate mapping between technical features and educational semantics.

2.4.1 Trajectory spatiotemporal embedding decoding

The core task of trajectory spatiotemporal embedding decoding is to convert the multi-scale fused features F_fusion into trajectory-level embedding representations that combine spatiotemporal continuity and semantic consistency, solving the problem of semantic disconnection between features and trajectories, and providing a structured trajectory representation for subsequent analysis. This process consists of two progressive steps: embedding vector generation and trajectory line reconstruction, respectively achieving the transformation from "features to trajectory points" and "trajectory points to trajectory lines."

In the embedding vector generation phase, the semantic mapping and dimensional adaptation of the features are implemented through a fully connected layer. Although the multi-scale fused feature F_fusion has aggregated the spatiotemporal and semantic information of the trajectory, it still exists in the image feature space and needs to be linearly mapped to trajectory point-level embedding vectors. Specifically, F_fusion first undergoes global average pooling to extract global features, and then passes through a two-layer fully connected network, ultimately outputting trajectory point-level embedding vectors (e_i∈R^D, i=1,2,...,N), where N is the number of trajectory points and D is the embedding dimension. This mapping process is shown in formula (12), where GAP( ) is the global average pooling operation, ReLU( ) is the activation function, and (W₁, b₁) and (W₂, b₂) are the weights and biases of the two fully connected layers. Through this design, e_i simultaneously encodes the spatiotemporal location and semantic association of the trajectory points, achieving a deep representation of the trajectory points.

$e_i=W_2 \cdot \operatorname{ReLU}\left(W_1 \cdot \operatorname{GAP}\left(F_{\text {fusion}}\right)+b_1\right)+b_2$  (12)

In the trajectory line reconstruction phase, DTW is used to address the issue of temporal misalignment caused by uneven sampling of trajectory points, and to aggregate discrete trajectory points into a complete trajectory line. During the learning trajectory collection process, the sampling frequency of devices such as eye trackers and handwriting boards is often influenced by the operation rhythm, leading to differences in sampling density at different moments. Direct concatenation would disrupt the temporal continuity of the trajectory. DTW solves this issue by calculating the optimal alignment path of the trajectory point embedding vectors for adaptive temporal matching. First, an N×N distance matrix D_dtw is constructed, where each element D_dtw(i,j) is the Euclidean distance between the trajectory point e_i and e_j. Then, dynamic programming is used to calculate the cumulative distance matrix Cdtw and backtrack to find the alignment path with the smallest cumulative distance, as shown in formula (13), where C_dtw(i,j) is the minimum cumulative distance for the first i and j trajectory points. After DTW alignment, the discrete points e₁,e₂,...,e_N are reconstructed into a complete high-dimensional trajectory embedding representation E∈R^N×D, which retains both the temporal continuity and the semantic features of each trajectory point.

$\begin{gathered}C_{d t w}(i, j)=D_{d t w}(i, j)+\min \left\{C_{d t w}(i-1, j), C_{d t w}(i, j-1), C_{d t w}(i-1, j-1)\right\}\end{gathered}$                  (13)

2.4.2 Behavior pattern discovery process

The behavior pattern discovery process takes the reconstructed trajectory embedding E as input and uses a three-step design: "trajectory segment segmentation–improved DBSCAN clustering–semantic labeling verification" to inductively discover interpretable behavior patterns from the trajectories. The core innovation of this process lies in optimizing segmentation and clustering strategies to account for the density heterogeneity and semantic relevance of learning trajectories, ensuring the accuracy and educational interpretability of the patterns.

The trajectory segment segmentation phase uses the semantic similarity of embedding vectors to semantically segment the trajectory, dividing the complete trajectory into continuous segments with unified educational semantics. The semantic features of learning trajectories exhibit local consistency—such as the highly similar semantics of trajectory points during the "question analysis" phase—while the transition from "question analysis" to "computation" will accompany a semantic mutation. Therefore, the cosine similarity of the trajectory point embedding vectors can be used to determine semantic continuity. Specifically, the cosine similarity Sim(e_i,e_i+1) between adjacent trajectory points e_i and e_i+1 is calculated, as shown in formula (14). When the similarity falls below a preset threshold θ, the trajectory is considered to have a semantic boundary, and segmentation is performed. The optimization of threshold θ is based on expert-annotated trajectories: using a validation set of 300 annotated trajectories, a grid search is performed over θ∈[0.5, 0.9], the θ with the highest segment segmentation accuracy is selected as the final threshold. After segmentation, a trajectory segment set S={S₁,S₂,...,S_M} is obtained, with each segment corresponding to a single educational semantic, such as "question analysis," "computation," or "checking."

$\operatorname{Sim}\left(e_i, e_{i+1}\right)=\frac{e_i \cdot e_{i+1}}{\left\|e_i\right\| \cdot\left\|e_{i+1}\right\|}$                   (14)

The improved DBSCAN clustering phase addresses the issue of traditional density clustering's poor adaptability to trajectory density heterogeneity by implementing an adaptive radius design for precise clustering of candidate behavior patterns. The density distribution of learning trajectories is significantly heterogeneous, such as high-density points in the "focused computation" region and low-density points in the "random exploration" region. A fixed radius parameter can result in over-segmentation in high-density areas or missing clusters in low-density areas. To address this, we propose an adaptive radius strategy based on trajectory density: for each trajectory segment S_m, the average Euclidean distance to its K-nearest neighbors is calculated as the local density radius r_m for that segment, and the median of all r_m values is taken as the clustering radius Eps. The core point threshold MinPts is then set proportionally to the total number of segments. During clustering, the average embedding vector e_m of each trajectory segment is used as the clustering unit, and similarity between segments is measured using the Euclidean distance. The final output is a set of candidate behavior patterns P={P₁,P₂,...,P_K}.

The semantic labeling and verification phase involves expert participation and quantitative optimization to assign educational semantics to the candidate patterns and calibrate parameters, ensuring the practical value of the patterns. This phase follows a closed-loop process of "expert labeling - consistency verification - parameter optimization": first, three education experts with over five years of teaching experience are invited to label the candidate patterns P based on characteristics such as spatial distribution and temporal rhythm of the trajectory segments. The labeling categories include goal-oriented, random exploration, hesitation, etc. Then, the consistency of labeling is verified using the Kappa coefficient, with Kappa≥0.75 considered consistent. If consistency is not reached, expert discussions are used to reach a consensus. Finally, the labeled results are used as ground truth to construct a confusion matrix to analyze clustering errors, and parameters K and MinPts are adjusted to minimize clustering error rates. After optimization, the final set of learning behavior pattern labels L={l₁,l₂,...,l_K} is obtained. This label set not only aligns with the objectivity of technical clustering but also provides educational interpretability for practical applications.

2.5 Efficiency optimization and loss function

Efficiency optimization and loss function design are key components in ensuring the practicality and performance integrity of the model. Efficiency optimization focuses on the computational constraints of edge devices in educational scenarios. Through a full-process strategy of feature dimensionality reduction, lightweight decoding, and redundancy pruning, the model complexity is reduced while controlling precision loss. The loss function targets the multi-task objectives of "change map generation - trajectory embedding - pattern clustering," adopting a hybrid loss architecture to achieve collaborative optimization of multi-dimensional performance, providing precise guidance for model training.

2.5.1 Efficiency optimization strategy

The efficiency optimization strategy is based on the design principle of "full-process lightweight," optimizing three key components: encoder feature output, decoder upsampling, and feature channels. The strategy aims to achieve a collaborative improvement of model parameters and inference speed while ensuring that the core performance of trajectory analysis and pattern discovery is not significantly impacted. The design logic of this strategy originates from the practical constraints of educational edge devices—such as smart tablets and classroom terminals, whose computational power is typically only 1/10 to 1/5 of that of professional training GPUs. Therefore, targeted optimization is required to break through deployment bottlenecks.

The feature dimensionality reduction and redundancy pruning strategy focuses on compressing the dimensionality of encoder output features, reducing the computational burden of subsequent processing while retaining core trajectory information. At the output of each stage in the STv2 encoder, a 1×1 convolution layer is applied to linearly transform the feature channels, reducing the number of channels to half of the original size. This operation uses information aggregation along the channel dimension for dimensionality reduction, avoiding the loss of trajectory spatial information caused by traditional pooling operations. Redundant feature pruning dynamically selects features based on the statistical properties of feature response values: the L2 norm of each channel feature is calculated as the response strength indicator, and after sorting the response values in descending order, the top 80% of channels are retained, while channels with weak responses are pruned.

The lightweight decoder design addresses the high parameter count issue of change map generation by adopting a two-stage upsampling strategy of "bilinear interpolation + 1×1 convolution," replacing traditional transposed convolution. Although transposed convolution can achieve high-precision upsampling, it is prone to checkerboard artifacts, which blur the boundaries of the change map, and its parameter count is more than three times that of bilinear interpolation at the same resolution. The two-stage upsampling process in this study is as follows: In the first stage, the multi-scale fused feature F_fusion undergoes 2x bilinear interpolation, combined with 1×1 convolution to adjust the channel dimension and suppress noise; in the second stage, 8x bilinear interpolation is applied to restore the feature map to the original resolution, followed by 3×3 convolution to refine the boundaries of the change regions.

2.5.2 Loss function design

The design goal of the loss function is to simultaneously optimize the accuracy of change map generation, the quality of trajectory embedding, and the clustering effect of behavior patterns. A single loss function cannot meet the needs of multi-task objectives, so a hybrid loss architecture of "cross-entropy loss - Dice loss - contrastive loss" is constructed, with weight distribution to achieve collaborative optimization of each task goal. The weights of each loss component are determined through grid search, with the highest comprehensive score on the validation set, evaluated by "change map mIoU + trajectory embedding similarity + pattern recognition F1 score." The final weights are determined as: cross-entropy loss weight 0.4, Dice loss weight 0.3, and contrastive loss weight 0.3.

For the change map generation task, a combination of cross-entropy loss and Dice loss is used to solve the class imbalance and boundary accuracy issues in pixel-level classification. Cross-entropy loss is a classic loss function for pixel classification tasks, optimized by quantifying the logarithmic difference between predicted probabilities and true label values, as shown in formula (15), where CM_i,j is the predicted probability of the change map at pixel (i,j), and CM_gt,i,j is the corresponding true label at that location. Since the behavior change region typically occupies only 15%-30% of the image, leading to significant class imbalance, Dice loss is introduced to optimize the sample distribution bias, as shown in formula (16), where TP is the true positive pixel count, FP is the false positive pixel count, and FN is the false negative pixel count. The combination of both loss functions optimizes both classification probability and overall matching of the change regions, improving pixel-level accuracy in the change map.

$\begin{aligned} \operatorname{Loss}_{C E}= & =\frac{1}{H \times W} \sum_{i=1}^H \sum_{j=1}^W C M_{g t, i, j} \log \left(C M_{i, j}\right)+\left(1-C M_{g t, i, j}\right) \log \left(1-C M_{i, j}\right)\end{aligned}$                  (15)

Loss $_{\text {Dice }}=1-\frac{2 T P}{2 T P+F P+F N}$                    (16)

Contrastive loss is used to optimize the discriminability of trajectory point embedding vectors, providing a high-quality embedding foundation for subsequent behavior pattern clustering. The core logic of this loss is to force the embedding vectors of similar trajectory points to converge and the embedding vectors of dissimilar trajectory points to separate, as shown in formula (17), where e_i and e_j are the embedding vectors of similar trajectory points, e_k is the embedding vector of dissimilar trajectory points, and $\tau=0.1$ is the temperature parameter used to adjust the steepness of the similarity distribution. This loss improves the semantic distinction of embedding vectors through contrastive learning, meaning the cosine similarity of similar trajectory points is reinforced, and the similarity of dissimilar trajectory points is suppressed. The final form of the hybrid loss function is shown in formula (18):

Loss $_{\text {contra}}=-\log \frac{\exp \left(e_i \cdot e_j / \tau\right)}{\sum_{k\neq j} \exp \left(e_i \cdot e_k / \tau\right)}$  (17)

Loss $=0.4 \times$ Loss $_{C E}+0.3 \times$ Loss $_{\text {Dice}}+0.3 \times$ Loss $_{\text {contra}}$                    (18)