A Multi-View Image Registration-Integrated Method for Three-Dimensional Human Motion Reconstruction in Physical Education

To systematically address the critical issues inherent in existing multi-view three-dimensional human reconstruction methods—including poor modular coupling, inadequate view-adaptive capacity, insufficient utilization of temporal motion features, and weak compatibility with pedagogical applications—an integrated reconstruction and quantitative evaluation technical framework tailored for dynamic sports motion scenarios is constructed. A closed-loop, full-process pipeline from multi-view image inputs to pedagogical motion feedback is realized through four progressively structured functional modules. The entire approach takes high-precision image registration as its core entry point. First, cross-view geometric association information is mined through a human semantic-driven collaborative feature matching strategy, from which highly reliable cross-view matching correspondences are output, thereby effectively overcoming registration deviations caused by high-speed sports motions and limb occlusions. On this basis, a temporal motion constraint mechanism is introduced to accomplish feature temporal propagation and dynamic view screening, effectively suppressing pose jitter induced by frame-independent reconstruction while balancing reconstruction accuracy and computational efficiency. The method further incorporates multi-dimensional view quality assessment metrics to achieve adaptive weighted feature fusion, and a more robust three-dimensional human motion sequence is output through an iterative optimization strategy of the parametric human model. Based on the high-precision motion data obtained from reconstruction, the method ultimately accomplishes extraction of sports motion keyframes, quantification of multi-dimensional motion parameters, and visual analysis of deviations from standard motions, thereby realizing deep integration of three-dimensional vision reconstruction technology with physical education instructional scenarios. In this chapter, the design rationale, technical principles, and specific implementation approaches of each core module are elaborated layer by layer, providing a comprehensive presentation of the overall technical framework of the proposed method.

2.1 Human semantic-aware multi-view collaborative feature matching

Conventional cross-view feature matching methods predominantly accomplish feature association based on two-view independent processing, which renders them inadequate for complex scenarios involving dynamic deformations and frequent self-occlusions characteristic of sports motions. Local visual information from a single viewpoint is highly prone to inducing matching ambiguities and erroneous correspondences. To enhance registration robustness under dynamic human body scenarios, a dual-branch feature extraction architecture is constructed, in which human pose semantic information and deep visual features are simultaneously mined. Feature encoding is accomplished through dual backbone networks respectively, with human joint heatmaps being output by a pre-trained visual Transformer network to characterize the spatial probability distributions of human key body parts, while scale-robust visual features are extracted using a high-resolution network. Spatial max-pooling is applied to the full-channel joint heatmaps, from which semantic attention maps focusing on human body regions are generated. Semantic-adaptive weighted enhancement of visual features is realized through normalization operations, with the specific computation expressed as:

$\widetilde{F}_i=F_i \odot \operatorname{softmax}\left(A_i / \tau_a\right)$              (1)

where, $F_i$ denotes the original visual features, $A_i$ represents the semantic attention map, $\tau_a$ is set to 0.5 to regulate the focusing intensity of the attention distribution, and $\odot$ denotes element-wise feature weighting. This encoding mechanism automatically enhances feature weights in joint-critical regions while suppressing invalid interferences from complex backgrounds and clothing textures, thereby providing stable semantic prior support for accurate cross-view matching. Figure 1 illustrates the flowchart of the human semantic-aware multi-view collaborative feature matching process.

Figure 1. Flowchart of human semantic-aware multi-view collaborative feature matching

To overcome the informational limitations inherent in two-view matching, multiple camera viewpoints are organized into a complete graph topological structure, with feature maps from each viewpoint serving as network nodes, and a cross-view global feature association mechanism is established. Based on the principle of differentiable feature matching, original features are mapped into a unified metric space through learnable projection matrices, and an association score matrix between pixels across different viewpoints is computed as:

$S_{i j}=\operatorname{softmax}_{\text {row }}\left(\frac{\left(W_Q \widetilde{F}_i\right)\left(W_K \widetilde{F}_j\right)^{\top}}{\sqrt{d}}\right)$              (2)

where, $W_Q$ and $W_K$ denote the query and key projection matrices, respectively, and d represents the unified feature dimension. The score matrix quantifies the matching probabilities between pixels across different viewpoints. On this basis, a dynamic graph attention network is introduced to accomplish multi-level global information interaction, through which complementary feature information from all adjacent viewpoints is aggregated via a multi-head attention mechanism. The iterative feature update process is expressed as:

$\widetilde{F}_i^{(l+1)}=\widetilde{F}_i^{(l)}+F F N\left(A G G_{k=1}^{K_{\text {head }}} \sum_{j \in N(i)} a_{i j}^{k,(l)} W_V^{k,(l)} \widetilde{F}_j^{(l)}\right)$             (3)

The attention coefficient $\alpha_{i j}^{k,(l)}$ is adaptively solved through a dynamic activation function and feature concatenation operations, by which information fusion weights are dynamically adjusted according to the feature association quality across different viewpoints. After four layers of iterative graph network inference, single-view features are sufficiently fused with global multi-view geometric and semantic information, effectively mitigating matching uncertainties induced by dynamic occlusions and viewpoint distortions.

To balance matching efficiency with sub-pixel localization accuracy, a coarse-to-fine hierarchical matching strategy is adopted for feature point pair screening and refinement. In the coarse matching stage, spatial local maxima are extracted based on the global association score matrix, and an initial matching point set is filtered with a fixed confidence threshold, through which low-relevance invalid matching combinations are rapidly eliminated, substantially reducing the computational overhead of subsequent fine optimization. In the fine refinement stage, for the feature point pairs obtained from coarse matching, a second-order Taylor expansion model is constructed within local neighborhood windows, and the pixel offset correction is derived by solving the second-order partial derivative matrix of the feature distribution:

$\delta p=-\left(\frac{\partial^2 S_{i j}}{\partial p^2}\right)^{-1} \frac{\partial S_{i j}}{\partial p}$             (4)

Matching coordinates are iteratively updated until parameter convergence, thereby achieving sub-pixel accuracy optimization of matching positions. This hierarchical optimization scheme avoids redundant computations associated with global fine matching while effectively correcting pixel offset errors induced by high-speed sports motions, significantly enhancing localization accuracy in cross-view feature matching.

To ensure that the matching results comply with the geometric constraint principles of the multi-camera system, a dual-verification mechanism incorporating epipolar geometry and homography is introduced for the final screening of the matching point set. Based on the camera calibration parameters, the fundamental matrices between different viewpoints are solved, and a bidirectional symmetric epipolar distance constraint criterion is constructed as:

$d_{e p i p}\left(p_i, p_j\right)=\frac{\left|p_j^{\top} F_{i j} p_i\right|}{\sqrt{\left(F_{i j} p_i\right)_1^2+\left(F_{i j} p_i\right)_2^2}}+\frac{\left|p_i^{\top} F_{j i} p_j\right|}{\sqrt{\left(F_{j i} p_j\right)_1^2+\left(F_{j i} p_j\right)_2^2}}<\tau_{e p i p}$             (5)

where, $F_{i j}$ denotes the fundamental matrix between viewpoints, and $\tau_{\text {epip}}$ is set to 2.5 pixels to constrain the epipolar projection error of matching points. A homography matrix reprojection verification rule is simultaneously introduced to further eliminate outlier point pairs with excessively large geometric mapping deviations. Following multi-level screening and optimization, the module ultimately outputs highly reliable cross-view matching point sets, corresponding matching confidences, and relative pose parameters between camera viewpoints, thereby providing an accurate geometric matching foundation for subsequent temporal constraint optimization, multi-view feature fusion, and human model parameter solving.

2.2 Motion-aware temporal consistency constraints and dynamic registration

To fully exploit the inherent temporal continuity characteristics of sports motions and to address the issues of matching redundancy and dynamic ambiguities inherent in frame-independent registration, an inter-frame dense feature propagation mechanism is constructed, through which adaptive registration initialization driven by temporal priors is realized. Figure 2 illustrates the principles of the temporal consistency constraints and dynamic view-adaptive registration mechanism. Based on the motion correlation properties of consecutive video frames, pixel-level temporal motion offset patterns are modeled. The feature matching search range for subsequent frames is constrained using forward optical flow fields, through which the high computational cost of global exhaustive matching is effectively avoided. The temporal adaptive search region update rule is formulated as:

$R_{t+1}(p)=R_t\left(p+\Phi_{t \rightarrow t+1}(p)\right)+N\left(0, \sigma^2 I\right)$                 (6)

where, $\Phi_{t \rightarrow t+1}$ denotes the dense optical flow field between adjacent frames, which is used to precisely characterize pixel-level temporal motion trajectories, and N represents a two-dimensional Gaussian perturbation term with a standard deviation of 3 pixels, which is introduced to cover motion prediction deviations and ensure completeness of the search region. For scenarios where optical flow estimation fails—such as motion boundaries and limb occlusions—the algorithm automatically reverts to a global search strategy, thereby guaranteeing matching effectiveness under extreme dynamic conditions. A forward-backward optical flow consistency verification mechanism is simultaneously introduced, with an error threshold of 1.5 pixels being set to filter highly reliable optical flow features, through which temporally anomalous matching results are further eliminated. This mechanism reduces matching computational complexity from quadratic to linear order, substantially improving dynamic registration efficiency, while cross-view matching ambiguities in high-speed sports motions are effectively mitigated through the temporal motion constraints.

To fundamentally resolve the issues of pose discontinuities and sequence jitter caused by frame-independent reconstruction, a second-order temporal smoothing regularization constraint is introduced, through which a trajectory optimization mechanism conforming to human motion patterns is constructed. Based on the three-dimensional joint coordinates of consecutive temporal frames, a motion acceleration field is constructed to quantify the instantaneous kinematic variation characteristics of human limbs, by which the dynamic evolutionary patterns of sports motions are precisely characterized. The acceleration field is computed as:

${a}_k^{(t)}=X_k^{(t-1)}-2 X_k^{(t)}+X_k^{(t+1)}$               (7)

Based on the acceleration field, a weighted least-squares smoothing loss function is constructed to achieve temporal constraint optimization over the entire motion sequence, with the specific expression given as:

$L_{\text {smooth }}=\sum_{t=2}^{T-1} \sum_{k=1}^K \gamma_k \cdot\left\|X_k^{(t-1)}-2 X_k^{(t)}+X_k^{(t+1)}\right\|_2^2$             (8)

where, X^(t)_k denotes the three-dimensional coordinates of the k-th human joint at frame t, and $\gamma_k$ represents the joint-adaptive weight coefficient. In accordance with the physiological characteristics of human motion, higher weights are assigned to distal joints—such as wrists and ankles—which possess greater degrees of motion freedom, while lower weights are assigned to core joints—such as the torso and pelvis—which exhibit greater stability, thereby accommodating the motion differences across different body parts. The solution matrix of this regularization term exhibits a pentadiagonal banded structure, which enables efficient iterative solving through Cholesky decomposition, with the overall computational complexity being linear with respect to the total number of video frames, thereby maintaining high algorithmic efficiency while achieving high-precision temporal smoothing.

Figure 2. Temporal consistency constraints and dynamic view-adaptive registration mechanism

To accommodate the dynamic characteristics of rapid movement and instantaneous posture changes in sports motions, while balancing three-dimensional reconstruction accuracy and inference efficiency, real-time prediction of human motion states is realized through an extended Kalman filter, and dynamic view-adaptive screening is accomplished through the establishment of multi-dimensional quantitative criteria. A nine-dimensional motion state vector is constructed by integrating the spatial position of the human center of mass, motion velocity, and acceleration, and accurate prediction of human motion trends is achieved through temporal state transition equations. The core prediction formula is given as:

$\hat{s}^{(t \mid t-1)}=A \cdot s^{(t-1)}+w$              (9)

where, A denotes the temporal state transition matrix, with the temporal update relationship being constructed according to the camera sampling frequency of 30 fps, and w represents the Gaussian process noise, which is used to fit the uncertainty associated with non-uniform human motion. The predicted three-dimensional human center of mass is projected onto each view's image plane, and a view quality evaluation function is constructed by integrating spatial position deviation, human body visibility, and imaging scale:

$q_i^{(t)}=\exp \left(-\frac{\left\|\hat{x}_i^{(t)}-c_i\right\|_2^2}{2 \sigma_c^2}\right) \cdot I\left(v i s_i^{(t)}>\tau_{v i s}\right) \cdot\left(1+\frac{\operatorname{scale}_i^{(t)}}{\max _i \operatorname{scale}_i^{(t)}}\right)$           (10)

where, $c_i$ denotes the image center coordinate of the view, $\sigma_c$ regulates the spatial position penalty weight, $v i s_i^{(t)}$ characterizes the effective visible proportion of the human body, and $\operatorname{scale}{ }_i^{(t)}$ characterizes the projected imaging area of the human body. Through this evaluation function, the real-time informational contribution capacity of each viewpoint is quantified, high-quality active views are selected for subsequent reconstruction, and low-quality views with severe occlusion and insufficient effective information are discarded, thereby enabling adaptive allocation of view resources under dynamic scenarios and achieving balanced optimization between reconstruction accuracy and computational efficiency.

2.3 Confidence-guided adaptive multi-view feature fusion and skinned multi-person linear parameter optimization

To address the issue of imbalanced information quality across different viewpoints in multi-view imaging, a multi-dimensional quantification system is established for precise assessment of view-specific imaging quality, thereby providing adaptive weighting foundations for subsequent feature fusion and model optimization. Figure 3 illustrates the workflow of confidence-guided three-dimensional feature fusion and skinned multi-person linear model optimization. Three complementary evaluation dimensions—pose detection confidence, limb occlusion degree, and spatial observation angle—are integrated, and a view comprehensive confidence computation model is constructed to achieve dynamic adaptive solving of each view's contribution at every frame. The overall computation is expressed as:

$c_i^{(t)}=\alpha \cdot c_i^{\text {pose }(t)}+\beta \cdot c_i^{\text {occ }(t)}+\gamma \cdot c_i^{\text {angle }(t)}$             (11)

where, the weight parameters are determined through grid search on the validation set, with $\alpha, \beta$, and $\gamma$ set to 0.35, 0.35, and 0.30, respectively. The pose confidence is computed through weighted aggregation of multi-joint detection scores, with weights assigned according to limb motion amplitudes, thereby enhancing the evaluation proportion of dynamically moving joints. The occlusion confidence is derived from the proportion of effectively visible human body regions obtained from semantic segmentation results, with the output weights of heavily occluded views being suppressed through an exponential decay function. The viewing angle confidence is adaptively assigned based on the spatial angle between the human body orientation and the camera viewing direction, ensuring that perpendicular viewing angles are granted higher weight advantages. Following the fusion of multi-dimensional scores, global view confidences are subjected to normalization, while invalid information inputs from inactive views are simultaneously masked, thereby achieving dynamic and precise allocation of view weights.

Figure 3. Confidence-guided three-dimensional feature fusion and skinned multi-person linear model optimization workflow

Based on the quantified view confidences, a differentiable projection fusion mechanism is constructed, through which two-dimensional multi-view features are uniformly mapped into a three-dimensional feature space for high-precision feature aggregation. A fixed reference view coordinate system is selected as the global reference frame, and a plane-sweeping strategy is employed to discretely sample the depth dimension, from which a uniformly scaled three-dimensional feature volume structure is constructed. For any voxel coordinates in the three-dimensional space, the mapping from spatial coordinates to image plane coordinates is accomplished through camera projection matrices, and cross-view feature sampling is achieved through an eight-neighborhood trilinear interpolation algorithm. The single-view three-dimensional feature mapping process is expressed as:

$V_i(x)=\sum_{q \in N(u)} w_q \cdot F_i^{(t)}(q)$              (12)

where, $w_q$ denotes the neighborhood pixel interpolation weights, which are used to ensure continuity and sub-pixel accuracy in feature sampling. On this basis, view confidences are introduced to perform weighted fusion, through which noise feature interference from low-quality views is attenuated. The final expression for the fused three-dimensional feature volume is given as:

$V_{\text {fused }}^{(t)}(x)=\frac{\sum_{i=1}^N c_i^{(t)} \cdot V_i^{(t)}(x)}{\sum_{i=1}^N c_i^{(t)}+\epsilon}$               (13)

where, $\epsilon$ denotes a small constant introduced to prevent numerical division-by-zero anomalies. This fusion scheme dynamically adapts to dynamic deformations in sports motions and fluctuations in view quality, significantly enhancing the stability and interference immunity of the three-dimensional feature representation.

Fine-grained solving of three-dimensional poses and meshes is accomplished through a parametric human model, and a confidence-guided staged parameter optimization strategy is constructed to achieve model fitting under multiple coupled constraints. A differentiable skinned multi-person linear human model is adopted to characterize three-dimensional human morphology and pose, with human mesh generation being jointly driven by pose parameters and shape parameters. The model skinning transformation is formulated as:

$M(\theta, \beta)=\sum_{b=1}^\beta w_b \cdot G_b(\theta, \beta) \cdot T(\beta)$              (14)

where, $T(\beta)$ denotes the shape-adaptive template mesh, $G_b$ characterizes the bone spatial transformation relationships, and $w_b$ represents the fixed skinning weights. To accommodate dynamic variations in view-specific quality, a view confidence-weighted reprojection loss function is constructed, and a Geman–McClure robust kernel is incorporated to suppress interference from outlier detection points. The core reprojection loss is expressed as:

$\begin{gathered}L_{\text {reproj }}^{(t)}=\sum_{i \in V_{\text {active }}^{(t)}} c_i^{(t)}  \cdot \sum_{k=1}^J \rho\left(\left\|\Pi_i\left(J_k\left(M\left(\theta^{(t)}, \beta\right)\right)\right)-\hat{J}_{i, k}^{(t)}\right\|_2\right)\end{gathered}$             (15)

A complete optimization objective function is constructed by integrating temporal smoothness constraints, human shape priors, and pose priors, with the model fitting process being collaboratively regulated through multiple constraints. A staged iterative optimization strategy is adopted, in which pose parameters and joint parameters are optimized in separate stages while global shape parameters are fixed to maintain body shape consistency. High-precision parameter solving is achieved through the Limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) optimizer, effectively balancing model fitting accuracy, temporal stability, and human physiological plausibility.

To address the reconstruction distortion problem caused by observation failure of local joints under extreme occlusion scenarios, a human skeletal topology constraint is introduced to achieve adaptive completion of missing regions. When a joint is completely occluded across all active views, two-dimensional observational information becomes entirely unavailable, and reconstruction accuracy cannot be guaranteed through feature fitting alone. To this end, a bone length regularization constraint is incorporated into the global objective function, through which mesh deformation deviations are constrained based on the inherent human physiological topology. The constraint is formulated as:

$L_{\text {bone }}=\sum_{(k, l) \in \varepsilon_{\text {bone }}}\left\|\left\|J_k(M)-J_l(M)\right\|_2-\bar{b}_{k l}\right\|_2^2$               (16)

where, $\varepsilon_{\text {bone}}$ denotes the standard human skeletal topology set, and $\bar{b}_{k l}$ represents the statistically derived standard bone length. This constraint forces the optimization process to maintain reasonable limb bone scales, and the spatial positions of occluded joints are inferred through effective motion constraints from parent and adjacent joints. The reconstruction deficiency and morphological distortion in fully occluded regions are thereby effectively addressed, further enhancing the completeness and physiological plausibility of three-dimensional human reconstruction under complex sports motion scenarios.

2.4 Motion parametric evaluation and visual feedback for physical education instruction

Based on the three-dimensional human motion temporal features, a keyframe adaptive extraction mechanism driven by motion energy spectrum is constructed, through which accurate screening of core postures in sports motions is achieved, effectively avoiding data redundancy and feature redundancy introduced by frame-by-frame analysis. Figure 4 illustrates the interface of the motion quantitative evaluation and visual feedback system tailored for physical education instruction. The instantaneous motion states of all human joints are integrated, and velocity and acceleration features are fused to construct a global motion energy function, which quantifies the motion intensity and postural feature saliency of individual frames. The specific expression is given as:

$E^{(t)}=\sum_{k=1}^K\left\|v_k^{(t)}\right\|_2^2+\eta \cdot \sum_{k=1}^K\left\|a_k^{(t)}\right\|_2^2$              (17)

where, v^(t)_k and a^(t)_k denote the instantaneous velocity and instantaneous acceleration of the k-th joint, respectively, and $\eta$ represents the weight coefficient balancing the two types of motion features, which is set to 0.3 based on prior expert knowledge from physical education instruction. To eliminate anomalous fluctuations in the energy curve caused by motion noise, the energy sequence is smoothed using a Savitzky–Golay filter, thereby ensuring stability in peak detection. Initial keyframe candidates are obtained by identifying local maxima in the smoothed energy sequence, and a non-maximum suppression algorithm is introduced to eliminate redundant neighboring frames with a fixed suppression window length. Ultimately, key posture frames that fully characterize the core phases of sports motions—including initiation, force exertion, and peak execution—are selected, laying the foundation for subsequent precise motion evaluation.

4.png

Figure 4. Motion quantitative evaluation and visual feedback interface for physical education instruction

To achieve standardized and digitized evaluation of sports motions, a multi-dimensional quantitative parameter system adapted to instructional scenarios is constructed based on the reconstructed high-precision three-dimensional joint coordinate sequences, from which a structured motion parameter matrix is generated. Guided by human limb motion principles and physical education teaching evaluation criteria, over ten categories of core motion indicators are precisely computed. Among these, the computation of each joint angle is formulated as: