© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
The measurement of dynamic surface velocities in rivers holds significant importance for hydrological studies, environmental monitoring, and water resource management. With the rapid evolution of image processing technologies, methods based on imagery for velocity measurement have garnered widespread attention due to their nonintrusive nature and operational simplicity. Addressing the invasive nature, limited measurement capabilities, and poor adaptability to complex environments inherent in traditional velocity measurement techniques, a novel technical scheme is proposed. Initially, an enhanced Mean shift algorithm is employed for effective tracking of river surface targets, overcoming stability and accuracy issues faced by conventional algorithms in complex aquatic environments. Subsequently, a new velocity measurement method, integrating optical flow techniques with calibration technology, is introduced to augment accuracy and mitigate environmental interferences. This research not only enhances the reliability of noncontact velocity measurement technologies but also offers new perspectives and tools for river monitoring, contributing significantly to the sustainable management and utilization of water resources.
dynamic river surface velocity, image processing, noncontact measurement, Mean shift algorithm, optical flow technique, calibration technology
As environmental monitoring technologies continue to evolve, the measurement of dynamic surface velocities in rivers has emerged as a critical topic in the fields of hydrology, environmental science, and engineering management [1, 2]. Traditional velocity measurement methods often rely on contact sensors or are based on simplified assumptions for indirect estimation. These methods face numerous limitations in practical applications, such as invasiveness to the environment, measurement limitations, and operational complexity [35]. With the maturity of image processing technologies, noncontact measurement of river surface velocities using video or image data has gradually become a research focus. This approach can provide more continuous, intuitive, and comprehensive velocity information [68].
Noncontact velocity measurement technology not only minimizes interference with river ecosystems but also enhances the safety and convenience of measurements [9]. Especially under extreme weather conditions and adverse geographical environments, the limitations of traditional measurement methods become more pronounced [10]. Moreover, this technology supports the construction of water resource management and flood warning systems through extensive monitoring data, playing a significant role in promoting the rational use of river resources, maintaining ecological balance, and ensuring the sustainable development of human society [11, 12]. However, existing imagebased velocity measurement technologies still exhibit some flaws and shortcomings. For example, traditional image tracking algorithms have limited stability and accuracy under complex conditions such as reflections and ripples on the river surface [1316]. Although optical flow methods can estimate velocity fields, their adaptability and precision in dynamic river environments need further improvement. Additionally, calibration techniques in practical applications face challenges with complex operations and susceptibility to environmental influences. These technical limitations restrict the application scope and effectiveness of noncontact velocity measurement methods [1719].
In response to these shortcomings, two major improvements are proposed in this study. Firstly, a river surface target tracking technique based on an improved Mean shift algorithm is studied. This technique, through algorithm optimization, enhances the identification and tracking capabilities of dynamic water surface targets, significantly improving stability and accuracy in complex environments. Secondly, a new method combining optical flow techniques and calibration technology for measuring dynamic river surface velocities is introduced. This method can estimate velocities more accurately while reducing the impact of external environmental factors. The integration of these two improvements provides a new technical path for noncontact measurement of river velocities, offering promising application prospects and research value. Through the exploration and implementation of these technologies, not only can the precision and efficiency of velocity measurements be enhanced, but more scientific data support can also be provided for river management and environmental protection.
The traditional Mean shift algorithm exhibits noticeable limitations in tracking river surface targets. Typically employing a single statistical feature modeling, such as color histograms, to represent the target model, the algorithm struggles with the complex and variable river surface environments. Factors such as lighting conditions, surface reflections, and wave changes easily affect the stability of color features, rendering the algorithm ineffective in accurately capturing and tracking targets. Moreover, the limited descriptive capability of a single feature often leads to tracking failure when target morphology changes or when the background features resemble those of the target. To address these deficiencies, an improved Mean shift algorithm that integrates grayscale statistical features with Histogram of Oriented Gradients (HOG) features is proposed. This hybrid model retains the nonparametric search advantages of the original Mean shift algorithm while enhancing target morphology and structural information extraction capabilities through the inclusion of HOG features, thereby effectively enriching and bolstering target representation robustness. Grayscale features exhibit strong robustness to lighting variations, and HOG features are sensitive to target edges and shapes. The combination of these significantly improves the algorithm's tracking performance in complex water surface environments. Assuming the gradients of point (a,b) in the horizontal and vertical directions are represented by H_{a}(a,b) and H_{b}(a,b), respectively, the process of calculating the HOG features is as follows:
$\begin{align} & {{H}_{a}}\left( a,b \right)=U\left( a+1,b \right)U\left( a1,b \right) & {{H}_{b}}\left( a,b \right)=U\left( a,b+1 \right)U\left( a,b1 \right) \end{align}$ (1)
Based on the aforementioned calculations, the gradient magnitude and direction at point (a,b) can be further determined using the following formula:
$\begin{align} & H\left( a,b \right)=\sqrt{{{H}_{a}}{{\left( a,b \right)}^{2}}+{{H}_{b}}{{\left( a,b \right)}^{2}}} & \beta \left( a,b \right)=ARCTAN\left( \frac{{{H}_{a}}\left( a,b \right)}{{{H}_{b}}\left( a,b \right)} \right) \end{align}$ (2)
In the context of this study on dynamic river surface velocity measurement, it is crucial to construct an accurate target tracking model to ensure measurement accuracy and reliability under various environmental conditions. Initially, for a given river surface image area, preprocessing steps are employed to optimize the image for better feature extraction. Preprocessing includes noise reduction and contrast enhancement, among other steps, to mitigate the impacts of environmental noise and lighting variations. Subsequently, two parallel feature extraction operations are performed on the preprocessed image area: calculation of the grayscale histogram and extraction of HOG features. The grayscale histogram is constructed by tallying the frequency of each gray level present in the image, providing information on the distribution of image brightness. The process of extracting HOG features involves calculating the gradient direction and intensity in local areas of the image. Figure 1 illustrates the construction of the target model integrating grayscale and HOG features.
Figure 1. Schematic for establishing a target model integrating grayscale and HOG features
Figure 2. Basic principle of the Meanshift algorithm
Following the extraction, the grayscale histogram and HOG feature vectors undergo normalization to eliminate disparities between different feature magnitudes, ensuring their comparability during subsequent integration. Subsequently, the two normalized feature vectors are merged into a unified feature space to construct the final target description model. This process is based on the joint probability distribution, specifically estimated through a statistical method that assesses the joint distribution of the two features. Assuming the normalized pixel position centered on the target is represented by c*_{u}, with the target center coordinates denoted by (a_{0},b_{0}), the grayscale value at pixel position c_{uk} within the target area's interval index value in the grayscale histogram is indicated by β(c_{uk}), and the HOG value at pixel position c_{uk} within the target area's interval index value in the HOG histogram is represented by α(c_{uk}). The interval index value vector divided by the grayscale histogram is denoted by i, and the interval index value vector divided by the HOG histogram is denoted by n. The target model expression is provided as follows:
$w=z\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{J\left( c_{u}^{*}{{}^{2}} \right)}\sigma \left[ \beta \left( {{c}_{uk}}i \right) \right]\sigma \left[ \alpha \left( {{c}_{uk}} \right)v \right]n}$ (3)
where,
$\begin{align} & z=\frac{1}{\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{J\left( c_{u}^{*}{{}^{2}} \right)}}} \\ & c_{u}^{*}={{\left( \frac{{{\left( {{a}_{uk}}{{a}_{0}} \right)}^{2}}+{{\left( {{b}_{uk}}{{b}_{0}} \right)}^{2}}}{a_{0}^{2}+b_{0}^{2}} \right)}^{\frac{1}{2}}} \\\end{align}$ (4)
Based on this model, the iterative search process of the Meanshift algorithm is utilized to identify and locate dynamically changing candidate regions within continuous video frames. Figure 2 illustrates the basic principle schematic of the Meanshift algorithm. In each iteration, by calculating the similarity between the current candidate area and the original model, combined with the method of probability distribution, the search window is dynamically adjusted to adapt to the movement and morphological changes of the target on the river surface. Thus, the description template of each candidate area is updated in realtime, accurately capturing the latest state of the target. The center position of the candidate target is denoted by d, and the size of the kernel function window is represented by g. The expression for the description template o of the corresponding candidate target area is given as follows:
$o=z\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{J\left( {{\left\ \frac{d{{c}_{uk}}}{g} \right\}^{2}} \right)}}\sigma \left[ \beta \left( {{c}_{uk}} \right)i \right]\sigma \left[ \beta \left( {{c}_{uk}} \right)n \right]$ (5)
The Meanshift iteration process for the specific river surface target tracking algorithm includes defining the Bhattacharyya coefficient. In the context of target tracking, W represents the feature distribution based on the target model, while O represents the feature distribution of the candidate target area. With the use of fused variables, the definition is as follows:
$\vartheta \left( O,W \right)=\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{\sqrt{{{o}_{uk}}{{w}_{uk}}}}}$ (6)
To simplify the calculation, the Bhattacharyya coefficient is expanded using Taylor series to obtain its approximate expression:
$\vartheta \left( o,w \right)\approx \frac{1}{2}\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{\sqrt{o\left( {{d}_{0}} \right)w}}+\frac{Z}{2}\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{o\left( d \right)}\sqrt{\frac{w}{o\left( {{d}_{0}} \right)}}}}$ (7)
The core objective of the Meanshift iteration equation is to maximize the approximation of the Bhattacharyya coefficient, thereby making the feature distribution of the candidate area as similar as possible to the feature distribution of the target model. In each iteration, by calculating the Bhattacharyya coefficient between the current candidate area and the target model, then moving the center of the candidate area to maximize this coefficient. The direction of movement is determined by the difference between the current position and the probability centroid. This iterative process continues until convergence is reached, that is, when the candidate area no longer undergoes significant changes, or the increment of the Bhattacharyya coefficient is below a certain threshold. The statistical value of the grayscale index u and HOG index k in the candidate template is denoted by w_{uk}, and the statistical value of the grayscale index u and HOG index k in the target template is denoted by O_{uk}. The corresponding iterative equations are provided as follows:
${{d}_{j+1}}={{d}_{j}}+\frac{\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{{{\mu }_{uk}}\left( {{d}_{j}}{{c}_{uk}} \right)}h\left( \left\ \frac{{{d}_{j}}{{c}_{uk}}}{g} \right\ \right)}}{\sum\limits_{u=1}^{l}{\sum\limits_{k=1}^{v}{{{\mu }_{uk}}h\left( \left\ \frac{{{d}_{j}}{{c}_{uk}}}{g} \right\ \right)}}}$ (8)
${{\mu }_{uk}}=\sum\limits_{u=1}^{8}{\sum\limits_{k=1}^{9}{\sqrt{\frac{{{w}_{uk}}}{{{o}_{uk}}\left( d \right)}}\sigma \left[ \beta \left( {{c}_{uk}} \right)i \right]\sigma \left[ \alpha \left( {{c}_{uk}} \right)n \right]}}$ (9)
Hu invariant moments, as shape descriptors, provide crucial characteristic information about the shape of target objects, which remains invariant for target identification and tracking, even under image scaling, rotation, or reflection, thus maintaining consistency. This study opts to integrate Hu invariant moments with template similarity coefficients to establish new iterative convergence criteria. By incorporating template similarity coefficients, a more comprehensive assessment of the similarity between the candidate and model targets can be achieved. This assessment encompasses not only features such as grayscale and texture but also shape information unaffected by changes in viewpoint. Such iterative convergence criteria aid in enhancing the adaptability and robustness of the river surface target tracking algorithm in complex environments. It ensures that, even under complex conditions like surface reflection and wave changes, targets can still be accurately captured and tracked, thereby providing stable and reliable data support for velocity measurement. This study defines the following seven Hu moments:
$\begin{align} & {{g}_{1}}={{\lambda }_{20}}+{{\lambda }_{02}} \\ & {{g}_{2}}={{({{\lambda }_{20}}{{\lambda }_{02}})}^{2}}+4\lambda _{11}^{2} \\ & {{g}_{3}}={{\left( {{\lambda }_{30}}3{{\lambda }_{12}} \right)}^{2}}+{{\left( 3{{\lambda }_{21}}{{\lambda }_{03}} \right)}^{2}} \\ & {{g}_{4}}={{\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)}^{2}}+{{\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)}^{2}} \\ & {{g}_{5}}=\left( {{\lambda }_{30}}{{\lambda }_{12}} \right)\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)\left[ {{\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)}^{2}}3{{\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)}^{2}} \right] \\ & +\left( 3{{\lambda }_{21}}{{\lambda }_{03}} \right)\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)\left[ 3{{\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)}^{2}}{{\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)}^{2}} \right] \\ & {{g}_{6}}=\left( {{\lambda }_{20}}{{\lambda }_{02}} \right)\left[ {{\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)}^{2}}{{\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)}^{2}} \right]+4{{\lambda }_{11}}\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right) \\ & {{g}_{7}}=\left( 3{{\lambda }_{21}}{{\lambda }_{03}} \right)\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)\left[ {{\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)}^{2}}3{{\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)}^{2}} \right] \\ & +\left( 3{{\lambda }_{12}}{{\lambda }_{03}} \right)\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)\left[ 3{{\left( {{\lambda }_{30}}+{{\lambda }_{12}} \right)}^{2}}{{\left( {{\lambda }_{21}}+{{\lambda }_{03}} \right)}^{2}} \right] \\\end{align}$ (10)
Assuming the candidate and target models based on fused grayscale and HOG features are represented by O_{i} and W_{i}, respectively, and the Hu moment vectors of the candidate and target models are denoted by O_{g} and W_{g}, the Euclidean distance between Hu invariant moment vectors is represented by DI(O_{g}(d)W_{g}). Taking into consideration both the Hu invariant moments and the similarity coefficients between O_{i} and W_{i}, the expression for the algorithm's convergence criterion is provided as follows:
$\vartheta =MAX\left( \sqrt{{{o}_{i}}\left( d \right){{W}_{i}}} \right)\hat{\ }MIN\ DI\left( {{P}_{g}}\left( d \right){{w}_{g}} \right)$ (11)
The process steps of the river surface target tracking algorithm based on the improved Meanshift are detailed as follows, as depicted in Figure 3:
(a) At the initiation stage of measuring the dynamic surface velocity of the river, the first frame of the video sequence is acquired. Through the method of differencing, this step effectively separates moving targets from the background, further calculating the target area markers to indicate the initial position and size of the target.
(b) Based on the position of the target, the Region of Interest (ROI) is cropped. Within this area, the Hu invariant moment vector, as well as the grayscale histogram and the orientation gradient histogram vector of the ROI, are calculated. These features, together with the Hu moment vector, constitute the description template of the target.
(c) The next frame image is read, and the candidate target area corresponding to the previously located ROI is cropped. The description template for this candidate area, including the Hu moment vector and its weight matrix, is calculated, and based on this, the next possible position of the target is determined. The weight matrix is calculated based on the feature distribution within the candidate area, reflecting the contribution of different pixels to target localization.
(d) The similarity between the ROI and the candidate area is evaluated by calculating the Bhattacharyya coefficient between them. Additionally, the similarity between their Hu invariant moment vectors is measured using the Euclidean distance. By combining these two similarity indices, the proximity between the candidate area and the true target area can be more accurately judged.
(e) If both the Bhattacharyya coefficient and the similarity of the Hu vectors reach the threshold set by the algorithm, it can be concluded that the optimal position of the target in the current frame has been found. If the predetermined threshold is not reached, the algorithm continues the iterative search until convergence.
Once the optimal position of the target is determined, the area of the target in the current frame is calculated using the region growing method. At this step, it is necessary to decide whether to update the target description template to adapt to possible changes in the target during motion. Meanwhile, the Bhattacharyya coefficient is calculated, and whether to update the model is judged based on the preset threshold. If an update is required, the algorithm returns to the second step, recalculating the ROI and the target description template.
(f) The algorithm determines whether all frames have been processed. If not, the third step is continued with the processing of the next frame image. If all frames have been processed, the algorithm concludes.
Figure 3. River surface target tracking algorithm based on the improved Meanshift
In the study of dynamic river surface velocity measurement techniques, the calculation of optical flow values for river surface targets typically involves the use of optical flow theory to estimate the motion changes of each pixel point over time. By integrating the category, size, and location attributes of water surface targets obtained through the improved Meanshift algorithm, this study employs the FlowNet to further analyze the motion of these targets between consecutive video frames. Specifically, the FlowNet first predicts the displacement vectors between the current and the next frame using deep learning methods, combined with target attributes and spatiotemporal information. These displacement vectors represent the optical flow. Subsequently, the velocity and motion trend of the water flow can be calculated through the vector magnitude and direction of the optical flow field. Figure 4 presents the framework of the method.
Camera calibration is crucial when measuring the dynamic surface velocity of rivers, as this process involves converting twodimensional image information captured by the camera into threedimensional information in the real world. Since the size, position, and optical flow speed of river surface targets captured by the camera are relative to the camera's own parameters rather than absolute physical dimensions or speeds, it is essential to precisely define the camera's intrinsic parameters such as focal length and field of view, as well as extrinsic parameters like the camera's position and orientation. Through camera calibration, the correspondence between pixels and actual physical units can be established, allowing pixel movement to be converted into actual distance movement, and thereby calculating the true velocity of river surface targets. Without calibration, the data captured by the camera would lack the necessary context to quantify image data into actual physical quantities, resulting in inaccurate measurements and analysis of river surface flow velocities.
Specifically, this study assumes a point l_{0} in the physical world with a true velocity represented by n_{0}, which corresponds to a velocity on the camera's projection plane represented by n'_{0}. In a unit time ds, the distances they travel are n_{0}ds and n'_{0}ds, respectively, with the camera's focal length represented by q, and the distance between the camera and the river surface target represented by c. Based on the principle of projection, the relationship between n_{0}ds and n'_{0}ds is as shown:
$\frac{{{n}_{0}}ds}{c}=\frac{n{{'}_{0}}ds}{q}$ (13)
Since the magnitude of the optical flow value representing the relative velocity of the river surface target to the camera should be proportional to the velocity of the river surface target on the projection n'_{0}, with the proportionality value represented by j_{0}, it follows that:
${{n}_{OP}}={{j}_{0}}n_{0}^{'}$ (13)
Combining the above two expressions, the following is obtained:
${{n}_{0}}:{{n}_{OP}}=c:\left( d{{j}_{0}} \right)=c:j$ (14)
In the study of dynamic river surface velocity measurement techniques, prior information serves as an important reference, aiding in resolving the uncertainty between the target size in camera images and its actual physical dimensions. The prior size of river surface targets may be derived from historical measurement data, known geographical features, or information provided by specialized measurement equipment. With the aid of this prior knowledge, researchers can establish a relationship between the image size of the target and its actual size, thereby estimating the distance between the camera and the target. This study opts to utilize the prior size of river surface targets to estimate the distance between the camera and the river surface targets. This method becomes particularly important in situations where environmental conditions limit the use of other positioning technologies or in the absence of sufficient control points for spatial analysis. Specifically, the type of river surface target is first identified, and its prior size information is obtained. Upon acquiring this data, the principle of similar triangles is employed to establish the relationship between the pixel size of the target in the image and its actual size. For instance, if the realworld length of a specific target is known, then by measuring its pixel length in the image and combining this with parameters such as camera focal length and sensor characteristics, the distance from the camera to that target can be estimated. This distance estimation is crucial for subsequent optical flow velocity calculations, as it affects the conversion factor from pixel speed to actual physical speed. Assuming the height of the plane where the river surface target is captured by the camera is represented by g_{SU}, the prior height of the river surface target is represented by g_{PR}, the ratio of the target's height to the image height is represented by e_{YL}, based on the working principle of the lens, the following is obtained:
$c=\frac{{{g}_{SU}}}{TAN\varphi }=\frac{{{g}_{PR}}}{{{e}_{YL}}TAN\varphi }$ (15)
TANϕ can be determined through calibration:
$TAN\varphi =\frac{{{g}_{2}}}{{{c}_{2}}}$ (16)
By combining the above two expressions, the following is obtained:
$c=\frac{{{g}_{OB}}{{c}_{2}}}{{{t}_{OB}}{{g}_{2}}}$ (17)
Assuming the known constants obtained through calibration are represented by c_{2}, g_{2}, n_{1}, n_{OP}_{1}, and c_{1}, the output of the FlowNet is represented by e_{YL}, the actual height of the river surface target is represented by g_{PR}, and the optical flow reference value is represented by n_{OP}. Then, the following can be obtained:
${{n}_{0}}=\frac{{{g}_{PR}}{{c}_{2}}}{{{e}_{YL}}{{g}_{2}}}\frac{{{n}_{1}}}{{{n}_{OP1}}{{c}_{1}}}{{n}_{OP}}$ (18)
Through the above formula, the optical flow of river surface targets can be converted into their physical velocity.
Optical flow methodology inherently estimates visual motion by analyzing the movement of pixel points between sequences of images, naturally providing motion information within the twodimensional plane, specifically in the a and b directions on the image plane. However, this method does not directly offer information about the object's motion along the depth direction (caxis). In the measurement of river surface flow velocity, the threedimensional dynamics of water flow constitute crucial information, particularly the longitudinal flow velocity perpendicular to the observation plane. Therefore, to overcome the limitations of optical flow technology in capturing threedimensional motion information, this study applies the overhead angle optical flow method on the twodimensional plane. By positioning the camera at an overhead angle, the threedimensional motion of the water flow is projected onto a twodimensional plane, allowing the longitudinal flow velocity component, which is otherwise unattainable, to be indirectly measured through its projection on the twodimensional plane. Figure 5 displays a schematic diagram of the conversion of baxis optical flow from an overhead perspective. Assuming the overhead angle of the camera is represented by β, the height of the camera by g, and the focal length of the camera by D. The distance of the river surface target on the baxis from the midpoint in the image is denoted by f. Based on the pinhole imaging principle and angle relationships, the following is obtained:
$XY=\frac{g}{SIN\beta }$ (19)
$XZ'=XY*\frac{f}{d}=\frac{g*f}{s*SIN\beta }$ (20)
$\angle XYZ=ARCTAN\frac{XZ}{XY}=ARCTAN\frac{f}{d}$ (21)
The following can be derived from the Law of Sines:
$\frac{XZ}{SIN(\frac{\tau }{2}ARCTAN\frac{f}{d})}=\frac{XZ}{SIN(\beta +ARCTAN\frac{f}{d})}$ (22)
A geometric transformation model is established to decompose the optical flow vectors on the twodimensional image plane into twodimensional velocity vectors on the river plane. Assuming the vertical axis optical flow on the image is represented by i', and the real optical flow on the baxis is represented by i. The following expression provides the relationship between the baxis optical flow velocity and the vertical axis optical flow velocity obtained from the image:
$i=i'*\frac{XZ'}{CZ}=i'*\frac{SIN\left( \frac{\tau }{2}ARCTAN\left( \frac{f}{d} \right) \right)}{SIN\left( \beta +ARCTAN\left( \frac{f}{d} \right) \right)}$ (23)
Through this transformation, actual water flow velocity components, including velocities in both the horizontal and vertical directions, can be extracted from the twodimensional optical flow.
Figure 4. Framework of the dynamic river surface velocity measurement method based on optical flow and calibration techniques
Figure 5. Schematic diagram of baxis optical flow conversion from an overhead perspective
From Table 1, it is observed that for a 15*20 river surface target subjected to various transformations, the calculated Hu values were obtained. According to the theory of Hu invariant moments, these transformations should not significantly alter the Hu values. The data in the table indicate that for g_{1}, the Hu values remain close under different transformations, demonstrating the consistency of g_{1} through these changes and proving its invariance. For g_{2} to g_{7}, although numerical values changed following translation, rotation, and scaling operations, the extent of change is generally small, which to some extent supports the stability of Hu invariant moments. Notably, the changes in g_{3} and g_{4} are very minimal, indicating these moments are particularly stable against rotation and scaling. It is important to note that while Hu invariant moments possess invariance to these operations, due to limitations in calculation and representation accuracy, especially in digital image processing, theoretical invariance may manifest as approximate invariance in practical applications, which could be one of the reasons for the slight variations in Hu values. The invariance of Hu values to flipping, as well as subsequent rotation and scaling, especially the values of g_{1}, indicates that these moments are stable against such transformations. Integrating Hu invariant moments as image features for river surface target tracking technology studied in this article is reasonable. Through the improved Meanshift algorithm, these Hu invariant moments can be used as template matching features within the tracking algorithm to enhance the identification and tracking capabilities of dynamic water surface targets. In complex environments, even as targets undergo transformations such as rotation, scaling, or flipping, Hu invariant moments provide stable features for the algorithm, thereby aiding in the accurate tracking of targets, which is especially important in dynamic and variable river environments. Furthermore, combining similarity coefficients with templates can further enhance the robustness of the algorithm, ensuring stability and reliability in the tracking process in practical applications.
Comparing the tracking accuracy of various algorithms on both the training and testing sets, as illustrated in Figure 6, for different centroid error thresholds, it can be concluded that the algorithm proposed in this study exhibits slightly higher accuracy compared to deep learning based Simple Online and Realtime Tracking (DeepSORT), and significantly outperforms You Only Look Once (YOLO) and Mask RegionConvolutional Neural Network (RCNN) on both the training and testing sets. This indicates that the improved Meanshift algorithm provides stable tracking performance, especially maintaining high accuracy even with larger centroid errors. The river surface target tracking method based on the improved Meanshift algorithm demonstrates excellent stability on the training set and good generalization capabilities on the testing set. This suggests that the algorithm proposed in this study is better adapted to changes in targets, especially in complex dynamic environments, providing stable and accurate tracking results. Therefore, the algorithm proposed in this study is effective and valuable for river surface target tracking tasks.
Analyzing the processing time of different algorithms for river surface image sequences as shown in Table 2, the following conclusions can be drawn: The algorithm proposed in this study demonstrates a significant advantage in processing time, especially when compared to the computationally intensive DeepSORT and Mask RCNN. Even when compared to YOLO, the proposed algorithm exhibits competitive processing speeds in most cases and is faster in certain sequences. This aspect is critically important for scenarios requiring realtime or rapid processing, such as river surface target tracking, which may necessitate swift responses to ensure navigational safety or conduct environmental monitoring. The river surface target tracking method based on the improved Meanshift algorithm not only excels in tracking accuracy but also shows superiority in processing time. The low latency of processing times makes this method particularly effective for realtime applications that require quick responses. Therefore, the algorithm proposed in this study is not only technically feasible but also valuable in practical applications, especially in environments where resources are limited or where time sensitivity is crucial.
(a) Training set
(b) Testing set
Figure 6. Comparison of tracking accuracy of different algorithms for river surface targets
Table 1. Hu values for a 15*20 river surface target after various transformations
Hu Moment 
Original Image 
Translation (4,5) 
Rotation by 15° 
Reduction by 1.5 Times 
Flipping 
Flipping Followed by a 15° Rotation 
Flipping Followed by a 1.5 Times Reduction 
g_{1} 
0.001247 
0.001156 
0.001213 
0.001147 
0.001234 
0.001147 
0.001241 
g_{2} 
2.41258E08 
1.74515E08 
4.23584E08 
2.35481E08 
2.45175E08 
4.11258E08 
2.31452E08 
g_{3} 
8.52147E13 
5.21258E13 
1.14526E10 
8.41152E13 
8.52147E13 
1.24582E10 
8.52314E13 
g_{4} 
4.52134E14 
6.12472E12 
5.42613E12 
3.85461E14 
4.52134E14 
4.15428E12 
3.85467E14 
g_{5} 
3.45218E27 
7.12458E24 
4.75412E23 
5.21425E27 
5.32134E27 
5.42158E23 
5.12348E27 
g_{6} 
5.3235E18 
7.12579E17 
4.75156E16 
5.12485E18 
5.32154E18 
5.41238E16 
5.12485E18 
g_{7} 
8.52134E27 
8.12458E24 
1.12458E22 
5.15218E27 
8.52134E27 
6.12475E23 
5.23145E27 
Table 2. Processing time of different algorithms for river surface image sequences

YOLO 
Mask RCNN 
DeepSORT 
The Proposed Algorithm 
Sequence 1 
43.25 
78.64 
124.58 
31.24 
Sequence 2 
101.23 
153.24 
368.12 
104.23 
Sequence 3 
187.26 
134.98 
668.23 
158.34 
Sequence 4 
13.26 
22.69 
47.56 
12.74 
Sequence 5 
31.24 
44.78 
64.23 
32.12 
Sequence 6 
67.58 
57.91 
278.14 
68.57 
Sequence 7 
48.23 
43.25 
77.96 
41.28 
Sequence 8 
215.48 
342.9 
723.12 
176.23 
Sequence 9 
11.57 
12.89 
17.53 
9.12 
Table 3. Comparison of results using different dynamic river surface flow velocity measurement methods
Method 
Image Sequence 
Actual Flow Velocity 
Measured Flow Velocity 
Error Ratio 
Buoy tracking method 
Sequence 1 
106 
112.3 
9.78% 
Sequence 2 
101 
108.4 
6.12% 

Sequence 3 
100 
106.2 
7.12% 

Sequence 4 
97 
107.4 
12.34% 

Particle image velocimetry method 
Sequence 1 
104 
123.4 
17.25% 
Sequence 2 
102 
112.4 
17.36% 

Sequence 3 
101 
113.8 
13.26% 

Sequence 4 
97 
103.4 
6.54% 

The proposed algorithm 
Sequence 1 
103 
107.1 
3.32% 
Sequence 2 
101 
102.6 
1.45% 

Sequence 3 
102 
102.8 
1.23% 

Sequence 4 
97 
93.2 
2.36% 
Table 4. Comparison of dynamic river surface flow velocity measurement results across different distances
Measurement Distance (meter) 
Image Sequence 
Actual Flow Velocity 
Measured Flow Velocity 
Error 
45 
Sequence 1 
106 
112.2 
6.34% 
Sequence 2 
101 
104.2 
2.78% 

Sequence 3 
100 
101.5 
1.74% 

Sequence 4 
97 
93.7 
3.26% 

60 
Sequence 1 
104 
107.4 
3.12% 
Sequence 2 
102 
102.5 
1.45% 

Sequence 3 
101 
101.4 
1.12% 

Sequence 4 
97 
95.3 
2.26% 

75 
Sequence 1 
103 
114.7 
6.78% 
Sequence 2 
101 
102.7 
1.65% 

Sequence 3 
102 
101.6 
2.23% 

Sequence 4 
97 
101.4 
4.25% 

81 
Sequence 1 
104 
112.8 
7.56% 
Sequence 2 
101 
104.7 
3.74% 

Sequence 3 
102 
102.3 
3.21% 

Sequence 4 
97 
98.2 
5.57% 
Table 5. Comparison of dynamic river surface flow velocity measurement results for multiple sections at the same distance
River Section 
Image Sequence 
Actual Flow Velocity 
Measured Flow Velocity 
Error 
Section 1 
Sequence 1 
106 
112.4 
8.21% 
Sequence 2 
101 
111.4 
2.23% 

Sequence 3 
100 
102.5 
1.74% 

Sequence 4 
97 
92.8 
4.23% 

Section 2 
Sequence 1 
104 
107.4 
3.24% 
Sequence 2 
102 
102.3 
1.45% 

Sequence 3 
101 
101.5 
1.12% 

Sequence 4 
97 
93.8 
2.26% 

Section 3 
Sequence 1 
103 
114.2 
8.56% 
Sequence 2 
101 
103.6 
2.65% 

Sequence 3 
102 
1.402 
3.12% 

Sequence 4 
97 
92.3 
4.25% 

Section 4 
Sequence 1 
104 
112.4 
11.21% 
Sequence 2 
101 
104.5 
3.23% 

Sequence 3 
102 
103.6 
3.74% 

Sequence 4 
97 
92.5 
5.57% 
Analyzing the results of different river dynamic surface velocity measurement methods as presented in Table 3, the following conclusions can be drawn: The method proposed in this study not only demonstrates high accuracy in individual sequences but also maintains stable measurement precision across all sequences, indicating strong robustness and reliability of the proposed method. Although the other two methods perform well in certain sequences, larger errors in other sequences suggest they may be more susceptible to external environmental factors. The river dynamic surface velocity measurement method proposed in this study, combining optical flow and calibration techniques, outperforms traditional buoy tracking and particle image velocimetry methods in experimental sequences. With a maximum error ratio not exceeding 4%, significantly lower than the other methods, its high accuracy is validated. Particularly when the actual flow velocity values are close to 100, the measured values closely approximate the true values, with an error ratio below 2%, further emphasizing the high precision characteristics of this method.
Table 4 displays the comparison of river dynamic surface velocity measurement results against actual flow velocity values at various distances using the method proposed in this study. Analysis of this data leads to the following conclusions: Overall, a slight increase in error is observed as the measurement distance increases, yet even at a distance of 81 meters, the highest error remains only at 7.56%. At shorter measurement distances (45 meters and 60 meters), the method proposed in this study demonstrates exceptional measurement accuracy. Compared to traditional methods, the proposed method exhibits lower error rates across all measurement distances, particularly at shorter distances, where its stability and accuracy are notably significant. Even as measurement distance increases, the increase in error with the proposed method is not significant, indicating its adaptability and stability. The river dynamic surface velocity measurement method, incorporating optical flow and calibration techniques, maintains high measurement accuracy across different distances. Although measurement error rises with increasing distance, it remains at a relatively low level overall, suggesting strong applicability and reliability of the proposed method in practical applications. Even at longer distances, the proposed method can provide relatively accurate flow velocity measurements, which is of significant importance for actual river monitoring, offering robust data support for water resource management, hydraulic engineering, and environmental protection. Therefore, the measurement method proposed in this study is deemed effective and holds potential for application in river flow velocity measurement.
Table 5 provides dynamic surface velocity measurement results for different sections of the river at the same distance. It is observed that there exists a certain level of error between the actual flow velocity values and the measured velocity values. It is noted that in each river section, the error for image sequence 1 is significantly higher than for the other sequences, particularly in the last measurement of section 1, where the error reached 11.21%. The errors for image sequences 2, 3, and 4 across the river sections are relatively lower, mostly below 5%, indicating a degree of measurement stability. Despite the presence of errors, the majority of the measurement values maintain errors within a relatively low range, especially sequences 2 and 3, demonstrating higher accuracy. The higher error in image sequence 1 might be attributable to specific environmental factors or measurement inaccuracies, necessitating further investigation to improve measurement accuracy. Considering the measurement results from different sections, except for a few cases of high error, the method proposed in this study provides consistent measurement results across different river sections at the same distance. Integrating the data analysis from Table 5, the river dynamic surface velocity measurement method proposed in this study, combining optical flow and calibration techniques, is overall deemed effective. The majority of the measurement results are within an acceptable error range, especially when measuring at the same distance across different sections of the river, the method can provide relatively accurate and consistent velocity values. However, for the few cases where the error is larger, further analysis is recommended to identify the causes of these errors and to improve the measurement method, enhancing its accuracy and reliability for comprehensive application. In future applications, this method will be beneficial for precise determination of river flow velocity in water resource management, hydraulic engineering, and environmental monitoring.
The research presented in this study primarily focuses on enhancing the accuracy and stability of dynamic river surface target tracking and flow velocity measurement. Through the improvement of the Meanshift algorithm, an enhanced capability for the identification and tracking of dynamic water surface targets has been achieved. The refined algorithm is particularly suited to complex environments, improving the stability and accuracy of tracking, which is crucial for the dynamic and variable river surface environments. A new method for measuring flow velocity, combining optical flow and calibration techniques, has been proposed. This method not only increases the accuracy of flow velocity estimates but also reduces the interference of external environmental factors, which is often a challenge in traditional measurement methods.
A series of experiments were conducted to validate the effectiveness of the proposed methods. Comparative experiments on the accuracy of river surface target tracking have confirmed the performance advantages of the improved Meanshift algorithm in tracking river targets. Comparisons of processing times between different algorithms have demonstrated the enhanced efficiency of the improved algorithm while ensuring tracking accuracy. Comparative experiments on the results of different measurement methods have highlighted the accuracy advantages of the river flow velocity measurement method that combines optical flow and calibration techniques. Comparisons of flow velocity in different measurement areas have proven the applicability and stability of the method in various environments. Comparisons of river flow velocity measurement results across different sections at the same distance have ensured the repeatability and consistency of the proposed method across continuous sections.
The target tracking technique based on the improved Meanshift algorithm proposed in this study significantly surpasses traditional methods in the identification and tracking of dynamic river surface targets, especially in adapting to complex environmental changes, demonstrating stronger stability and accuracy. The novel flow velocity measurement method combining optical flow and calibration techniques provides a more accurate and robust way of estimating river flow velocity, effectively resisting interference from external environments. The method proposed in this paper requires further optimization and validation before practical application, especially for cases with larger errors which necessitate indepth analysis and corresponding measures for improvement, to achieve accurate measurements under broader and more complex conditions.
This paper was supported by Scientific and Technological Research Program of Chongqing Municipal Education Commission (Grant No.: KJQN201903806); Scientific and Technological Research Program of Chongqing Municipal Education Commission (Grant No.: KJQN202203808).
[1] Ngoma, D.H., Wang, Y. (2018). Hhaynu micro hydropower scheme: Mbulu–Tanzania comparative river flow velocity and discharge measurement methods. Flow Measurement and Instrumentation, 62: 135142. https://doi.org/10.1016/j.flowmeasinst.2018.05.007
[2] Yeh, M.T., Chung, Y.N., Huang, Y.X., Lai, C.W., Juang, D.J. (2019). Applying adaptive LSPIV with dynamically adjusting detection region approach on the surface velocity measurement of river flow. Computers & Electrical Engineering, 74: 466482. https://doi.org/10.1016/j.compeleceng.2017.12.013
[3] Fujita, I., Kumano, G., Asami, K. (2014). Evaluation of 2D river flow simulation with the aid of imagebased field velocity measurement techniques. In River Flow 2014, pp. 19691977.
[4] Li, Z., Wang, C. (2013). Study on noncontact Yangtze River flow measurement algorithm based on surface velocity. In Informatics and Management Science VI, pp. 355362. https://doi.org/10.1007/9781447148050_42
[5] Ye, X.Q., Liu, J.G. (2013). Measurement and analysis of velocity and flow direction of water in rivercrossing section of Line 2 of Hangzhou metro. Journal of Railway Engineering Society, 30(9): 105110.
[6] Schweitzer, S.A., Cowen, E.A. (2021). Instantaneous riverwide water surface velocity field measurements at centimeter scales using infrared quantitative image velocimetry. Water Resources Research, 57(8): e2020WR029279. https://doi.org/10.1029/2020WR029279
[7] Khalid, M., Pénard, L., Mémin, E. (2019). Optical flow for imagebased river velocity estimation. Flow Measurement and Instrumentation, 65: 110121. https://doi.org/10.1016/j.flowmeasinst.2018.11.009
[8] Patalano, A., García, C.M., Rodríguez, A. (2017). Rectification of image velocity results (RIVeR): A simple and userfriendly toolbox for large scale water surface Particle Image Velocimetry (PIV) and Particle Tracking Velocimetry (PTV). Computers & Geosciences, 109: 323330. https://doi.org/10.1016/j.cageo.2017.07.009
[9] Schumacher, D., Karcher, C. (2014). Flow measurement in metal melts using a noncontact surface velocity sensor based on timeofflight Lorentz force velocimetry. International Journal of Applied Electromagnetics and Mechanics, 44(2): 183191. https://doi.org/10.3233/JAE141758
[10] Wu, J., Ding, S.Q., Yu, K., Li, X.B. (2010). Noncontact measurement of surface flow velocity using photoelectric method. Optics and Precision Engineering, 18(2): 349356.
[11] Wang, T., Hao, S., Ma, X. (2021). Water velocity and level monitoring based on UAV radar. In 2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Chongqing, China, pp. 12321236. https://doi.org/10.1109/IMCEC51613.2021.9482041
[12] Vyas, J.K., Perumal, M., Moramarco, T. (2020). Discharge estimation using Tsallis and Shannon entropy theory in natural channels. Water, 12(6): 1786. https://doi.org/10.3390/w12061786
[13] Iguchi, M., Hanazaki, K., Iguchi, D. (2003). Noncontact sensor for the measurement of meniscus velocity of molten metal flow. ISIJ International, 43(6): 836841. https://doi.org/10.2355/isijinternational.43.836
[14] Kanda, N., Abe, C., Goto, S., Yamada, K., Nakai, K., Saito, Y., Asai, K., Nonomura, T. (2022). Proofofconcept study of sparse processing particle image velocimetry for real time flow observation. Experiments in Fluids, 63(9): 143. https://doi.org/10.1007/s00348022034710
[15] Nazarov, N.A., Terekhov, V.V. (2024). High level GPUaccelerated 2D PIV framework in Python. Computer Physics Communications, 295: 109009. https://doi.org/10.1016/j.cpc.2023.109009
[16] Pagliara, S., Felder, S., Boes, R.M., Hohermuth, B. (2024). Intrusive effects of dualtip conductivity probes on bubble measurements in a wide velocity range. International Journal of Multiphase Flow, 170: 104660. https://doi.org/10.1016/j.ijmultiphaseflow.2023.104660
[17] Niu, P., Wang, D., Wei, P., Yang, Y., Pan, Y., Wang, S., Yu, X. (2020). Liquid flow measurement using phase isolation and an imaging method in horizontal gas–liquid twophase flow. Measurement Science and Technology, 31(9): 095303. https://doi.org/10.1088/13616501/ab83a1
[18] Zhang, X.K., Zhang, H.B., Zhang, B., Zhu, S.Y., Bai, B.F. (2023). Study on the phosphorescence characteristics of diacetyl molecules and the application of molecular tagging velocimetry. Journal of Engineering Thermophysics, 44(9): 24862495.
[19] Zheng, D., Lu, G., Zhang, J., Wang, M. (2016). Experimental method of threedimensional velocity field measurement in circular pipe based on PIV. In 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, China, pp. 15841589. https://doi.org/10.1109/WCICA.2016.7578539