© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
In modern surveillance systems intended for surveilled areas, Unmanned Aerial Vehicles (UAVs) equipped with computer vision capabilities fulfill an essential role in tracking objects within dynamic and high-risk monitored regions. This paper presents a novel approach SP-TSA to estimate the areas where objects are likely to be present by analyzing their trajectories, which are estimated through UAV-based computer vision. Each trajectory is represented by a series of points in 3D space, with each point acting as the center of a sphere. The spatial uncertainty of the object’s position is captured by the sphere’s radius, providing a comprehensive probabilistic model of potential object locations. To model the area where an object could be present, the intersections of these spheres are analyzed, and the regions where the spheres overlap are used to form a continuous tubular surface along the trajectory. We introduce a Non-Linear Objective Function to optimize the estimation of these areas and minimize uncertainties in object location. This innovative approach ensures computational efficiency and adaptability to complex trajectories, making it suitable for real-time applications. The method offers a precise and robust way to predict the object’s presence within a given space, providing valuable insights for decision-making in dynamic surveillance environments. Simulation results validate the SP-TSA method, demonstrating its superior accuracy in estimating object presence compared to traditional methods, particularly in scenarios involving non-linear and erratic object trajectories.
UAV surveillance, object trajectory estimation, non-linear optimization, spherical projections, tubular surface approximation, area of presence estimation
The rapid advancement and proliferation of Unmanned Aerial Vehicles (UAVs) have dramatically transformed various sectors, including surveillance, navigation, disaster management, and logistics. UAVs offer unparalleled agility, adaptability, and remote operational capabilities, making them essential tools in modern autonomous systems. However, these advancements bring significant challenges, particularly in ensuring real-time security and optimizing object trajectory estimation in high-risk environments. Traditional methods of object detection and tracking often struggle with issues such as occlusion, cluttered backgrounds, and the high speeds involved in aerial operations. Recent studies have proposed various innovative methodologies to address these challenges, leveraging techniques ranging from deep learning to advanced filtering algorithms. One notable approach is the use of pixel labeling and particle filter algorithms to enhance vehicle detection and tracking in UAV imagery. The method proposed by Yusuf et al. [1] segments the image using geo-referencing and image segmentation to retrieve foreground objects, followed by vehicle detection via template matching and tracking through a particle filter. The system outperforms traditional methods in terms of detection and tracking accuracy, demonstrating its potential for applications such as traffic management and security surveillance. This approach aligns with our research by emphasizing the importance of efficient object detection and tracking, which is integral to accurately estimating the presence of objects in a surveillance area, especially in dynamic environments. In the context of UAV trajectory planning, Muzammul et al. [2] introduced the Instructed Reinforcement Q-Learning Algorithm (IR-QLA) to optimize UAV flight paths in unknown environments. By integrating received signal strength (RSS) as a reward metric, the IR-QLA method accelerates the learning process, ensuring faster and more precise trajectory optimization. This is particularly relevant to our approach, where non-linear optimization techniques are employed to estimate object trajectories in a dynamic surveillance environment. The application of reinforcement learning methods could further enhance the adaptability of UAVs, allowing them to respond in real-time to unexpected changes in object positions, a key component of our proposed optimization system. For tracking fast-moving objects in environments such as alpine skiing, a deep learning and correlation filter-based approach was proposed in the study [3]. This method overcomes challenges such as jitter, blur, and occlusion by combining neural network tracking with correlation filters to improve detection and tracking accuracy. Our research similarly addresses the difficulty of tracking objects across different altitudes and distances, where rapid movement and occlusions are common. The multi-sensor fusion approach discussed in the study [3] offers a valuable reference for enhancing the robustness of our trajectory estimation, particularly in scenarios where visual information may be intermittently available due to environmental factors. In swarm-based UAV operations, conflict detection and resolution (CDR) are crucial for maintaining safe and efficient flight paths. The distributed CDR algorithm proposed in the study [4] utilizes a consensus approach and strategy coordination to resolve conflicts in UAV formations. This methodology, which ensures that UAVs in a swarm can communicate and adjust their trajectories in real-time to avoid collisions, shares similarities with our non-linear optimization approach. Both methods rely on real-time decision-making processes, with the key difference being the focus on object area estimation in our research, which can be further improved by adopting strategies for dynamic conflict resolution. Finally, the challenge of tracking multiple objects in drone aerial videos, especially in complex environments, has been addressed by Yuan et al. [5] through the development of the Box-MeMBer and MB-OSNet frameworks. These frameworks enable robust multi-object tracking by leveraging a hierarchical connection structure in the OSNet network to capture rich semantic information. Our research similarly deals with the complexities of tracking multiple objects in a surveillance area, where object interactions and environmental dynamics need to be considered for accurate trajectory estimation. In air-ground collaborative systems, the integration of UAVs and Unmanned Ground Vehicles (UGVs) is crucial for enhancing efficiency in multi-target detection and path planning. The study [6] presents a Mixed Integer Linear Programming (MILP)-based model that optimizes task assignment and path planning in such systems, incorporating kinematic constraints, dynamic collision avoidance, and energy consumption considerations. This approach aligns with our research objectives, especially in optimizing UAV operations for surveillance tasks, as it demonstrates how to efficiently allocate resources and reduce energy consumption while ensuring safe and effective mission execution. By integrating these advancements into our proposed approach, we aim to push the boundaries of UAV-based surveillance systems, focusing on optimizing energy consumption and accuracy in estimating object presence in dynamic environments. The methods described above, particularly those related to object detection, tracking, trajectory optimization, and collaborative systems, provide a solid foundation for further enhancing the capabilities of UAVs in industrial and security surveillance. Another area gaining traction is the detection and tracking of UAVs using radio-frequency (RF) and WiFi-based technologies. While RF-based solutions currently dominate the field, recent reviews have highlighted the untapped potential of WiFi-based systems, calling for further research into these technologies to develop more efficient UAV surveillance methods [7]. UAV technology is also evolving in terms of collaborative multi-UAV operations. The GPR-MADDPG model, for example, combines machine learning techniques to optimize the coverage of moving convoys, significantly enhancing the coordination and effectiveness of multiple UAVs in tracking operations [8]. Moreover, advanced multi-target tracking algorithms, such as SLM-IPDA, offer solutions for navigating cluttered environments and tracking UAVs without prior knowledge of their positions improving the overall accuracy and reliability of UAV tracking systems [9]. The SP-TSA framework is positioned within this advanced operational context using spherical projections and tubular approximations to create a precise nonlinear model for accurate area estimation around tracked objects. The integration of emerging technologies like 6G networks and edge computing further enhances UAV capabilities particularly in terms of knowledge fusion and energy efficiency. Federated Learning (FL) frameworks like DECKS enable decentralized communication among UAVs allowing them to collaboratively train models while reducing energy consumption making them highly suitable for urban autonomous operations [10]. Advances in UAV surveillance have demonstrated the effectiveness of lightweight hardware and optimized algorithms for real-time monitoring and energy efficiency, such as the Raspberry Pi 4B with Intel VPU for industrial surveillance [11]. Similarly, combining YOLOv4 object detection with ArUco Markers has achieved high accuracy in agricultural area measurement [12]. Complementing these technological advances, the application of deep learning-based object detection algorithms to low-altitude UAV datasets has shown promise in improving detection accuracy in aerial surveillance. However, the unique challenges posed by low-altitude datasets, such as small object sizes and high object density, underscore the need for continued research to enhance the performance of detectors like Faster R-CNN, YOLO, and RetinaNet in this context [13]. This paper advances current UAV-based surveillance by introducing a new method of nonlinear optimization in the SP-TSA model, which improves trajectory estimation and spatial object prediction in complex surveilled areas. Through this approach, we provide robust surveillance capabilities tailored for unpredictable and adversarial environments, with implications for real-time UAV surveillance and safety-critical applications.
The remainder of this paper is organized as follows: Section 2 provides a comprehensive review of the related work, exploring existing methodologies in object detection and area estimation within UAV surveillance systems. Section 3 introduces the theoretical framework of Object Area Estimation, detailing the proposed approach for accurately defining the regions where objects are likely to be present, based on their trajectories in the surveillance area. Finally, Section 4 presents the simulation and experimental setups, highlighting the scenarios used to validate the proposed method, emphasizing the effectiveness of our approach in estimating object positions and optimizing UAV performance for real-time surveillance.
Unmanned Aerial Vehicles (UAVs) have garnered significant attention in recent years for their applications in surveillance, object detection, and area estimation. Numerous methods have been proposed to enhance the accuracy and efficiency of UAVs in these tasks, particularly in the surveillance of dynamic environments. Ma et al. [14] present a novel correlation filter algorithm for real-time UAV tracking, termed SOCF, which addresses challenges like background disturbances and lack of attention to the tracked object. The algorithm integrates a spatial disturbance suppression strategy using historical response maps to detect and suppress background interference. However, the scalability and computational efficiency of their method become limited when dealing with multiple objects, a challenge addressed by later works. Building on this, Boulares and Barnawi [15] introduced a deep learning framework for object detection in urban environments, employing a convolutional neural network (CNN) to process video data from UAVs. This allowed real-time classification and localization of objects. Despite its effectiveness in structured environments, the method struggled with occlusions and rapid changes in object trajectories. SP-TSA framework enhances these efforts by incorporating nonlinear optimization to predict object trajectories in cluttered and unstructured environments, addressing the issues of occlusions and trajectory variability.
Further advancing the field, Pierpaoli and Rahmani [16] explored a probabilistic approach to object presence estimation, incorporating Bayesian networks to model uncertainties in object movements. While this method improved robustness in unpredictable environments, it was computationally intensive and unsuitable for real-time applications with limited UAV resources, such as battery life or processing power. Similarly, the study [17] tackled object detection and trajectory prediction using support vector machines (SVMs). Although this approach utilized hand-crafted features and provided moderate accuracy, newer methods like deep learning have significantly improved detection accuracy, especially in unstructured environments. The contribution of the research [18] is a hybrid system combining deep learning with edge computing to reduce UAV energy consumption during object detection tasks. Their system dynamically switches between local and cloud-based processing based on the UAV’s altitude and energy levels, closely related to our approach of optimizing UAV energy during surveillance missions. While Mandal et al. [18] focused on energy efficiency during object detection, while SP-TSA framework complements these efforts by efficiently estimating the spatial presence of objects, ensuring that UAVs operate effectively in energy-constrained conditions. Meanwhile, Azid et al. [19] proposed a swarm-based UAV network for distributed computation, where multiple drones collaborate to cover larger areas. Although this increases coverage, it also adds complexity in terms of communication and synchronization, a challenge we aim to avoid by focusing on single UAV solutions. In the study [20], reinforcement learning was applied to optimize UAV path planning, improving coverage and reducing energy consumption. Trajectory prediction methods like those in the research [21], which utilize recurrent neural networks (RNNs), demonstrate strong sequential data-handling capabilities. However, these approaches are resource-intensive and unsuitable for real-time UAV applications. In contrast, SP-TSA method relies on computationally efficient spherical projections and tubular approximations, making it better suited for resource-constrained UAV systems. Another approach, described in the study [22], introduced a geometric algorithm for estimating the area where an object is likely to be present based on the UAV’s camera view and the objects’ speeds. Although this geometric model provides a useful foundation, it does not account for multiple objects or non-linear trajectories, both of which our method addresses. The SP-TSA model overcomes these limitations by using a Non-Linear Objective Function to estimate the intersection of spherical projections along object trajectories, thus accommodating complex motion patterns and multiple targets. Banerjee and Corbetta [23] took a different approach by fusing data from both UAV cameras and ground sensors, improving object detection accuracy but introducing delays due to the need to synchronize multiple data sources. In the study [24], a particle filter was introduced for tracking objects, employing probabilistic modeling to estimate object positions. This method performed well in noisy environments but struggled with fast-moving objects. Traditional image processing methods like background subtraction and edge detection were explored in studies [25, 26], but these lightweight methods lack the sophistication required for modern surveillance challenges such as complex object movements and trajectory intersections. The SP-TSA framework addresses these gaps by integrating advanced computer vision with a robust optimization process, enabling precise area estimation and trajectory tracking in dynamic environments. Our proposed approach integrates Non-Linear Objective Functions to define areas of object presence, improving upon these previous methods by using a dynamic system that adapts to object trajectories and efficiently estimates positions in real-time. By leveraging the intersection of spheres around each trajectory point, our method addresses the limitations of earlier models and enhances object presence estimation in complex surveillance scenarios.
The SP-TSA (Spherical Projections and Tubular Surface Approximation) methodology is designed to estimate the area of object presence in UAV surveillance systems, leveraging computer vision and advanced mathematical modeling. This section provides a detailed explanation of the key components of the proposed framework.
Figure 1. UAV surveillance of multiple object trajectories in the monitored area
In the “object area estimation” problem, we aim to estimate the potential areas where moving objects are located within a surveilled zone, as detected by a UAV. The UAV captures real-time frames data of the environment, including multiple dynamic objects whose positions and movements are continuously tracked. Figure 1 depicts the scenario where the drone follows its own trajectory TD, while observing distinct objects with their respective trajectories T1, T2, T3. The objective of this work is to estimate the area around each object’s trajectory to define a spatial region where the objects may be present at any given time.
3.1 Trajectory representation and sphere approximation
To address this, we propose modeling each object’s trajectory as a series of points, with each point serving as the center of a sphere. These spheres, centered along the trajectory, collectively form the potential space where the object may be located. By applying a Non-Linear Objective Function (NLOF), we dynamically adjust the size of each sphere based on the object’s motion and proximity to other objects. We also account for the intersection areas between the spheres to refine the estimate, ensuring that overlapping regions are minimized in our overall estimation of object presence. This approach provides a robust and flexible framework for accurately determining object positions, even in complex and dynamic environments.
We define the trajectory of an object in a surveilled area as a parametric curve in three-dimensional space. The trajectory Ti(t) is described by three spatial coordinates as functions of time t in Eq. (1).
$T_i(t)=\left(X_{i g}(t), Y_{i g}(t), Z_{i g}(t)\right)$ (1)
At each point Ti(t) on the trajectory, a sphere is centered with a radius Γ(t), which represents the uncertainty or possible region of the presence of the object in Figure 2. The equation of the sphere is defined by Eq. (2).
$\begin{aligned}\left(X-X_{i g}(t)\right)^2 & +\left(Y-Y_{i g}(t)\right)^2+\left(Z-Z_{i g}(t)\right)^2=\Gamma_i(t)^2\end{aligned}$ (2)
Figure 2. Spherical representation of object uncertainty along its trajectory
The spherical approximation is utilized to model spatial uncertainty surrounding object trajectories, wherein each trajectory point serves as the center of a sphere delineating the probable region of object presence. This representation is particularly advantageous due to its isotropic nature, which facilitates simplified mathematical operations for intersections and projections, while ensuring computational efficiency. Compared to alternative geometrical constructs, such as ellipsoids or bounding boxes that necessitate directional variance estimation and entail higher computational demands, spheres offer uniformity across all dimensions. This property renders them particularly well-suited for this application where prior information regarding object dynamics is sparse, providing a robust and computationally efficient framework for spatial uncertainty modeling.
To account for the uncertainty or probability of the object’s presence within a certain radius, we define a Non-Linear Objective Function $\mathfrak{I}\left(T_i(\mathrm{t}), \Gamma_i(\mathrm{t})\right)$ in Eq. (3) that increases non-linearly with the radius.
$\mathfrak{J}\left(T_i(t), \Gamma_i(t)\right)=\alpha \cdot \mathfrak{J}\left(T_i(t)^n\right.$ (3)
The core of the SP-TSA methodology involves a Non-Linear Objective Function that estimates the intersection area of spheres along object trajectories. The objective function minimizes the overlap of spheres, defining a Tubular Region around each trajectory.
When the object moves along the trajectory, the spheres centered at successive points Ti(t) and Ti+1(t+δt) might overlap. The distance $\varepsilon$ between the centers of two spheres at times t and t+δt is given by Eq. (4).
$\epsilon=\left\|T_i(t)-T_{i+1}(t+\delta t)\right\|$
$\epsilon=\sqrt{\left(X_{(i+1) g}(t+\delta t)-X_{i g}(t)\right)^2+\left(Y_{(i+1) g}(t+\delta t)-Y i g(t)\right)^2+\left(Z_{(i+1) g}(t+\delta t)-Z_{i g}(t)\right)^2}$ (4)
If $\epsilon \leq \Gamma_i(t)+\Gamma_{i+1}(t+\delta t)$, the spheres overlap. The area of intersection between two spheres can be approximated by Eq. (5).
$A_{intersection} \approx \frac{\pi\left(\Gamma_i(t)+\Gamma_{i+1}(t+\delta t)-\epsilon\right)^2}{\epsilon}$ (5)
To compute the region where the object could potentially be located, we need to subtract the intersection areas from the total volume of all spheres.
The total area of presence (i.e., the complement of the intersection) can be expressed as shown in Eq. (6).
$\begin{aligned} & A_{presence}=V_{totalspheres}-\sum_{i=1}^{N-1} A_{intersection}(t, t+\delta t)\end{aligned}$ (6)
where, the volume of a single sphere at any trajectory point is defined by Eq. (7).
$V_{sphere}(t)=\frac{4}{3} \pi \Gamma_i(t)^3$ (7)
We now formulate an optimization problem to minimize the uncertainty regarding the object’s location. The objective is to reduce the sum of uncertainties across all points on the trajectory while ensuring the object stays within the allowed radius at each point in Eq. (8).
$\min _{T_i(t)} \sum_{i=1}^N \Im\left(T_i(t), \Gamma_i(t)=\min _{T_i(t)} \alpha \cdot \Gamma_i(t)^n\right.$ (8)
The parameters $\alpha$ and n in the Non-Linear Objective Function are crucial for shaping how the radius Γi(t) influences the optimization process, ultimately impacting the precision of object presence estimation. The parameter a acts as a scaling factor, controlling the overall impact of changes in the radius on the objective function’s value. A larger $\alpha$ implies that even small variations in Γi(t) will significantly affect the system’s performance, highlighting the need for careful tuning through empirical data and simulations to determine its optimal value. On the other hand, the parameter n dictates the degree of non-linearity in the relationship between Γi(t) and the function. As n increases, the function’s growth concerning the radius becomes more pronounced, which is ideal for scenarios where higher precision is required, penalizing larger radii that indicate greater uncertainty. A balanced choice of $\alpha$ and n, guided by practical guidelines and real-world data analysis, ensures that the function accurately reflects the dynamics of the object’s position while optimizing computational efficiency for different surveillance scenarios.
The subject to the constraint is given by Eq. (9).
$\begin{aligned}\left(X-X_{i g}(t)\right)^2+ & \left(Y-Y_{i g}(t)\right)^2+\left(Z-Z_{i g}(t)\right)^2 \leq \Gamma_i(t)^2\end{aligned}$ (9)
which ensures that the object stays within the sphere at time t. This optimization process helps to define the most likely regions of object presence while taking into account uncertainty and spatial constraints.
For two consecutive points Ti(t)=(Xig(t), Yig(t), Zig(t)) and Ti+1(t+δt)=(X(i+1)g(t+δt), Y(i+1)g(t+δt), Z(i+1)g(t+δt)) on the trajectory, we can define their proximity using the Euclidean distance between the points. The condition for the points being “close” is based on this distance ϵ in Eq. (4). We define a threshold ϵthreshold, such that the points are considered close in Eq. (10) verified.
$\epsilon \leq \epsilon_{threshold}$ (10)
Given two spheres centered at Ti(t) and Ti+1(t+δt) with Γi(t) and Γi+1(t+δt), the condition for sphere overlap is that the distance between their centers $\epsilon$ d is less than or equal to the sum of their radis in Eq. (11).
$\epsilon \leq \Gamma_i(t)+\Gamma_{i+1}(t+\delta t)$ (11)
If the condition for sphere overlap is satisfied, the spheres intersect, creating a shared region that signifies the possible area of object presence. The extent of this intersection is crucial as it directly correlates with the degree of overlap between the spheres. In cases where the overlap is significant, the spheres merge to form a continuous tubular surface along the object’s trajectory. This tubular structure is particularly advantageous, as it enables a more precise determination of the object’s spatial extent, enhancing the accuracy of the area estimation.
3.2 Tubular surface approximation
Once the sphere approximation has been established, we move into a more refined representation: The Tubular Surface Approximation. As the points on each object’s trajectory are often densely packed or close to one another, the series of spheres can be approximated as a continuous tube that envelopes the object’s trajectory. This approximation provides a smoother and more efficient way to calculate the object’s possible location by reducing computational complexity, especially in environments where objects move along predictable paths.
For closely spaced points, the intersection of spheres will generate a cylindrical or tubular surface. To approximate this, we can treat the overlapping spheres as generating a tube along the trajectory. The radius of the tube will be approximately the average of the radii of the two spheres in Eq. (12).
$\Gamma_{\text {tube }} \approx \frac{\Gamma_i(t)+\Gamma_{i+1}(t+\delta t)}{2}$ (12)
Thus, Eq. (13) can approximate the equation of the tube’s surface in a local frame around the trajectory.
$\begin{aligned}\left(X-X_{i g}(t)\right)^2 & +\left(Y-Y_{i g}(t)\right)^2+\left(Z-Z_{i g}(t)\right)^2=\Gamma_{t u b e}^2\end{aligned}$ (13)
where, Ti(t) is the parametric trajectory function.
To express the surface of the tube in parametric form, we use a cylindrical coordinate system around the trajectory. Let $\psi$ be the angle around the axis of the tube (the trajectory), and let $\tau$ be the distance along the trajectory in Eq. (14).
$\begin{aligned} \Gamma_{t u b e}(\tau, \psi)=T(\tau) & +\Gamma_{t u b e}(\cos (\psi) \eta(\tau) +\sin (\psi) v(\tau))\end{aligned}$ (14)
The surface area of the tube can be computed by integrating over the length of the trajectory and around the circumference in Eq. (15).
$A_{t u b e}=\int_0^L \int_0^{2 \pi} \Gamma_{t u b e} d \psi d \tau=2 \pi \Gamma_{\text {tube }} L$ (15)
The tubular surfaces are formed by connecting the intersected spherical regions along the trajectory, approximating the 3D space around the object’s motion path. This tubular model provides a continuous representation of object presence, which is critical for tracking dynamic objects. Tubular surfaces provide a robust and adaptive framework for representing non-linear trajectories, demonstrating distinct advantages over conventional constructs such as convex hulls or bounding boxes. By intuitively modeling regions of interest, they maintain computational efficiency. Moreover, the tubular approximation effectively captures the spatial dynamics of objects exhibiting irregular or curved paths, thereby preserving high accuracy and reliability in trajectory estimation within complex and dynamic environments.
Since the trajectory is represented by discrete points corresponding to the object’s coordinates at each time t, we substitute L with a summation that accounts for these discrete points along the trajectory, as defined by Eq. (16). This approach ensures that the calculation accurately captures the object’s movement by considering its position at each specific moment in time.
This formula sums the distances between all consecutive points along the trajectory, providing the total length. The shorter the distances between the points, the more accurate this piecewise approximation becomes in reflecting the actual trajectory length.
$L=\sum_{i=1}^{n-1} \sqrt{\begin{array}{l}\left(X_{(i+1) g}(t+\delta t)-X_{i g}(t)\right)^2+ \\ \left(Y_{(i+1) g}(t+\delta t)-Y i g(t)\right)^2+ \\ \left(Z_{(i+1) g}(t+\delta t)-Z_{i g}(t)\right)^2\end{array}}$ (16)
We study the performance of the proposed method via computer simulations using different trajectory patterns and object positions. This simulation is designed to replicate real-world UAV surveillance conditions, focusing on dynamic and complex trajectories. In this section, we demonstrate the effectiveness of the proposed method under many trajectories, to assess its capability for area estimation. Then, we investigate the impacts of several important factors on the proposed method’s accuracy, including the target position and the distance between points corresponding to the object position.
In this phase of our study, parameters a and n for the Non-Linear Objective Function were chosen to optimize the radii of spheres representing the object’s position along the trajectory. We set the scaling factor $\alpha$ = 0.5 based onan empirical analysis that balanced computational efficiency with sensitivity, where we aimed to moderate the influence of the sphere’s radius Γi(t) on the optimization function. This choice of a ensured that our model-maintained sensitivity to changes in the radius without causing disproportionate amplification of minor variations. The degree of non-linearity, denoted as n, was set n = 2. This quadratic relationship between Γi(t) and the objective function was selected to emphasize moderate penalization for larger radii, which aligns with the physical interpretation of growing uncertainty in the object’s position as the radius increases. To validate the model, the simulation parameters were carefully chosen to replicate diverse real-world conditions. During the simulation, we computed the total trajectory length using a piecewise approximation, where the cumulative sum of distances between consecutive points was calculated to capture the object’s motion path. We observed that the precision of this approximation improved significantly when the distances between points were minimized, accurately reflecting the trajectory’s real-world characteristics. Next, we applied the tubular surface approximation to regions with closely positioned points, setting the tube’s radius to the average of overlapping spheres, thereby providing a continuous representation of the object’s motion. This tubular model highlighted areas of dense object movement. This approach emphasizes areas of dense object presence and frequent movements, critical for UAV surveillance applications requiring high spatial accuracy.
Following the generated paths, targets move in their respective trajectories, as shown in Figure 3, where each trajectory is distinctly visualized to highlight its specific movement pattern.
We demonstrate in Figure 4 the representation of individual points along these trajectories as separated spheres, which correspond to the object’s exact position at discrete moments in time.
When considering close points on these trajectories, the overlapping regions of spheres become more pronounced, as illustrated in Figure 5, highlighting areas where objects are in proximity, which is crucial for identifying zones of interaction.
As shown in Figure 6, the tube approximation for the trajectory of these close points provides a continuous representation of the object’s motion path. This tubular structure accentuates regions with dense object presence or frequent movement, emphasizing the most significant areas of surveillance. This method ensures accurate area estimation while minimizing computational overhead.
Figure 3. Distinct object trajectories for area estimation analysis
Figure 4. Regions of spheres for separated points on trajectories
Figure 5. Regions of spheres for close points on trajectories
Figure 6. Tube approximation depicting continuous object motion path for close points
As the distance between trajectory points decreases, the tube approximation becomes more defined, illustrating the higher density of object movement in those zones. These observations validate the SP-TSA method’s efficiency in accurately estimating object presence and optimizing UAV surveilled strategies in real-time surveillance.
The results underscore the method’s potential to outperform traditional approaches, in terms of accuracy and computational efficiency, particularly in dynamic and unpredictable environments. The NTLLCM algorithm [15] effectively detects UAV cluster targets in infrared images, focusing on multiscale detection and computational efficiency but lacks trajectory estimation capabilities. The deep learning-based method [22] employs YOLO and SAP for UAV pursuit-evasion, achieving real-time detection and control but relies on bounding boxes, limiting precision in modelling trajectory uncertainty. In contrast, SP-TSA provides more accurate trajectory estimation using spherical projections and tubular surface approximations, outperforming these methods in dynamic and complex UAV surveillance scenarios.
This paper introduced the SP-TSA method for estimating object presence in a surveillance area using UAVs by modeling the object’s trajectory as a parametric curve with spheres representing position uncertainty. The proposed method offers an innovative framework that combines non-linear optimization and geometric modeling to address challenges in UAV-based surveillance. We optimized the detection of these regions through a Non-Linear Objective Function, minimizing uncertainty by analyzing sphere intersections. The integration of a tubular surface model provided a robust solution for continuous trajectory representation, particularly in dynamic and complex conditions, ensuring high localization accuracy. Simulation results confirmed the method’s adaptability, precision, and potential for enhancing UAV-based surveillance in dynamic environments. Key findings demonstrate that the SP-TSA method significantly outperforms traditional techniques in accurately estimating object positions, even under erratic trajectories, while maintaining computational efficiency. This validates its suitability for real-world applications, particularly in scenarios requiring rapid and precise area surveillance. The broader implications of this work extend to improving UAV operational efficiency and enabling integration with advanced computer vision techniques for real-time object tracking. These advancements could revolutionize UAV applications in security, disaster management, and resource monitoring.
Future research directions include extending the SP-TSA framework to handle multi-agent systems and incorporating adaptive mechanisms to dynamically adjust parameters based on environmental conditions. Additionally, investigating the integration of this method with machine learning-based predictive models could further enhance its effectiveness in highly complex and unpredictable scenarios.
Xig(t),Yig(t), and Zig(t) |
Respective coordinates and Spheres Centers at time t |
n |
Non-linearity Degree |
Vtotalspheres |
Volumes of all individual spheres |
Aintersection |
Area of intersection between neighboring spheres |
Apresence |
Area where object present |
Atube |
Tube area |
L |
Total length of the trajectory |
Greek symbols |
|
Γi(t) |
Radius at time t |
ℑ(Ti(t), t) |
Non-Linear Objective Function |
$\alpha$ |
Scaling factor |
ϵ |
Solid volume fraction |
ϵthreshold |
The threshold at which points are considered close |
ψ |
Angle around the tube axis |
$\tau$ |
Distance along object trajectory |
η($\tau$) and ν($\tau$) |
Orthonormal vectors perpendicular to the trajectory at each point $\tau$ |
Γtube |
Tube radius |
[1] Yusuf, M.O., Hanzla, M., Rahman, H., Sadiq, T., Mudawi, N.A., Almujally, N.A., Algarni, A. (2024). Enhancing vehicle detection and tracking in UAV imagery: A pixel labeling and particle filter approach. IEEE Access, 12: 72896-72911. https://doi.org/10.1109/ACCESS.2024.3401253
[2] Muzammul, M., Assam, M., Ghadi, Y.Y., Innab, N., Alajmi, M., Alahmadi, T.J. (2024). IR-QLA: Machine learning-based Q-learning algorithm optimization for UAVs faster trajectory planning by instructed-reinforcement learning. IEEE Access, 12: 91300-91315. https://doi.org/10.1109/ACCESS.2024.3420169
[3] Qi, J.S., Li, D.G., Zhang, C., Wang, Y. (2022). Alpine skiing tracking method based on deep learning and correlation filter. IEEE Access, 10: 39248-39260. https://doi.org/10.1109/ACCESS.2022.3166949
[4] Wan, Y., Tang, J., Lao, S.Y. (2019). Distributed conflict-detection and resolution algorithm for UAV swarms based on consensus algorithm and strategy coordination. IEEE Access, 7: 100552-100566. https://doi.org/10.1109/ACCESS.2019.2928034
[5] Yuan, Y., Wu, Y., Zhao, L., Chen, J., Zhao, Q. (2023). DB-tracker: Multi-object tracking for drone aerial video based on Box-MeMBer and MB-OSNet. Drones, 7(10): 607. https://doi.org/10.3390/drones7100607
[6] Ma, T.X., Lu, P., Deng, F.W., Geng, K.K. (2024). Air–ground collaborative multi-target detection task assignment and path planning optimization. Drones, 8(3): 110. https://doi.org/10.3390/drones8030110
[7] Bisio, I., Garibotto, C., Haleem, H., Lavagetto, F., Sciarrone, A. (2024). RF/WiFi-based UAV surveillance systems: A systematic literature review. Internet of Things, 26: 101201. https://doi.org/10.1016/j.iot.2024.101201
[8] Garg, A., Jha, S.S. (2023). Deep deterministic policy gradient based multi-UAV control for moving convoy tracking. Engineering Applications of Artificial Intelligence, 126: 107099. https://doi.org/10.1016/j.engappai.2023.107099
[9] Memon, S.A., Ullah, I. (2021). Detection and tracking of the trajectories of dynamic UAVs in restricted and cluttered environment. Expert Systems with Applications 183: 115309. https://doi.org/10.1016/j.eswa.2021.115309
[10] Nawaz, M.W., Zhang, W.Q., Flynn, D., Zhang, L., Swash, R., Abbasi, Q.H., Imran, M.A., Popoola, O. (2024). 6G edge-networks and multi-UAV knowledge fusion for urban autonomous vehicles. Physical Communication, 67: 102479. https://doi.org/10.1016/j.phycom.2024.102479
[11] Mezouari, A., Benaly, M., El Karch, H., Dahou, H., Hlou, L., Elgouri, R. (2024). Security surveillance using UAVs and embedded systems in industrial areas. International Journal of Safety and Security Engineering, 14(2): 387-397. https://doi.org/10.18280/ijsse.140207
[12] Masykur, F., Adi, K., Nurhayati, O.D. (2024). Measuring agricultural area using Yolo object detection and ArUco markers. Ingénierie des Systèmes d’Information, 29(1): 95-106. https://doi.org/10.18280/isi.290111
[13] Mittal, P., Singh, R., Sharma, A. (2020). Deep learning-based object detection in low-altitude UAV datasets: A survey. Image and Vision Computing, 104: 104046. https://doi.org/10.1016/j.imavis.2020.104046
[14] Ma, S.G., Zhao, B., Hou, Z.Q., Yu, W.S., Pu, L., Yang, X.B. (2024). SOCF: A correlation filter for real-time UAV tracking based on spatial disturbance suppression and object saliency-aware. Expert Systems with Applications 238: 122131. https://doi.org/10.1016/j.eswa.2023.122131
[15] Boulares, M., Barnawi, A. (2021). A novel UAV path planning algorithm to search for floating objects on the ocean surface based on object’s trajectory prediction by regression. Robotics and Autonomous Systems, 135: 103673. https://doi.org/10.1016/j.robot.2020.103673
[16] Pierpaoli, P., Rahmani, A. (2018). UAV collision avoidance exploitation for noncooperative trajectory modification. Aerospace Science and Technology, 73: 173-183. https://doi.org/10.1016/j.ast.2017.12.008
[17] Xie, J.H., Feng, S.J., Huang, S.C., Wei, D.Z. (2023). UAV cluster detection algorithm based on weighted tri-layer window local contrast. Infrared Physics & Technology, 133: 104795. https://doi.org/10.1016/j.infrared.2023.104795
[18] Mandal, P., Roy, L.P., Das, S.K. (2024). Tracking of invader drone using hybrid unscented Kalman-Continuous Ant Colony Filter (HUK-CACF). ISA Transactions, 152: 38-50. https://doi.org/10.1016/j.isatra.2024.06.018
[19] Azid, S.I., Kumar, K., Cirrincione, M., Fagiolini, A. (2021). Wind gust estimation for precise quasi-hovering control of quadrotor aircraft. Control Engineering Practice, 116: 104930. https://doi.org/10.1016/j.conengprac.2021.104930
[20] Babinec, A., Apeltauer, J. (2016). On accuracy of position estimation from aerial imagery captured by low-flying UAVs. International Journal of Transportation Science and Technology, 5(3): 152-166. https://doi.org/10.1016/j.ijtst.2017.02.002
[21] Kim, M., Ali Memon, S., Shin, M., Son, H. (2021). Dynamic based trajectory estimation and tracking in an uncertain environment. Expert Systems with Applications, 177: 114919. https://doi.org/10.1016/j.eswa.2021.114919
[22] Huang, H.L., Savkin, A.V., Ni, W. (2022). Online UAV trajectory planning for covert video surveillance of mobile targets. IEEE Transactions on Automation Science and Engineering, 19(2): 735-746. https://doi.org/10.1109/TASE.2021.3062810
[23] Banerjee, P., Corbetta, M. (2020). In-time UAV flight-trajectory estimation and tracking using Bayesian filters. In 2020 IEEE Aerospace Conference, Big Sky, MT, USA, pp. 1-9. https://doi.org/10.1109/AERO47225.2020.9172610
[24] Arola, S., Akhloufi, M.A. (2019). UAV pursuit-evasion using deep learning and search area proposal. In Proceedings of the IEEE International Conference on Robotics and Automation. https://api.semanticscholar.org/CorpusID:226225628.
[25] Doostmohammadian, M., Taghieh, A., Zarrabi, H. (2022). Distributed estimation approach for tracking a mobile target via formation of UAVs. IEEE Transactions on Automation Science and Engineering, 19: 3765-3776. https://doi.org/10.1109/TASE.2021.3135834
[26] Hu, S.Y., Ni, W., Wang, X., Jamalipour, A., Ta, D. (2020). Joint optimization of trajectory, propulsion, and thrust powers for covert UAV-on-UAV video tracking and surveillance. IEEE Transactions on Information Forensics and Security, 16: 1959-1972. https://doi.org/10.1109/TIFS.2020.3047758