Real-Time Ball Tracking and Prediction for Autonomous Wheeled Soccer Robot

The Sustainable Development Goals (SDGs) continue to attract worldwide attention. The ninth SDG target focuses on industry, innovation, and infrastructure; therefore, this issue needs to be addressed through the development of innovative products, cost-effective performance, and reliable robotic systems [1]. Research in this field has mainly focused on several applications, including autonomous navigation, real-time object tracking, and predictive control. Wheeled soccer robots can serve as challenging test platforms for evaluating tracking algorithms in image processing, which can also be applied to industrial and service robots [2]. Technological sustainability can be achieved through the integration of robotics and computer vision using advanced tracking systems. Surveillance systems, automated warehouse systems, and logistics applications require fundamental functions such as object tracking and trajectory prediction. The main objective of this research is to improve the control capability of wheeled soccer robots for real-time ball tracking and movement prediction, thereby supporting the development of sustainable and high-performance automated technologies.

Autonomous mobile robots are increasingly employed in industrial, service, recreational, and competitive environments. Their effective operation requires the integration of reliable mobile platforms, environmental sensing, autonomous navigation, and coordinated control capabilities [3]. The RoboCup competition, which began in the late 1990s, provides a standardized yet dynamic environment in which robots must perceive their surroundings, track moving objects, coordinate with teammates, and execute complex actions [4, 5]. Unlike conventional industrial automation systems operating under relatively controlled conditions, soccer robots must cope with moving objects, temporary occlusions, illumination variations, and interactions among multiple robots. These characteristics make robotic soccer a demanding and useful platform for evaluating vision-based navigation, object-tracking, and predictive-control algorithms. A fundamental capability required for autonomous soccer robots is the reliable detection and tracking of the ball under real-time conditions [6, 7]. In this research, the ball is considered the primary object of interest, and the robot’s ability to locate, approach, and manipulate the ball directly affects its overall performance. Vision-based ball tracking systems must address several challenges, including accurate detection under varying lighting conditions, robust tracking during ball occlusion, real-time processing for rapid control responses, and trajectory prediction to anticipate future movements [8, 9]. Traditional approaches commonly use color-based segmentation through Hue, Saturation, and Value (HSV) thresholding, which provides efficient performance under different lighting conditions [10, 11]. However, relying solely on color-based detection may generate false positives from similarly colored objects on the field. Therefore, additional shape validation processes, such as the Circular Hough Transform (CHT), are required to verify candidate ball regions [12]. The combination of color segmentation and shape validation has been applied in various soccer robot applications due to its computational efficiency without requiring the high computational resources associated with deep learning methods.

Although the aforementioned detection algorithms can identify the ball in a single image frame, autonomous robots require sequential position information to intercept and control moving objects effectively [13, 14]. The latency introduced between vision processing and control output can reduce performance in dynamic interception tasks, particularly when the ball moves at high speed. Predictive filtering methods, such as the Kalman filter, can address these challenges by estimating the ball’s state, including position and velocity, and predicting its future trajectory. The Kalman filter employs a recursive framework that enables optimal state estimation under uncertainty by combining noisy measurements with motion models to obtain more accurate estimates. For wheeled soccer robots operating in environments with relatively predictable ball movements, the Kalman filter provides high computational efficiency and accurate trajectory prediction performance [15].

The whole process of the proposed method is shown in Figure 2. The proposed system integrates an omnidirectional camera, a wheeled soccer robot, and a control system to achieve autonomous ball interception with a real-time image processing stage. The proposed method is divided into six major stages: image acquisition and preprocessing, color detection and segmentation, shape validation, centroid detection and estimated ball position calculation using a third-order polynomial equation, Kalman filter prediction, and finally action execution. The proposed method runs on hardware and software specifications as shown in Table 2 below.

Figure 2. The overall methodology of the proposed system

Table 2. System hardware and software specifications

3.1 Image acquisition and pre-processing

The system begins with the initialization and activation of an omnidirectional camera with a convex reflector placed on the top of the robot. The camera used is an Omnivision ELP Full HD webcam wide-angle camera with an OmniVision OV2710 CMOS sensor, Full HD 1080p resolution, and a 3.6 mm M12 lens. The camera is directed towards the convex mirror to produce a 360° view of the robot’s surroundings in a single image frame, as shown in Figures 3 and 4.

Figure 3. Omnidirectional camera

Figure 4. 360° panoramic view of omnidirectional camera

There are several advantages that omnidirectional camera can offer such as: (1) full field coverage that enables robot to observe the ball, teammates, opponents, goalposts, and field boundaries at the same time without any movement mechanical parts [24]; reduced latency because there are no moving parts so it will eliminate delay; (3) temporal consistency because all its environmental objects/features are captured in the same time so it will support for accurate state prediction.

Following image capture using an omnidirectional camera, the system applies Adaptive Histogram Equalization (AHE) to enhance image contrast with the parameter values 2.0 for clip limit and 8 × 8 for grid size. This preprocessing technique adaptively redistributes pixel intensity values to achieve a uniform histogram, thereby improving the visibility of objects in both underexposed and overexposed image regions. This transformation maps the original intensity values to new values that approximate a uniform distribution, effectively spreading the most frequent intensity values and compressing less frequent ones. The result is enhanced local contrast, which is particularly beneficial for detecting the ball under varying illumination conditions commonly encountered on soccer fields, such as shadows, reflections, and changing sunlight angles [25].

After the AHE process, the system converts the color space from RGB to HSV. This conversion is important because RGB is more sensitive to lighting variations, while HSV separates color or hue information from intensity or value. Thus, the ball color detection process can run more stable. In the implementation in OpenCV, the HSV calibration range is shown in Table 3.

Table 3. Hue, Saturation, Value (HSV) calibration ranges

3.2 Ball and field color detection with background subtraction

After conversion of color space, the system conducts ball and field color detection using a thresholding method. This dual detection method results several purposes: detecting the ball as main target object and setting the green field to determine the boundary of play areas.

For each pixel in the image, the system examines whether the HSV color values are inside the ranges. Pixels that fulfill the conditions are set as potential ball or field pixels that will results binary masks for each object. If the color detection process fails to detect both ball and field colors so the system activates background subtraction process. This method removes non-relevant objects from the image by subtracting current frame with a background model [26]. Specifically, the frame differencing method computes the absolute difference between the current frame $I_t(x, y)$ and the reference background $B_t(x, y)$, expressed as:

$D_t(x, y)=\left|I_t(x, y)-B_t(x, y)\right|$                       (1)

The difference image obtained from the above formula is then thresholded to generate a binary mask, where pixels with values higher than the threshold are classified as foreground (moving objects), while the remaining pixels are considered background. This method detects moving regions in the scene and removes static regions, making it suitable for real-time object detection in dynamic environments. After the background subtraction phase, the system returns to the color detection phase, forming an iterative loop that continues until the ball is detected. This closed-loop approach ensures robust performance even in crowded environments. The results of the background subtraction phase, including the remaining noise, are shown in Figure 5.

Figure 5. Background subtraction result with its noise

3.3 Noise filtering and shape validation using Circular Hough Transform

After the color detection and background subtraction phase, the system implements noise filtering that can remove small dots, artifacts, and noise caused by camera sensor. The noise filtering phase uses opening and closing as part of morphological operations. Opening performs erosion process followed by a dilation process in order to smooth object contours, while closing performs dilation process followed by an erosion process in order to fill in small holes and gaps. In the system implementation, opening and closing use a 5 × 5 rectangular Kernel. The HSV mask of the orange ball is also cleaned using 3 × 3 erosion and 5 × 5 dilation. Based on the test, the closing operation produces a more intact ball contour than opening. The result of noise filtering is shown below in Figure 6.

Figure 6. Noise filtering result

After noise filtering, the system implements the CHT process, a feature extraction method that designed to detect circular or ellipse pattern in images. The CHT is suitable for ball detection because ball appears as circle or ellipse in the image plane [27].

The CHT results several parameters of the detected circles: coordinates $\left(x_c, y_c\right)$ and radius r. If CHT detects one or more than one circles so the system conducts contour detection process. If CHT cannot detect circular objects so the system returns to Stage 1 (image acquisition), it creates feedback loop until circular objects are detected. CHT runs on several parameters as shown in Table 4 below.

Table 4. Circular Hough Transform (CHT) parameter configuration

3.4 Centroid detection and estimated ball position calculation using third-order polynomial equation

After the shape validation phase is successful, the system conducts contour detection to detect the boundaries of the detected ball precisely. Contour detection algorithms track the external boundaries of interconnected regions inside a binary image. The result of contour detection is a list of contours, where every contour is defined as vector of (x, y) points. Then the system chooses contours based on its properties such as: Area, Perimeter, Circularity, Convexity, and solidity [28]. The contour retrieval component and simple contour approximation method are utilized for contour parameters setting. Then the system finds the largest contour by comparing the area of each detected contour. This method is based on the reasonable assumption that in the robot's field of view, the ball is the largest object that the robot can see. The robot targets the largest detected ball and tracks it. The result of centroid detection of the ball is shown in Figure 7 below.

Figure 7. Centroid detection result

After the largest contour is detected, the system calculates the centroid of the largest contour to determine its center coordinate. This can be calculated by using image moments, where the spatial moments $M_{00}, M_{10}$, and $M_{01}$ are calculated from the region of the largest contour. The centroid coordinates $\left(x_c, y_c\right)$ are then obtained using:

$x_c=\frac{M_{10}}{M_{00}}, y_c=\frac{M_{01}}{M_{00}}$                         (2)

where, $M_{00}$ shows the area of the largest contour, while $M_{10}$ and $M_{01}$ show to the first-order moments along the $x$ and $y$ axes, respectively. This centroid of the largest contour shows the center of detected ball in the field and is used as reference coordinate point for tracking the ball.

After centroid detection of the ball, the system converts the ball position in the image into real-world coordinates (cm). This conversion process is important because the use of convex mirror as reflector produces high nonlinear distortion that causing the Euclidean distance in pixels ($d_p$) to increase in nonlinear way with actual distance ($d_{cm}$). It differs with conventional pinhole cameras; the convex mirror cannot be approximated by linear or inverse model. The resulting third-order polynomial equation to convert Euclidean distance in pixels to actual distance in cm is:

$\begin{gathered}d_{c m}=-0.000108 \cdot d_p^3+0.0429 \cdot d_p^2-4.13 \cdot d_p +135.8\end{gathered}$              (3)

3.5 Ball position prediction using Kalman filter

The Kalman filter is a recursive algorithm that calculates the state of moving system from measurements that contain noise and incomplete data. In soccer robot applications, the Kalman filter can predict future locations of both the ball position and velocity [29]. The Kalman filter works in two main stages: prediction and update. At the prediction step, this filter utilizes the previous state estimation result to predict the current state of the ball position and velocity. At the update step, the result of prediction state is compared with the actual result that obtained from centroid of detected ball. Kalman filter runs on several parameters as shown in Table 5. Comparing the predicted position with the actual detected position continuously, the Kalman filter refines its estimations over time so it results the ball tracking in smoother and more accurate way. This iterative process makes the robot can anticipate the ball trajectory, allows more responsive, and precise movement during play. The result of Kalman filter prediction with blue colored circle and the actual ball position with orange colored circle is shown in Figure 8.

Figure 8. Kalman filter prediction result

3.6 Data transmission and robot action

After calculating the actual and predicted positions of the ball, the system transmits the obtained data to the microcontroller unit (MCU) STM32 F407 VGT6 series. The data transfer between the computer vision processing unit that occurs in the Intel NUC and the MCU for motor control is important for real-time performance, demanding a high-speed serial communication interface configured at 115200 bps to ensure low latency and reliable data transfer between vision and motor control unit.

The MCU processes the received ball position data and triggers commands for the robot’s wheels. The robot moves forward to the predicted location rather than moving to actual ball location. This predictive control movement enables smoother and more efficient motion. In addition, by using the predicted position from the Kalman filter, the system decreases the effect of noise and quick changes in the detected position [30, 31]. This results in better navigation and decreases unimportant oscillatory movement. The control algorithm updates the robot trajectory every time based on new data that have been received, enabling adaptive behavior in the environment.

This study showed a comprehensive approach for ball detection, distance conversion, and trajectory prediction in wheeled soccer robots that used a camera system reflected to convex mirror. The proposed method combines a third-order polynomial model to convert a distance and integrated HSV, CHT and Kalman filter that used to get real-time ball detection and trajectory prediction. According to the experimental results and discussion, the following conclusions can be obtained.

The standard tracking accuracy achieved by conventional object detection methods under heavy convolutional models remains unsatisfactory, yielding only 51% for YOLOv8 and 48% for YOLO-NAS. By employing traditional color segmentation combined with circular feature extraction and utilizing Kalman filter prediction, the proposed HSV-CHT-Kalman method demonstrates a strong balance between high tracking accuracy (82.81%), good computational throughput (79.04 FPS), and physical deployment feasibility, vastly outperforming existing deep learning-based frameworks in terms of overall suitability for real-time robotic tracking systems.

These results validate the effectiveness of the proposed approach for real-time applications, particularly in scenarios where onboard computational resources are limited and highly reliable target localization is critical. The experiment results show that the wheeled soccer robot can track the target appropriately according to field scenarios, including varying angular displacements. Based on the benchmarking and experimental results, it was concluded that the proposed low-overhead system can be used as an alternative high-performance vision tracking method without requiring expensive GPU acceleration. In future research, we will investigate this optimized vision framework to control multiple collaborative autonomous robots to support advanced product innovation as stated in the ninth target of SDGs.