Evaluating Data Sources and Datasets in Intelligent Transport Systems Through a Weighted Scoring Model

In this section, we present data sources used to collect datasets in ITS. Then we present datasets collected to solve ITS problems. Using drones as sensors for traffic monitoring, then existing datasets for onboard sensors and driving simulators.

To collect datasets, a range of innovative methods are employed. Drones equipped with cameras and sensors are deployed to capture aerial views and collect data on traffic patterns, road conditions, and infrastructure monitoring. In addition, ground-based sensors installed along roadways provide real-time information on traffic volume, speed, and vehicle classification. Driving simulators allow researchers and developers to generate simulated environments, enabling them to study driver behavior, test algorithms, and evaluate new transportation strategies.

The use of camera-equipped drones to measure every vehicle’s position and movements from an aerial perspective is a novel approach that has the potential to revolutionize the way traffic flow is monitored and managed. By having a continuous, real-time bird’s eye view of traffic, bottlenecks, and congestion can be identified and addressed more quickly and effectively. Additionally, this data can be used to study driver behavior and create alert systems in vehicles and to police systems to make the necessary decisions. Also, to optimize traffic patterns and road safety. Sensors on series-production vehicles are used to measure the vehicle´s environment and collect the data [7]. The data collected by the sensors can be used to improve the safety and efficiency of the vehicle and driver, it can be used to provide safety warnings to the driver in the form of visual, auditory, or haptic feedback. The sensors can also be used to monitor the health of the vehicle and its components. The installation of infrastructure sensors at dedicated masts or streetlights located along road segments can permanently monitor a certain road segment for signs of wear and tear. This is especially useful for detecting changes in road conditions like the flux of traffic and detecting abnormal driver behavior that could potentially lead to accidents. By constantly monitoring the condition of the road, these sensors can help to improve the safety of drivers and passengers alike. A simulator of conduits can be used to collect datasets for a variety of purposes. It can be used to collect data on the performance of a system, or to collect data for research purposes. Additionally, a simulator of conduits can be used to collect data for educational purposes or to collect data for marketing purposes.

A key component of ITS is the availability of diverse datasets that enable the system to tackle transportation challenges effectively. These datasets encompass real-time traffic information, weather conditions, road infrastructure details, vehicle data, and user behavior patterns. They provide crucial insights for addressing congestion, optimizing routes, and predicting traffic flow.

Drones equipped with high-resolution cameras can record traffic from a so-called "bird's-eye view" with high position precision. We present the most popular datasets dedicated to ITS. The Stanford Drone Dataset [8] was the first dataset with the trajectories of several road users that was created from the point of view of a drone. It is publicly available and was published in 2016. It is suitable for the analysis of the behaviors and interactions of pedestrians. It consists of nine hours of data from 8 locations on the Stanford campus. The dataset includes 10,300 pedestrian, bicycle, automobile, skateboard, cart, and bus trajectories. Only around 7% of the targets in the sample that have been tagged are cars, compared to a large ratio of identified bikes and pedestrians. The highD dataset [7], which was published in 2018, is the first extensive naturalistic vehicle trajectory dataset on German highways using drone-captured video data. The observations were conducted at six separate locations and involved 110,000 vehicles traveling 45,000 kilometers in 16.5 hours for the highD dataset. The CITR and DUT, two drone-based datasets, were published in 2019 [9]. The dataset, which lasted for less than 30 minutes, was centered on investigating pedestrian behavior when interacting with cars. The controlled experiment used to create the CITR dataset took place in a parking lot, in contrast to the DUT dataset, which comprises pedestrians' naturalistic, uninstructed trajectories. The INTERACTION dataset [10] is a dataset that was produced utilizing drones and includes the realistic motions of numerous traffic participants. Several highly interactive driving scenarios are included in the collection, which comes from China, Bulgaria, Germany, and the United States. It contains measurements from 11 locations and the recording time is up to 16.5 hours. The dataset offers HD-map data in lanelet2 format for the first time. In 2020, the inD dataset [11], which was captured at four various unsignalized junctions in Germany, was published. Over the course of 10 hours, it contains a total of 13,599 trajectories. The inD dataset divides all users of the road into four categories: cars, trucks or buses, bicyclists, and pedestrians. Another urban dataset named the rounD dataset [12] has been published in 2020; it contains over 13,746 trajectories recorded over six hours at three different locations, unsignalized roundabouts in Germany. The openDD dataset [13] is collected in Germany in 2020. openDD contains 84,774 trajectories in 62 hours and HD map data of seven different unsignalized roundabouts. At the signalized intersection in China, a drone dataset SIND [14] was collected and published in 2022. SIND includes traffic light states and HD maps, which contain 7 hours of recording including 13,248 trajectories and include 7 road user types: cars, trucks, buses, tricycles, bikes, motorcycles, and pedestrians. The trajectory dataset called as CitySim dataset [15] was published in 2023 and was taken from drone videos. CitySim has vehicle interaction trajectories extracted from 19 hours at 12 different locations. More severe and significant critical safety events are present in CitySim dataset, which offer supportive scenarios for safety-focused research. The Driving Behavior Net (DBNet) [16] is a dataset for driving behavior research. It includes aligned video, point cloud, GPS and driver behavior (speed and wheel). The dataset is collected in 2018.

The most widely used vehicle motion dataset in the behavioral research fields is the Next Generation Simulation (NGSIM) dataset [17]. Cameras positioned on buildings gathered the raw data, which was then automatically processed. NGSIM has been registered in four different locations: Peachtree Street in Atlanta, Georgia; Lankershim Boulevard, located in Los Angeles, California; eastbound I-80 in Emeryville, California; and U.S. Highway 101 in Los Angele. In the Five Roundabouts Dataset [18], which was published in 2019, over 23 000 vehicles at five unsignalized roundabouts in Australia were followed using a total of six Ibeo LIDAR scanners onboard a vehicle parked close to the roundabouts, yielding more than 60 hours of data. The Strategic Highway Research Program 2 (SHRP 2) NDS [19] database includes data from 50 million vehicle miles and 5.4 million trips, SHRP 2 was collected by 3,147 volunteers using radar, raw-video, and video of the driver at 6 different sites in the United States: central Indiana; Erie County, New York; Tampa, Florida; Durham, North Carolina; central Pennsylvania; and Seattle, Washington. The 100-Car Naturalistic Driving Study dataset [20] contains several examples of excessive driver behavior and performance, like extreme weariness, impairment, mistakes of judgment, risk-taking, aggressive driving, and traffic violations. The collection contains data from a very competent instrumentation system, including 5 channels of video, various vehicle statuses, and kinematic sensors. It also contains data from roughly 2 million vehicle miles and almost 43 thousand hours of data. The European Commission is the founder of the UDrive [21], a large naturalistic driving study in Europe. More than 1,200 drivers contributed the information on more than 35 million kilometers driven in UDrive dataset. The information includes raw video, GPS position, onboard CAN-bus records, front-facing radar, and camera images. However, the datasets UDrive and SHRP 2 are not freely available to the public. The driver behavior dataset [22] is gathered across four car excursions that last, on average, 13 minutes each, using a smartphone in 2017. The Honda Research Institute Driving Dataset (HDD) [23] was published in 2018. The dataset comprises of 104 hours of real human driving in the San Francisco Bay Area, the data was collected using a vehicle fitted with various sensors. The purpose of this dataset is to study driver behavior in real-life environments. The UAH-DriveSet [24] is a dataset that was gathered from six different drivers and cars and is used for the analysis and classification of driving behavior. Three unique driving behaviors were included in the data: normal, drowsy, and aggressive.

According to some researchers, it is challenging to directly model using equations the interactions between human drivers. In order to solve this issue, simulations like CARLA [25] developed by researchers at Intel, and AirSim [26], developed by Microsoft, are examples of such simulators that are both open-source. may more easily imitate a human driver's behavior thanks to learning-based methods for characterization of human-driver behavior. The HRI Driver Behavior Dataset (HDBD) [27] contains driver behavior collected using simulator and real scene videos from 32 participants. Each participant recorded 4 sessions, each consisting of 10 intersections that last approximately eight minutes.

Multiple Criteria Decision Making is a subset of operations research dedicated to assessing and comparing various options or alternatives using multiple criteria or factors. It includes a variety of methods, like Weighted Scoring Model (WSM), Analytic Hierarchy Process (AHP), Analytic Network Process (ANP), etc. In this paper, we used the WSM [29] based on its simplicity and flexibility to compare data sources and datasets. In this approach, each criterion is assigned a weight, signifying its relative importance in the decision-making process. Subsequently, each option is evaluated and scored against these criteria. To calculate a weighted score for each option, the WSM multiplies the score of each criterion by its corresponding weight and then sums up these weighted scores. This process enables decision-makers to quantitatively analyze and rank the options, considering both the significance of each criterion and the performance of each option against those criteria. The WSM Method is utilized in this paper to compare the data source and dataset used in Intelligent Transport Systems (ITS). The application of this strategy involves the following steps:

Determine criteria: Firstly, the criteria that constitute the data collection methods are identified. These criteria serve as the basis for evaluating and comparing the data sources and datasets.

Assign weight to the criteria: Each criterion is assigned a weight that reflects its relative importance in comparison to the other criteria. The weights are determined based on the significance of each criterion in achieving the research objectives.

Create a table of criteria and measurement methods: A table is constructed, listing the chosen criteria and the corresponding measurement methods used to assess the data sources and datasets.

Table of weight: Next, a table is created that displays the assigned weights for each criterion. The scores indicate how well each element performs with respect to each criterion.

Calculation of method score: The WSM calculates a weighted score for each criterion in data source and dataset by multiplying the score of each criterion by its assigned weight and then summing them up. This results in an overall score for each element. The element with the highest weighted score is the one that you should choose.

In this paper, the Weighted Scoring Model (WSM) method is applied at the data source level to determine the best method for data collection. The WSM method is further applied to choose the best dataset among the datasets presented in section 2. Figure 7 illustrates the step-by-step process of this approach.

7.png

Figure 7. Our methodology

10.png

Figure 10. Criteria score for dataset

11.png

Figure 11. Multicriteria spider graph for datasets

In this analysis, we compare the datasets based on the five requirements. The comparison results are depicted in a radar chart format, as shown in Figure 11. According to the findings, the SinD dataset achieves the highest score of 3.4, followed closely by the rounD dataset with a score of 3.3. The OpenDD and inD datasets both obtain a score of 3.2 for each. Other dataset scores are shown in Table 8.

The radar chart visually illustrates the performance of each dataset concerning the established requirements, offering a clear and concise overview of their strengths and capabilities. These results enable researchers to make informed decisions when selecting the most suitable dataset for their specific research objectives in the domain of intelligent transport systems.

By considering the scores obtained by each dataset against the requirements, researchers can identify the datasets that align best with their research goals, ensuring the optimal choice for conducting in-depth studies on driver behavior and other related aspects in the field of intelligent transport systems.