Explore the vast range of titles available.

The integration of thermal imaging data with multimodal large language models (MLLMs) offers promising advancements for enhancing the safety and functionality of autonomous driving systems (ADS) and intelligent transportation systems (ITS). This study investigates the potential of MLLMs, specifically GPT-4 Vision Preview and Gemini 1.0 Pro Vision, for interpreting thermal images for applications in ADS and ITS. Two primary research questions are addressed: the capacity of these models to detect and enumerate objects within thermal images, and to determine whether pairs of image sources represent the same scene. Furthermore, we propose a framework for object detection and classification by integrating infrared (IR) and RGB images of the same scene without requiring localization data. This framework is particularly valuable for enhancing the detection and classification accuracy in environments where both IR and RGB cameras are essential. By employing zero-shot in-context learning for object detection and the chain-of-thought technique for scene discernment, this study demonstrates that MLLMs can recognize objects such as vehicles and individuals with promising results, even in the challenging domain of thermal imaging. The results indicate a high true positive rate for larger objects and moderate success in scene discernment, with a recall of 0.91 and a precision of 0.79 for similar scenes. The integration of IR and RGB images further enhances detection capabilities, achieving an average precision of 0.93 and an average recall of 0.56. This approach leverages the complementary strengths of each modality to compensate for individual limitations. This study highlights the potential of combining advanced AI methodologies with thermal imaging to enhance the accuracy and reliability of ADS, while identifying areas for improvement in model performance.

The deep neural networks (DNNs)-based autonomous driving systems (ADSs) are expected to reduce road accidents and improve safety in the transportation domain as it removes the factor of human error from driving tasks. The DNN-based ADS sometimes may exhibit erroneous or unexpected behaviours due to unexpected driving conditions which may cause accidents. Therefore, safety assurance is vital to the ADS. However, DNN-based ADS is a highly complex system that puts forward a strong demand for robustness, more specifically, the ability to predict unexpected driving conditions to prevent potential inconsistent behaviour. It is not possible to generalize the DNN model’s performance for all driving conditions. Therefore, the driving conditions that were not considered during the training of the ADS may lead to unpredictable consequences for the safety of autonomous vehicles. This study proposes an autoencoder and time series analysis–based anomaly detection system to prevent the safety-critical inconsistent behaviour of autonomous vehicles at runtime. Our approach called DeepGuard consists of two components. The first component- the inconsistent behaviour predictor, is based on an autoencoder and time series analysis to reconstruct the driving scenarios. Based on reconstruction error ( e ) and threshold ( θ ), it determines the normal and unexpected driving scenarios and predicts potential inconsistent behaviour. The second component provides on-the-fly safety guards, that is, it automatically activates healing strategies to prevent inconsistencies in the behaviour. We evaluated the performance of DeepGuard in predicting the injected anomalous driving scenarios using already available open-sourced DNN-based ADSs in the Udacity simulator. Our simulation results show that the best variant of DeepGuard can predict up to 93% on the CHAUFFEUR ADS, 83% on DAVE-2 ADS, and 80% of inconsistent behaviour on the EPOCH ADS model, outperforming SELFORACLE and DeepRoad . Overall, DeepGuard can prevent up to 89% of all predicted inconsistent behaviours of ADS by executing predefined safety guards.

In recent years, research concerning autonomous driving has gained momentum to enhance road safety and traffic efficiency. Relevant concepts are being applied to the fields of perception, planning, and control of automated vehicles to leverage the advantages offered by the vehicle-to-everything (V2X) communication technology. This paper presents a V2X communication-aided autonomous driving system for vehicles. It is comprised of three subsystems: beyond line-of-sight (BLOS) perception, extended planning, and control. Specifically, the BLOS perception subsystem facilitates unlimited LOS environmental perception through data fusion between local perception using on-board sensors and communication perception via V2X. In the extended planning subsystem, various algorithms are presented regarding the route, velocity, and behavior planning to reflect real-time traffic information obtained utilizing V2X communication. To verify the results, the proposed system was integrated into a full-scale vehicle that participated in the 2019 Hyundai Autonomous Vehicle Competition held in K-city with the V2X infrastructure. Using the proposed system, the authors demonstrated successful completion of all assigned real-life-based missions, including emergency braking caused by a jaywalker, detouring around a construction site ahead, complying with traffic signals, collision avoidance, and yielding the ego-lane for an emergency vehicle. The findings of this study demonstrated the possibility of several potential applications of V2X communication with regard to autonomous driving systems.

The inherent limitations of individual AI models underscore the need for robust anomaly detection techniques for securing autonomous driving systems. To address these limitations, we propose a comprehensive ensemble learning framework specifically designed for anomaly detection in autonomous driving systems. We comprehensively assess the effectiveness of ensemble learning models for detecting anomalies in autonomous vehicle datasets, focusing primarily on the VeReMi and Sensor datasets. Ensemble techniques are rigorously evaluated against individual models on binary and multiclass classification tasks. The analysis reveals that ensemble models consistently outperform individual models in terms of accuracy, precision, recall, false positive rates, and F1-score. On the VeReMi dataset, ensembles achieve high performance for binary classification, with a maximum accuracy of 0.80 and F1-score of 0.86, surpassing single models. For the Sensor dataset, ensemble models like CatBoost exhibit perfect accuracy, precision, recall, and F1-score, exceeding single models by 11% in accuracy. In VeReMi multiclass classification, Stacking and Blending gave a 5% increase in accuracy compared to single models. Moreover, XGBoost and CatBoost demonstrate perfect recall. Our proposed method enhanced performance despite the increased runtime required by ensemble models. In evaluating false positive rates, ensemble learning demonstrated significant gains, reducing false positives and thereby enhancing overall system reliability.

Open-source autonomous driving systems (ADS) have become a cornerstone of autonomous vehicle development. By providing access to cutting-edge technology, fostering global collaboration, and accelerating innovation, these platforms are transforming the automated vehicle landscape. This survey conducts a comprehensive analysis of leading open-source ADS platforms, evaluating their functionalities, strengths, and limitations. Through an extensive literature review, the survey explores their adoption and utilization across key research domains. Additionally, it identifies emerging trends shaping the field. The main contributions of this survey include (1) a detailed overview of leading open-source platforms, highlighting their strengths and weaknesses; (2) an examination of their impact on research; and (3) a synthesis of current trends, particularly in interoperability with emerging technologies such as AI/ML solutions and edge computing. This study aims to provide researchers and practitioners with a holistic understanding of open-source ADS platforms, guiding them in selecting the right platforms for future innovation.

At the current stage, the rapid Development of autonomous driving has made object detection in traffic scenarios a vital research task. Object detection is the most critical and challenging task in computer vision. Deep learning, with its powerful feature extraction capabilities, has found widespread applications in safety, military, and medical fields, and in recent years has expanded into the field of transportation, achieving significant breakthroughs. This survey is based on the theory of deep learning. It systematically summarizes the Development and current research status of object detection algorithms, and compare the characteristics, advantages and disadvantages of the two types of algorithms. With a focus on traffic signs, vehicle detection, and pedestrian detection, it summarizes the applications and research status of object detection in traffic scenarios, highlighting the strengths, limitations, and applicable scenarios of various methods. It introduces techniques for optimizing object detection algorithms, summarizes commonly used object detection datasets and traffic scene datasets, along with evaluation criteria, and performs comparative analysis of the performance of deep learning algorithms. Finally, it concludes the development trends of object detection algorithms in traffic scenarios, providing research directions for intelligent transportation and autonomous driving.

Simulators are widely used to test Autonomous Driving Systems (ADS), but their potential flakiness can lead to inconsistent test results. We investigate test flakiness in simulation-based testing of ADS by addressing two key questions: (1) How do flaky ADS simulations impact automated testing that relies on randomized algorithms? and (2) Can machine learning (ML) effectively identify flaky ADS tests while decreasing the required number of test reruns? Our empirical results, obtained from two widely-used open-source ADS simulators and five diverse ADS test setups, show that test flakiness in ADS is a common occurrence and can significantly impact the test results obtained by randomized algorithms. Further, our ML classifiers effectively identify flaky ADS tests using only a single test run, achieving F1-scores of 85%, 82% and 96% for three different ADS test setups. Our classifiers significantly outperform our non-ML baseline, which requires executing tests at least twice, by 31%, 21%, and 13% in F1-score performance, respectively. We conclude with a discussion on the scope, implications and limitations of our study. We provide our complete replication package in a Github repository (Github paper 2023 ).

Autonomous driving in dense urban environments requires accurate trajectory forecasting supported by interpretable contextual evidence. This study presents a multimodal framework that performs driving scene context (DSC)-aware trajectory prediction while providing risk-aware explanations to reveal the contextual cues behind predicted motion. The framework integrates temporal object states—trajectories, velocities, yaw angles, and motion status—with semantic information from forward-facing camera imagery, and is composed of four modules: object behavioral feature extraction, scene context extraction, DSC-augmented trajectory prediction, and risk-aware reasoning using a multimodal large language model (MLLM). Experiments on the Rank2Tell dataset demonstrate the feasibility and applicability of the proposed approach, achieving an ADE of 10.972, an FDE of 13.701, and an RMSE of 8.782. Additional qualitative evaluation shows that DeepSeek-R1-Distill-Qwen-7B generates the most coherent and contextually aligned explanations among the tested models. These findings indicate that combining DSC-aware prediction with interpretable reasoning provides a practical and transparent solution for autonomous driving in complex urban environments.

Cyber-physical systems (CPSs) that interact with each other to achieve common goals are known as collaborative CPSs. Collaborative CPSs can achieve complex goals that individual CPSs cannot achieve on their own. One of the examples of collaborative CPSs is the vehicular cyber-physical systems (VCPSs), which integrate computing and physical resources to interact with each other to improve traffic safety, situational awareness, and efficiency. The perception system of individual VCPS has limitations on its coverage and detection accuracy. For example, the autonomous vehicle’s sensor cannot detect occluded objects and obstacles beyond its field of view. The VCPS can combine its own data with other collaborative VCPSs to enhance perception, situational awareness, accuracy, and traffic safety. This paper proposes a collaborative perception system to detect occluded objects through the camera sensor’s image fusion and stitching technique. The proposed collaborative perception system combines the perception of surrounding autonomous driving systems (ADSs) that extends the detection range beyond the field of view. We also applied logistic chaos map-based encryption in our collaborative perception system in order to avoid the phantom information shared by malicious vehicles and improve safety in collaboration. It can provide the real-time perception of occluded objects, enabling safer control of ADSs. The proposed collaborative perception can detect occluded objects and obstacles beyond the field of view that individual VCPS perception systems cannot detect, improving the safety of ADSs. We investigated the effectiveness of collaborative perception and its contribution toward extended situational awareness on the road in the simulation environment. Our simulation results showed that the average detection rate of proposed perception systems was 45.4% more than the perception system of an individual ADS. The safety analysis showed that the response time was increased up to 1 s, and the average safety distance was increased to 1.2 m when the ADSs were using collaborative perception compared to those scenarios in which the ADSs were not using collaborative perception.

Traffic sign detection plays an important role in improving the capabilities of automated driving systems by addressing road safety challenges in sustainable urban living. In this paper, we present DSRA-DETR, a novel approach focused on improving multiscale detection performance. Our approach integrates the dilated spatial pyramid pooling model (DSPP) and the multiscale feature residual aggregation module (FRAM) to aggregate features at various scales. These modules excel at reducing feature noise and minimizing loss of low-level features during feature map extraction. Additionally, they enhance the model’s capability to detect objects at different scales, thereby improving the accuracy and robustness of traffic sign detection. We evaluate the performance of our method on two widely used datasets, the GTSDB and CCTSDB, and achieve impressive average accuracies (APs) of 76.13% and 78.24%, respectively. Compared with other well-known algorithms, our method shows a significant improvement in detection accuracy, demonstrating its superiority and generality. Our proposed method shows great potential for improving the performance of traffic sign detection for autonomous driving systems and will help in the development of safe and efficient autonomous driving technologies.