In the context of AV ODD, collision risk assessment faces challenging situations such as incorrect sensor information and unexpected algorithmic errors arising from uncertain environments (weather, traffic flow, road conditions, obstacles). To this end, research based on artificial intelligence has been developed in recent years and exploits previous experience and information from sensors (especially the camera image) to assess the risk of collision through visual cues.
Related Work
Through this, they have developed a system that allows us to monitor the safety of AVs in real situations by learning models that predict failures from the present to specific sequences in the future. The authors of [39] propose an AI model that allows us to predict the degree of risk of the current scene through a threatening situation element for AVs. The masked images are then fed into the convLSTM network model to learn how to predict the risky action in the actual lane changing situation.
However, the problem with these models has to do with an individual's subjectivity to the ground truth, because they use real-world driving data to predict risk. Furthermore, the real-world driving data is 1) difficult to handle all driving scenarios, 2) difficult to obtain collision data, and 3) cannot address algorithmic errors arising from component errors in the AVs. On the other hand, in [40] the author obtained accident data from simulations to predict impending collisions.
The advantage of this model is that simulations can be used to utilize real accident data as well as difficult real-world scenarios.
Contribution
The ground truth for determining the dangerous action was subjectively assessed in each case by ten commentators watching the video clip. Both of the above models can predict the risks arising from the challenging situations that AVs may face. However, they still do not address 1) systematic scenario generation to ensure realistic scenarios and 2) algorithmic errors due to component failures that can pose a fatal hazard in autonomous vehicles.
The collected data was used to train the convLSTM network to monitor future collisions after a specific time step. Simulations for various situations were performed to verify the effectiveness of the trained model against image-based models. Next, the methodology for generating concrete fault injection scenarios and the specific process for obtaining training data sets are described in Section 3.
In this section, the process of dataset acquisition necessary for training AI algorithms is demonstrated.
STPA and Ontology-based Scenario Generation
Finally, a concrete scenario is created to represent exactly one particular scene to perform the simulation. Various sampling techniques can be used for efficient exploration of corner cases while automatically generating concrete scenarios. In this paper, concrete scenarios are automatically generated using the Upper Confidence Bound (UCB) algorithm, one of the multi-armed bandit problem strategies.
The problem with multi-armed bandits is to find the most rewarding slots through multiple trials of N slots with different rewards. For this reason, the UCB algorithm was used to select slots that showed good rewards through fair exploration rather than random exploration, and that could be the optimum. In Formula 1, it is a hyperparameter that can adjust the degree of exploration. Nt(a) is the number of times the slot was selected.
Automated Driving System and Simulator
Atmospheric LiDAR attenuation rate, overall fall rate, zero fall intensity, noise stddev Table 1: Examples of various sensor errors.
Generation of Training Dataset
Therefore, convLSTM was used to train the safety monitoring system by setting the front camera image and information obtained through the simulation.
AI-based Safety Monitoring: An Overview
Insight into the Convolution Long-Short Term Memory Network
Typically, in deep learning models, RNN networks have played an important role in modeling inputs and outputs related to time. In this section, the process and components of the proposed AIFAF are described in detail. The spatial data is given by the network's input sensorX1, ..,Xt, the cell output isC1, ..,Ct, the hidden state H1, ..,Ht, and the gates are inputit, forgotft, and the outputot gate of the convLSTM cell is shown in Figure.14.
The input gate determines whether to include information about the memory cell and the forget gate ft plays a role in removing information on the state of the cell.
The Proposed System Architecture
In this section, previous experience and sensor information (especially camera image) are taught to the safety monitoring system, while the network is trained to predict the collision risk as a binary classification. The test dataset and simulation environment quantitatively analyze the performance of the security monitoring system.
Training the Proposed System Using Simulation-assisted Dataset
Testing the Proposed System on Simulation-assisted Dataset
We also present the difference in predictive performance when incorporating error information compared to baseline models. Scenarios such as GPS error, detection error due to camera errors, and localization error due to LiDAR error were used in different situations to quantitatively evaluate the trained risk prediction model. The risk predicted by the safety monitoring system was compared with the actual value of the simulation result (taken as the ground truth), saved for quantitative evaluation.
Figure.20 shows the confusion matrix derived from the actual and the result values predicted with these sequences.
Case Study the Proposed System on Simulation Environment
Comparative Analysis
Iyer, "Avfi: 자율 주행 차량을 위한 결함 주입", 2018년 신뢰할 수 있는 시스템 및 네트워크에 관한 제48차 연례 국제 eeee/ifip 컨퍼런스(dsn-w) 워크숍. 연구실 인턴으로 일하기 시작한 지 벌써 2년 반이 지났고, 벌써 석사 학위도 마쳤습니다. 오랜 시간 동안 많은 분들을 만나 힘들고 행복한 순간들을 함께 나누었고, 그분들 덕분에 제가 석사학위를 무사히 마칠 수 있었다는 마음을 담아 이 글을 쓰게 되었습니다.
먼저 지도교수이신 신권철현 교수님께 감사드립니다. 연구실에 사람이 별로 없던 시기에 인연을 시작했고, 인턴 시절부터 늘 내 관심 분야에 맞는 연구 방향을 생각해 주시고, 나에게 많은 기회와 다양한 경험을 주셨다. .선생님의 가르침이 제가 연구 외적으로 많은 진전을 이룰 수 있게 해주고, 앞으로 좋은 기회를 만들어가는 밑거름이 되었기 때문에 깊은 감사의 말씀을 전하고 싶습니다. 바쁜 일정에도 불구하고 석사과정의 검토를 기꺼이 맡아주신 손흥선 교수님, 오현동 교수님에게도 감사드립니다. 항상 건강하시길 바랍니다. 다음으로, 매 순간 함께해주신 연구실 팀원들에게 감사의 말씀을 전하고 싶습니다.
먼저, 인턴 시절부터 연구실 적응에 도움을 준 은민, 형철, 호정, 호진, 영임, 형준에게 인사를 전하고 싶습니다. 연구실 출퇴근부터 운동, 음주까지 모든 순간을 함께했던 준성, 일승과 영임에게 함께 일할 수 있어서 즐거웠다고 말하고 싶다. 연구와 프로젝트를 함께 해주신 현빈님, 상현님 정말 고생많고 고생하셨습니다. 항상 옆에서 즐거움을 선사해주는 상훈이, 광록이, 주상이에게도 꼭 챙겨주고 있다는 말을 꼭 전하고 싶다. 감사하다고 말씀드리고 싶습니다. 그리고 늘 열심히 배워가는 동화, 강민이에게 많은 것을 배울 수 있어서 감사하다고 말씀드리고 싶습니다. 그리고 2022년 산업자원부 자율주행대회는 정말 기억에 남고 멋진 시간이었고, 비록 매우 실망스러운 결과였지만 귀중한 시간이었다는 점을 모든 팀원, 교수님들께 전하고 싶습니다. 우리는 열심히 일했고 기쁨과 슬픔을 함께 나눌 수 있었기 때문입니다. 마지막으로, 제가 대학원 생활에 전념할 수 있도록 정기적으로 연락하지는 못하지만 격려와 지원을 해주신 아버지 어머니, 누나에게 감사와 사랑의 마음을 전하고 싶습니다.
