In this chapter, we demonstrated a method for OOD detection in learning-enabled CPS. The method is based on inductive conformal prediction and anomaly detection but uses VAEs and deep SVDD to learn models to efficiently compute the nonconformity of new inputs relative to the training set and enable real-time detection of high-dimensional OOD inputs. In addition, the saliency maps can be incorporated to improve the robust- ness of the detector. Our evaluation is based on two simulation case studies of an AEBS and an SDEC as well as a real-world dataset for autonomous driving . The results demonstrate a very small number of false posi- tives and detection delay while the execution time is comparable to the execution time of the original LECs.
Table 3.4: Missed alarms and average delay in AVSD.
NCM Parameters
(N,ω,τ)/(N,τ) Missed alarms
(False negative) Average delay (frames) VAE
5,4,24 1/50 12.48
10,6,90 0/50 11.68
20,15,180 0/50 12.02
VAE, I-GOS
5,4,24 0/50 15.04
10,6,92 0/50 14.38
20,15,175 0/50 14.68
SVDD
10,6 1/50 13.21
15,5 0/50 11.74
20,7 0/50 12.10
SVDD, VBP 10,6 0/50 13.75
15,5 0/50 12.22
20,7 0/50 12.84
Table 3.5: Execution times of VAE- and SVDD-based detection methods.
N min (ms) Q1(ms) Q2(ms) Q3(ms) max (ms)
AEBS N/A 3.48 3.85 3.91 3.96 4.20
SDEC N/A 2.20 2.37 2.45 2.36 3.31
RPNN N/A 0.51 0.52 0.52 0.53 0.56
I-GOS N/A 4.32 4.37 4.41 4.43 4.47
VBP N/A 1.37 1.38 1.38 1.39 1.41
VAE 5 15.43 15.47 15.49 15.50 15.60
10 31.33 31.41 31.43 31.45 31.77
20 64.54 64.70 64.72 64.75 65.00
VAE, I-GOS
5 20.18 20.21 20.24 20.29 20.32
10 36.44 36.46 36.49 34.52 34.61
20 70.54 70.59 70.64 70.71 70.84
SVDD
10 2.12 2.35 2.44 2.45 2.52
15 2.16 2.23 2.34 2.41 2.49
20 2.28 2.33 2.34 2.59 2.60
SVDD, VBP
10 3.43 3.49 3.50 3.65 3.70
15 3.46 3.51 3.60 3.68 3.79
20 3.53 3.57 3.69 3.70 3.82
Promising future work is to combine the outputs of the neural network into OOD detection. Another possible direction is to explore the physically realizable adaptive adversarial attacks and to adapt our approach to such attacks.
CHAPTER 4
Out-of-distribution Detection using Variational Autoencoder for Classification and Regression1
4.1 Introduction
Recently, machine learning techniques, such as Deep Neural Networks (DNNs), are extensively used in a broad range of domains since they can tackle complex tasks that conventional techniques cannot easily solve. On the other hand, Cyber-Physical Systems (CPSs) are generally deployed in environments with high uncertainty and variability, which requires a high level of autonomy. It is not surprising that CPSs increasingly employ Learning-Enabled Components (LECs) to perform different complex tasks [109]. Although LECs have achieved remarkable performance, their safety and reliability should be analyzed before deploying them to real-world systems, especially safety-critical systems. Unfortunately, the characteristics and complexity of the LECs complicate such analysis. Learning techniques, such as supervised and reinforcement learning, are typically used to train LECs. Such learning techniques are built upon an underlying assumption that the training and test distribution are similar. However, even if an LEC is trained extensively, Out-Of-Distribution (OOD) data are inevitably present when the LEC is used in the real world. OOD data may lead the LEC to be ineffective and incur erroneous predictions, which may undermine the safety of the system. Therefore, runtime OOD detection is very significant and necessary to guarantee the safety and reliability of the system.
The objective of OOD detection is to quantify the degree of difference between the new test instances and the training data, and raise false alarms indicating the LEC may compute a large-error output due to the OOD data.
Although many efforts exist for OOD detection in neural networks [7], different types of OOD data are not investigated systematically. The first contribution of this chapter is the definitions for different types of OOD data present in learning-enabled CPS. We first discuss the cause of OOD examples and then categorize them into four different types: OOD data caused by (1) covariate shift, (2) target shift, (3) concept shift, and (4) label concept shift. We also provide typical examples for each type of OOD data aiming at classification and regression tasks. The categorization for the OOD data is based on the categorization of the dataset shiftfor the training and test distributions [110]. Note that dataset shifts focus on differences between the distributions of the training dataset and test dataset, while OOD detection aims at comparing a single test example with the distribution of the training dataset.
The main contribution of the chapter is an approach for detecting a variety of OOD data in learning-
1This chapter is adapted with permission from [Detection of dataset shifts in learning-enabled cyber-physcial systems,” in4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), May 2021.]
enabled CPS. Typical OOD detection techniques may result in a large number of false alarms due to the dynamical nature of CPS. In Chapter 3, a method that aims to improve the robustness of detection by using multiple examples sampled from a Variational AutoEncoder (VAE) model is proposed. The method is based on Inductive Conformal Anomaly Detection (ICAD) [63], and it is efficient so it can be used online. The chapter follows a similar approach but utilizes VAE for classification and regression models. The main benefit of such models is that they take into account both the input and output of the LEC, which enables the detection of different types of OOD data present in CPS.
Another contribution of the chapter is the comprehensive evaluation using several datasets for classifica- tion and regression tasks. We design experiments for various types of OOD data and use the same model for OOD detection. The experimental results show the proposed approach can detect different types of OOD data with a very small number of false alarms. The execution time is comparable with the sampling period of the typical CPSs, which enables real-time detection.
The outline of this chapter is as follows. Section 4.2 formulates the problem of detection of OOD data and discusses different types of OOD data in learning-enabled CPSs. Section 4.3 introduces the VAE for classification and regression model and presents the detection algorithm based on this model. Section 4.4 shows the evaluation results, and Section 4.5 concludes the chapter.
4.2 Out-of-distribution Data in Learning-enabled Cyber-physical Systems