Veritex is an object-oriented software programmed in Python. It takes in two inputs as shown in Figure 7.1, the network model and safety properties. Veritex supports the standardized format ONNX and PyTorch for the network and the unified format VNN-LIB2for the safety property. In DNN verification, VNN-LIB is the emerging standard that can specify safety properties of a DNN by defining their input domains and their corresponding unsafe output domains. Roughly for specifications, it is an extension of SMT-LIB with additional assumptions. With the network model and its safety properties, Veritex can compute the exact or over-approximated output reachable domain and also the entire unsafe input space if exists. It supports the plotting of 2 or 3-dimensional polytopes. When the repair option is enabled, it will produce a provable safe network in ONNX or PyTorch format. Unlike tools era (2021); ver (2021); cro (2021); Bak (2021); Tran et al.
(2020b); Katz et al. (2019), Veritex does not involve LP problems in the reachability analysis and verification of DNNs. Therefore, it does not require any commercial optimization solvers, which makes its installation straightforward. The main features of Veritex are summarized in Table 7.1.
2http://www.vnnlib.org/standard.html
7.2.1 Engine and Components
The engine of Veritex contains two main modules: reachability analysis of DNNs and DNN repair, as shown in Figure 7.1. The former contains functions to compute the reachable domains of a DNN. The latter contains functions to repair unsafe DNN on multiple safety properties.
7.2.1.1 Reachability Analysis Module
The module includes a solver for the computation of the reachable domain and an analyzer for the safety verification and reachable-domain visualization. The solver constructs the incoming network and its safety properties with a network object and a set of property objects. It can compute its exact or over-approximated output reachable domain. It can also compute the exact unsafe input space using the backtracking algo- rithm Yang et al. (2021a).
The exact analysis utilizes set representations FVIM and Flattice to compute output reachable sets whose union is the exact output reachable domain. These reachable sets can be sent to the verifier for a sound and complete safety verification, which returns either ”safe” or ”unsafe”. The over-approximation utilizes the set representationV-zono to over approximate the output reachable domain. This reachable domain can be sent to the verifier for a sound but incomplete safety verification, which returns either ”safe” or ”unknown”. The visualizer plots a reachable domain by projecting it into a 2 or 3-dimensional space. This visualization is critical for the analysis of the impact of repair methods on DNN reachability.
7.2.1.2 DNN Repair Module
This module eliminates safety violations through optimization of a loss function in the retraining of a DNN.
In each iteration of repair, it interacts with the reachability analysis module. Given a DNN and its violated safety properties, they are first fed into the reachability analysis module, where its exact unsafe input-output reachable domain over these properties are computed. Recall that the reachable domain consists of reachable sets, which are convex polytopes. Then, the vertices of these sets are selected as representative data pairs (x, y)to fully represent this reachable domain. They distribute over this domain, including all its extreme points. They are used to construct the distance between the unsafe reachable domain and the safe domain of the DNN. By minimizing this objective function, the repair can gradually eliminate the unsafe reachable domain, generating a provably safe DNN. When there is a safe model as a reference for the repair, adversarial xcan be fed into this model to generate safe and correctyˆfor the repair. Otherwise,yˆis set to the closet safe output toyfor the minimal modification.
In addition to the objective function above, the repair also incorporates another objective function into the loss function, which aims to minimize the DNN parameter deviation. This is because slight changes in the
parameter can cause unexpected performance degradation. This function minimizes the difference between the predicted output of the repaired network for the training data and the true output in the training data. A weighted-sum method is applied for this multi-objective optimization problem. Two positive real-valuedα andβ represent the weights of each objective function andα+β = 1. This repair is named theminimal repair. If the original dataset is not available, it can be sampled from the original network. The sampled data are purified by removing unsafe data before the training. Or users can setα= 1andβ = 0to transform the optimization into a single-objective optimization. Then, only the objective function for repair is considered, which is named thenon-minimal repair.
In practice, the solving of the minimal repair is less efficient than the non-minimal repair due to the Pareto optimality issue in the multi-objective optimization, where one objective function cannot be optimized without worsening the optimization of other objective functions.
7.2.2 Work-stealing Parallel computation
In the exact analysis, different linearities that the ReLU activation function exhibits over its input ranges x ≥0 andx < 0are separately considered. Therefore, when an input reachable set to one ReLU neuron spans its two input ranges, this set will be divided into two subsets which are separately processed w.r.t. the linearity in that range. Afterward, these two subsets will be input sets to another neuron. Here, the state S= (S, l, N)is defined for this computation, whereS is a reachable set,ldenotes the index of that layer, andN denotes a list of neurons in the layer that will processS. After one neuron, the stateSspawns at most two statesS′s with updatedS′s andN′s. This state concept is also applied in the max-pooling layer. One pooling operation normally contains more linearities than the ReLU neuron and thus spawns more states. In the affine-mapping layer, such as fully-connected layer and convolutional layer, onlyS in the state will be transformed to one new reachable setS′accordingly.
In the work-stealing parallel computing, each processor computes their states and store additional states in a local queue for future processes. One processor becomes idle when its local queue is empty. Then this processor steals states from other processors with a globally-shared queue as the agent, such that it can enable the full use of the processors. The process of states will be terminated once they reach the end of the DNN, where different callback functions can be invoked. In this phase, the reachable setS in the state is an output reachable set of the DNN. The callback functions include the safety verification and the computation of unsafe input space with the backtracking algorithm.
7.3 Reachability Analysis and Set Representations