cept for each. We then investigate the theoretical properties of these solution concepts and the relationship between them.
We evaluate the performance of the robust Bayesian SAG in two environments: 1) a real environment associated with the audit logs of over 10 million real EHR accesses from Vanderbilt University Medical Center (VUMC) and 2) a simulated controlled environment derived from the real data. We specifically evaluate the expected utility of the auditor between the proposed solutions and the state-of-the-art auditing method in different condi- tions to demonstrate the value of the new auditing solutions and their scalability.
corresponding EHR.
• STEP 3: The system returns the requested record.
• STEP 4: The user interacts with the returned EHR.
A SAG is played in real time between an auditorand an attacker within a predefined audit cycle, as shown in Fig. 5.1a. The auditor assumes each incoming alert is triggered by an attacker such that all interactions shown are carried out each time. For each access request that triggers an alert in real time by the misuse detection system, the auditor needs to determine: 1) which signal to send to the requestor inreal time(e.g., warn the requestor or not), and 2) whether or not to audit the alert at the end of the audit cycle. The warning sent to the data requestor can take many forms, but it is typically presented as a message along the lines of “Your access might be investigated. Proceed or quit?”, along with one button forProceed and the other for Quit. The requestor can then click the button corre- sponding to their decision. When no warning is sent (or silent signal), the requested data will be returned to the requestor automatically without any further interaction. This process depends on four probabilities, as shown in Fig. 5.1b, which are defined as the signaling scheme.
Formally, p1denotes the joint probability that 1) a warning is sent to the requestor re- garding the triggered alert, and 2) this alert will be investigated by the auditor. By contrast, q1 is the joint probability that 1) a warning is sent to the requestor, and 2) this alert will not be investigated. Similarly, p0 and q0 are defined in the scenario where no warning is sent to the requestor. As a result, the probability of sending a warning is p1+q1 and the probability that an alert will be investigated–regardless of warning or not–is p1+p0. Due to the fact that there exist multiple predefined alert types, each of which corresponds to a potential type of violation (or attack type), the signaling scheme is designed to be alert type specific. We use{pt1,qt1,pt0,qt0}to represent the signaling scheme of alertt ∈T, where T is a finite set of alert types. Alerts with the same type are considered equivalent in terms of
the loss and reward to players.
Commit to a resource allocation strategy
Access request over a target Trigger an alert and
send a warning
Proceed to attack or quit Update available budget
randomly chooses alerts to audit
Alert at timeτ "
Auditcycle
Time
(a) An illustration of the SAG over time. Each block in the time- line denotes a series of interactions for an alert triggered by the system. Different colors represent different alert types.
Warning? Attack? Audit?
!"
!0
#"
#0
!0+#0
!"+#"
1
signalingscheme
(b) The decision tree of players and the signaling scheme to be optimized in SAG. White nodes represent the end points that are not in the space of the signaling scheme.
Figure 5.1: Interactions between auditor and attacker in SAG.
We define the payoff structures (i.e., quantified utility in terms of rewards and penal- ties) of players by{Ud,ct ,Ud,ut ,Ua,ct ,Ua,ct }t∈T, wheredandaindicate defender (auditor) and attacker, respectively, and c andu represent the scenarios where an attack is covered (or investigated) and not covered, respectively. If an alert of typet is indeed an attack, and it is not audited, then the auditor and the attacker will receive utilityUd,ut andUa,ut , respectively.
In the real world, it naturally holds true thatUa,ct <0<Ua,ut andUd,ct ≥0>Ud,ut .
The audit task for each audit cycle is constrained by an auditing budget B. We use Bτ to represent the available budget when alert τ of type t is triggered. Let δt be the probability of auditing alerts of type t and letdt be the number of alerts associated with this type. Naturally, the budget constraint ∑tδtVtdt ≤Bτ will be satisfied, where Vt is the cost to audit an alert of type t. Note that the available budget is updated after each round of interactions (as shown in Figure 5.1a). Specifically, if a warning signal was sent to the requestor for alert τ, then the available budget for the next alert τ+1 becomes Bτ+1 =Bτ−pt1/(pt1+qt1)·Vt. By contrast, if there was no warning sent out for alert τ, then the budget becomesBτ+1=Bτ−pt0/(pt0+qt0)·Vt .
To optimize the signaling schemes for each triggered alert and the budget allocation strategy over all alert types in an online manner,Yan et al.proposed a solution based on the budget constraint—Online Stackelberg Signaling Policy (OSSP)[38]. The core of OSSP is the following set of constraints:
Eta(util|warning) = pt1
pt1+qt1·Ua,ct + qt1
pt1+qt1·Ua,ut ≤0 ∀t∈T,
which forces the attacker’s expected utility over each target to be non-positive. In other words, this setup ensures that the attacker’s best response strategy to a warning is to quit.
In this scenario, both players will receive zero utility. As such, the expected utility for the attacker and the auditor is:
Eta/d(util) = pt0·Ua/d,ct +qt0·Ua/d,ut .
The OSSP is derived from the strong Stackelberg equilibrium [157, 86], which assumes that the attacker will break ties in favor of the defender. Thus, the OSSP is computed by solving multiple linear programs (LP), each assuming a distinct alert type is the best strategy for the attacker. The solution is the one that produces the largest expected utility
for the auditor.
To summarize, the SAG is essentially a leader-follower game that is a unique variant of the Stackelberg security game (SSG) [110]. The SAG leverages the time gap between moves of players and the potential impact of information exchange during this time, which induces a larger action space than a typical SSG and provide an opportunity to favor the auditor using their information advantage. Accordingly, the OSSP is a variant of a Strong Stackelberg equilibrium that is specific to a SAG, where 1) the auditor commits to a ran- domized joint signaling and auditing strategy in real time and 2) the attacker decides first about which alert type to induce and, subsequently, whether to proceed when receiving a warning.
Though a SAG provides mathematically effective auditing strategy, it is limited for practical use. This is because it oversimplifies the practical scenario where there could be more than one attacker types with distinct goals, each exhibiting a different payoff structure for the same target. In addition, the SAG neglects the fact that attackers often function under imperfect rationality. A failure to consider either of these facts in deriving audit solutions can lead to excessive loss for the auditor.