Chiang (National Eye Institute) for their extraordinary mentorship while I served on the student editorial board of the Journal of the American Medical Informatics Association. 78 4.4 The advantages of OSSP over online SSE in terms of the mean (and standard deviation) of the differences in the auditor's expected utility.
Introduction
At the same time, attacks are unfortunately never absent due to the important roles these mission-critical information systems play in facilitating human society, as well as the great value of the data they hold [7, 8, 9]. The detected suspicious events and their corresponding types are then presented to the system administrator (or auditor).
Summary of Contributions
While signaling-based online auditing that takes advantage of the auditor's information advantage has the potential to outperform non-signaling strategies, we specifically evaluate the auditor's expected utility among our solutions and the state-of-the-art auditing method under various conditions to demonstrate the value of the new auditing solutions and their scalability.
Dissertation Structure
Alert Prioritazation
Alert Frameworks
There have been a number of investigations into effective alert management strategies and effective auditing mechanisms for information systems. Although these methods generate alerts for investigators, they lead to a significant number of false positives—a problem that can be mitigated by alert prioritization schemes.
Alert Burden Reduction
Stackelberg Security Games
Since alert realizations are stochastic, this results in complex interactions between the defender and attackers and results in a very complex policy preference space for the defender.
Audit Games and Alert Prioritization
Specifically, neural reinforcement learning was used to calculate approximately optimal policies for each player in response to a fixed policy of the other player. Although improved audit performance compared to previous research is shown in controlled test scenarios, it is unclear whether the strong assumptions about attackers (eg, a full knowledge of the state of the detection environment) can weaken the protection effectiveness of this model in practice.
Signaling in security games
Laszkaet al. the first modeled the alert prioritization problem as a game in which the auditor determines the order of the audit based on alert types [102]. However, this game has two obvious drawbacks, assuming that 1) the identity of the particular attacker was unknown, and 2) an exhaustive audit strategy would be used for all types of alerts for a given order.
Imperfect Rationality
In this chapter, we present an auditing approach that accounts for adversarial behavior by (1) prioritizing the order in which alert types are investigated and (2) providing an upper bound on how many resources should be allocated to each type. Then we present a method that combines linear programming, column generation, and heuristic search to derive an audit policy.
Introduction
This is particularly problematic because large volumes of false alarms can easily exceed the capacity of administrative officials who are expected to follow up but have limited resources [117]. However, this is an inadequate strategy because would-be infringers may be strategic and thus make decisions about the specific infringements they can undertake by balancing the possibility of an audit against the benefits of infringement.
Game Theoretic Model of Alert Prioritization
System Model
Ft(n) Probability that most alerts are of type OOO The set of all priority rankings of alerts over T. Let T be the set of alert types or categorical labels assigned to different types of suspicious behavior.
Game Model
Second, we allow the auditor to choose an arbitrary policy over alert ordering, where pooo is the probability of orderingoooo over alert types, while the thresholds are deterministic and independent of the alert priorities chosen. The approximation comes from the fact that we use the benign counts in the denominator to approximate the sum of the false positive alerts and true positive alerts in type.
Solving the Alert Prioritization Game
Column Generation Greedy Search
Within the process of generating a new column, we useΓ′(ooo′′′+t) to denote the parameter column with the audit order(ooo′′′+t). The intuition behind CGGS is that while generating a new auditor, we eagerly add one alert type at a time to minimize the lower cost given the order generated so far.
Iterative Shrink Heuristic Method
Controlled Evaluation
Data Overview
The adversary's benefit to a successful attack, attack cost, and audit cost are directly related to the alert type, as shown in Table 3.2a. Additionally, the entrapment penalty is set to a constant value of 4. a) Parameters for warning types in the synthetic setting.
Optimal Solution with Varying Budget
To investigate the performance of the proposed audit model, we assigned a vector of audit budgetsB that has a wide range with respect to the scale of the resources of the alert types. As expected, it can be seen that as the budget increases, the optimal value of the objective function (minimized by the auditor) decreases monotonically.
Findings
Next, we consider the computational burden for ISHM to achieve an approximate target of the optimal solution. In contrast, a greater amount of effort is required in the middle of the budget range due to more frequent restarts (although this yields only a small improvement).
Model Evaluation
Data Overview
In the description field, the italicized words represent the purpose of the application, and the other words represent the property values. The vector of benefits to the adversary is for each of the alert types generated.
Comparison with Baseline Alternatives
The budget of 100 covers approximately 1/4 of the sum of the resources of the seven signaling types. This rating is less than 1/4 of the sum of the distribution means of all alarm types.
Disccusion
Thus, a fruitful direction of research is in the payoff structures and how they affect the performance of the model. A natural follow-up investigation is therefore to relax such a strong assumption by involving uncertainty in the knowledge of the players.
Conclusion
However, attacks in the wild may take multiple cycles to fully execute, allowing the auditor to catch the attacker before he completes his exploit. In this chapter, we formalize this audit problem as a Signaling Audit Game (SAG), where we model the interactions between an auditor and an attacker in the context of signaling and where the usability cost is represented as a factor of the auditor's payoff. .
Introduction
This can be addressed by strategically revealing information to the attacker, a mechanism referred to as targeting (or persuasion[137, 108]). We exploit the time gap between the access request made by the (potential) attacker and the actual execution of the attack to introduce the signaling mechanism.
Online Signaling in Audit Games
Motivating Domain
The results of a comprehensive comparison, performed over a range of conditions, indicate that the SAG consistently outperforms state-of-the-art game-theoretic alternatives that lack signals, achieving higher overall utility while incurring nominal increases in computational burden. Next, a subset of the alerts are retroactively audited at the end of each audit cycle, and the auditor determines what constitutes an actual policy violation.
Signaling Audit Games
If the auditor fails to check a victim alert of typet, the auditor and the attacker receive the utility Ud,ut and Ua,ut , respectively. The auditor continues to update the real-time probability of checking an alert (triggered or not) with respect to alert type and time.
Optimizing SAGs
Online SSG
Now let θt′(t) be the probability of auditing an alarm of type t′ when the attacker's best response is After solving |T| occurrences of LP (4.2), the best solution for the auditor will henceforth be referred to as the online SSE strategy (or simply SSE),θSSE.
Optimal Signaling
Finally, we select the best solution (in terms of the auditor's utility) among all the MPs as the SSE strategy. For each type t′, we set this loss as proportional to the product of the probability of sending alerts pt1′+qt1′, the probability of being deterred Pt′ and the expectation of the number of future false positive alerts until the end of the current audit.
The Ending Period of Audit Cycles
The first constraint in LP (4.3) ensures that the attacking type is the best response strategy for the attacker. We refer to the optimal solution among|T|instances of LP (4.3) as the Online Stackelberg Signaling Policy (OSSP).
Theoretical Properties of SAGS
The expected utility value of the auditor in OSSP is thus never worse than that in online SSE. This raises the following question: can the use of OSSP provide greater benefit to the expected utility of the auditor.
Model Evaluation
Dataset
We focus on the following alarm types: . employee and patient: 1) share the same last name, 2) work in the same department, 3) share the same residential address, and 4) are neighbors within a distance of less than 0.5 miles.
Experimental Setup
Remember that the SAG auditor must reserve a portion of the total audit budget for inspecting the repeat data requests at the end of each audit cycle. Considering that the estimated payoff structure may not be perfect, we also test the robustness of the results by varying the values in the given payoff structure.
Results
Here, this value is defined as the absolute improvement in the auditor's expected utility divided by the expected utility of the optimal auditor in the online SSE. First, it is notable that OSSP consistently outperforms the online SSE in terms of expected auditor utility in different audit environments.
Discussion
In addition, we tested the average run time for SAG optimization on a single alert over all test days. As a consequence, it is unlikely that system users will not perceive the additional processing time associated with SAG optimization in practice.
Conclusion
In this case, the auditor's optimal strategy for each alarm is the same as in the single attacker scenario. We demonstrate in both environments that our robust solutions greatly improve database access auditing performance compared to the state-of-the-art.
Introduction
As such, the types of attackers an auditor may encounter can largely influence the defense strategy and thus the auditor's expected utility. In other words, the auditor's defense strategy selection must be robust to attackers who are not perfectly rational.
Preliminary and Notations
As a result, the probability of sending a warning is p1+q1 and the probability that an alert will be investigated – regardless of warning or not – is p1+p0. If an alert of type is indeed an attack and is not audited, then the auditor and attacker will receive utilityUd,ut and Ua,ut, respectively.
Robust Bayesian SAG
Bayesian SAG
Following the utility cost design in [38], we update the auditor's expected utility for any time point as Eτd(util) =∑t ∑lθlzlt·(pt0Ud,ct,l +qt0Ud,ut,l) + (pt1+qt1) ·PtEτtCt. Here the utility cost ie. second term proportional to 1) the probability that the data requester is alerted when the alert is triggered, 2) the probability that a non-malicious user leaves after receiving the alert. Pt, 3) the expected number of false positive alerts from the current moment (that is, when the alert is triggeredτ) to the end of the audit cycleEτt, and 4) the loss (eg system efficiency).
Imperfect rationality and robust strategies
The left side of this inequality shows the auditor's loss when the attacker attacks the target (instead of choosing the best target). This design does not impose a hard breakpoint on attackers, but instead allows for a more gradual defense against an attacker's diversion.
Optimizing robust Bayesian SAG
Here, γl is used to represent the utility of the auditor (without utility cost) compared to the best strategy of the type attacker. Instead, we use the fourth constraint to limit the auditor's potential loss as a function of the attacker's loss.
Theoretical Properties
Clearly, the remaining constraints can be satisfied for the newly defined variables. This makes γl in MILP (5.2) the lower bound of the expected utility of an attacker in type l, which is indicated by the fourth constraint of MILP (5.2).
Model Evaluation
Dataset
The information associated with each EHR access event includes the user's and patient's names, their residential addresses, and if a patient is also an employee of VUMC, and their department affiliations. The description of alert types used in this study and their daily statistics are provided in Table 5.1.
Experimental Setup
To assign values to {Eτt}t∈T, we use the state at the first triggered alert on the first evaluation day (i.e. day 42) in the real environment. Similar to the evaluation in the real environment, we evaluate the performance based on a set of values of the two robustness parametersε andβ.
Results
We emphasize that ε-robust Bayesian OSSP for ε ≥800 and β-robust Bayesian OSSP for β ≤0.125 achieve the highest CEU of the auditor. First, ε-robust Bayesian OSSP and β-robust Bayesian OSSP consistently outperform Bayesian OSSP in terms of average auditor CEU in different audit settings.
Discussion and Conclusion
Consequently, this data set is likely to be representative of certain values of εinβalready. Second, although we performed the evaluation of different game profiles, we did not study how to adjust the values of ε or β in practice.
Summary
It incorporated real-time information sharing between players and made it an audience advantage to influence attackers. The most salient finding is that the auditor's informational advantage can be translated into the benefit of its expected utility in the data misuse audit scenario.
Future Investigations
Proceedings of the 2016 International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT), p. In Proceedings of the 2008 International Joint Conference on Autonomous Agents and Multi-Agent Systems, pp.
A legend of the notation used in this chapter
Description of Dataset Syn A
The optimal solution for the auditor under various budgets
The approximation of the optimal solutions obtained by ISHM at various
The approximation of the optimal solutions obtained by ISHM + CGGS at
The average precision over the budget vector B by applying ISHM and
The number of threshold vectors checked by ISHM with a given budget B
Description of the EHR alert types
Description of the defined alert types
A summary of the daily statistics per alert types
The payoff structures for the pre-defined alert types
