A5.4 RECOMMENDED READING
10.1 BENEFITS AND DIFFICULTIES OF STRESS TESTING
Box 10.1 Using Stress Tests
In the right hands, stress testing can be a very important and useful risk management tool, and stress tests can be used for risk management in at least three ways. The first is as a source of information, and the results of stress tests can be disseminated to all levels of management or decision-makers. Stress test results can be a particularly effective means of communicating risk information because the underlying conceptual experiment — i.e., what if . . . happens? — is easy to understand and free of any dependence on the probability notions that are inescapable when using VaR or ETL risk measures. However, it is important not to swamp recipients with unnecessary data, so it is best to give each level of manager or decision-maker only the stress test information relevant to them. When used in this way, stress tests can help to assess risks in the context of the firm’s risk appetite, as well as identify major contributors to the firm’s overall exposure and reveal hidden sources of risk that might not otherwise be apparent. If they are to provide up-to-date information, stress tests also need to be carried out on a reasonably frequent basis (e.g., every week or month).
The second main use of stress tests is to guide decision-making and, in particular, to help with setting position limits, allocating capital, and managing funding risks. The usefulness of stress tests for setting positions and allocating capital is self-evident, and stress tests can help manage funding risks by identifying the circumstances in which firms might get bad headlines and run into funding problems, so that managers can take appropriate pre-emptive action.
The third use of stress testing is to help firms design systems to protect against bad events — for example, to provide a check on modelling assumptions, to help design systems to protect against stress events (e.g., to protect the firm’s liquidity in a liquidity crisis), and to help with contingency planning.
r
A stress test could take account of the unusual features of a stress scenario (e.g., such as radicalised correlations, etc.), and so help to reveal exposures that a VaR procedure would often overlook.We can give many examples where stress testing highlights exposures that probabilistic approaches to risk measurement might easily overlook. An important example is in helping to identify an institution’s breaking point — it helps to identify those types of scenario (in terms of severity and conjunction of events, etc.) that would force the institution into insolvency. To quote the Federal Reserve chairman, Alan Greenspan:
In estimating necessary levels of risk capital, the primary concern should be to address those distur- bances that occasionally do stress institutional solvency — the negative tail of the loss distribution that is so central to modern risk management. As such, the incorporation of stress scenarios into formal risk modelling would seem to be of first-order importance.
(Greenspan (2000, p. 2)) A stress test could identify these scenarios much more explicitly than other methods, and so give management a much clearer idea of the scenarios that they had to worry about. Once these scenarios are identified, it becomes much easier to develop hedging or other risk management strategies to protect against them.
A stress test is also very good for identifying and quantifying liquidity exposures: a stress test can identify liquidity risk factors that might not otherwise be apparent. Liquidity effects — such as connections between interest rates and collateral requirements or credit triggers, the impacts of widening bid–ask spreads and increasing execution times, etc. — can be quite subtle. VaR systems cannot really do them justice, but they are quite amenable to well-designed stress tests. As with solvency tests, the information provided by liquidity stress tests can be crucial in determining how to deal with the risks concerned.
A stress test can be useful in identifying the consequences of large market moves. For example, given the leverage involved in options positions, a firm that delta hedges could be covered against a very small market move and destroyed by a very large one, and the only way to detect this sort of exposure is to run stress tests based on very large hypothesised market moves (e.g., moves of 5–10 standard deviations, or more). We might also use stress tests to examine some of the other potential consequences of a large market move, including the consequences of a drying up of market liquidity, or the possible funding consequences if positive-valued derivatives positions suddenly become major liabilities and force us to put up collateral or meet margin calls.
Stress testing is also good for examining the consequences of changes in volatility. Estimates of volatility based on historical data can be unreliable, and reliance on them can, on occasion, lead to much bigger losses than might have been expected. To illustrate the point, if we take a reasonable period prior to any major exchange-rate crisis, any historically based estimate of the VaR or ETL of a relevant cross-currency portfolio would have indicated relatively little exchange-rate risk: the exchange rate would have been stable for a considerable time, so no historical approach would have had any reason to indicate major exchange-rate risk. The exchange rate then changes very abruptly and anyone on the wrong side of the market would have taken major losses, and yet this vulnerability could easily be picked up by a simple stress test. Volatility can also change suddenly in other markets as well, particularly in equity and commodities markets.
Similarly, we can also use stress tests to highlight dependence on correlation assumptions. Since the risk of a portfolio depends on the expected correlations of the various positions included in it, a major change in correlation could leave our portfolio much more exposed than we thought it was going to be. Historical correlations can themselves be very volatile, and the most drastic changes in
correlations tend to occur in crises such as market crashes.3If we wish to survive such events, it is important that we not only examine our exposure to large market moves, but also examine what we stand to lose if ‘normal’ correlations break down and markets all move against us, and the only way to gauge this sort of exposure is to carry out scenario analyses.
Last, but not least, stress tests can be very useful for highlighting other weaknesses in our risk management set-up. The process of actually going through a stress testing exercise should force risk managers and senior managers to think through the ramifications of bad scenarios, as well as help them to pinpoint weaknesses that they might have underestimated or overlooked. If it is done well, it should not only give some indication of where the institution is vulnerable, but also show up flaws in contingency planning. In fact, what risk managers learn about these hidden weak- nesses is often as valuable for risk management purposes as the loss figures that the exercise finally produces.4
10.1.2 Difficulties with Stress Tests
Stress testing is generally much less straightforward than it looks. Stress tests are based on large numbers of decisions about the choice of scenarios and/or risk factors to stress, how risk factors should be combined, the range of values to be considered, the choice of timeframe, and so forth.
Stress testing is also completely dependent on the chosen scenarios and, hence, on the judgement and experience of the people who carry out the stress tests. This is a serious drawback because, as we all know, the negative events that we want to guard against can often be hard to predict. Choosing the ‘right’ scenarios is therefore an important but sometimes very difficult task. There have been many cases in the last few years of large companies being severely embarrassed or bankrupted by events that their management did not see coming (and, in some cases, by events that they clearly shouldhave seen coming). When portfolios are complex, it can also be very difficult to identify the risk factors to look at. The usefulness of stress testing therefore boils down to the skill, good sense and intuition of those who carry out the stress tests — and, in the final analysis, this is why good risk management is at least as much craft as science.
Another problem with stress testing is the sheer difficulty of working through scenarios in a consistent, sensible way,withoutbeing overwhelmed by a mass of different possibilities. There are three main issues here:
r
We need to be able to follow through scenarios, and the consequences of some scenarios can be very complex: a trigger event occurs, and affects a number of variables; each of these affected variables then impacts on a number of others, and each other; these affect other variables; and so on. A trigger event can rapidly lead to a plethora of possibilities, and if we are not careful, the number of possibilities can become unmanageable and make the whole exercise meaningless.3To illustrate the volatility of correlations, in the first quarter of 1993, the average correlation between the Nikkei 225 and the FT-SE 100 stock market indices varied from+0.9 to−0.9 (Jackson (1996, p. 181)). Similarly, over the first quarter of 1995, correlations between the Nikkei 225 and the US$/yen exchange rate varied from less than−0.4 to about+0.7 (Mori et al.(1996, chart 3)). The evidence also indicates that correlations can jump very suddenly, and not just when there is a major market crash.
4In order to make best use of stress tests, a good practice is to specify a threshold beyond which the loss would be regarded as a serious problem. This threshold would be set in terms of the institution’s capital or in terms of the capital allocated to the business unit concerned. If a stress test threw up a loss that exceeded this threshold, the institution would respond with a formal review to examine the circumstances under which a very high loss could occur (Lawrence (1996, p. 176)). This process would look closely at the co-movements leading to the loss and assess how likely the outcome is. An informed decision can then be made as to whether and, if so, how to cover the risk.
r
In working through scenarios, we will often (though not necessarily always) want to take account of the interactions of different risks. While it is sometimes useful to carry out scenario analyses in which all correlations are assumed to move in the most damaging ways, the fact is that we will not always want to make such assumptions and, on the contrary, will often want to take account of the interrelationships between different variables. Our stress test might indicate that the maximum loss could occur when one price rises and the other falls, and yet the prices of the two assets might be very strongly correlated. The stress test then ignores the likelihood that the two prices will move up or down together, and may produce a loss estimate much higher than any loss that could plausibly occur. In using stress tests, we must therefore decide when and, if so, how to allow for correlations.r
In designing our scenarios, we must also recognise that there are often situations where prices cannot move independently of each other because doing so would violate a zero-arbitrage condition.To carry out stress testing sensibly, we need to eliminate all co-movements that are inconsistent with zero arbitrage.
Stress tests can also run into various computational problems. (1) The first of these is the need to take account of the differing sensitivities of instrument prices to underlying risk factors. The point here is that pushing all prices by the same multiple of their standard deviation ignores the sensitivity of each position to the underlying risk factors: for example, an option that is deeply out-of-the- money is insensitive to a change in the underlying price, but an option that is in-the-money could be very sensitive to it. The probability of an option price change that isαtimes the option volatility is therefore much higher for a deeply in-the-money option than for a deeply out-of-the-money option.
Consequently, it does not make much sense to push all prices by the same number of standard deviations, when the probability of such a change varies considerably from one position to another.
The solution is not to push the individual prices by any particular multiple, but to push the un- derlying risk factors instead. (2) Stress tests can be computationally expensive, and computational considerations impose a limit on how frequently they can be carried out. This is often the case where options positions are fully revalued during stress tests using intensive procedures such as simulation methods. Many firms also face computational problems because of system incompatibilities of one sort or another. (3) There are serious difficulties in integrating market and credit risk factors in stress analysis, and a recent BIS survey of stress testing in financial institutions reported that none of the surveyed firms had systems that fully integrated market and credit risk in stress testing (Bank for International Settlements (2000, p. 15)). Much of the time, integration appears to have gone little further than taking account of the impact of credit-related changes in the prices of traded instruments.
There is also the issue of probability: since stress tests as such do not give any indication of likelihood, we always face the problem of judging the importance of stress test results. Suppose a stress test suggests that a particular event would drive our firm into insolvency. Does this matter?
The answer is that we cannot say without more information. If the event concerned could occur with a significant probability, then clearly the stress test result is important and should be taken seriously.
But if the probability of occurrence was negligible, there is no real point paying much attention to it:
rational people don’t waste time and resources dealing with dangers that are too improbable to worry about. As Berkowitz (2000a, p. 12) puts it, this absence of probabilities leaves ‘stress testing in a statistical purgatory. We have some loss numbers, but who is to say whether we should be concerned about them?’ In order to use stress tests meaningfully, we need to form some idea, even a very loose and informal one, of the likelihood of the events concerned.5
5Evaluating the plausibility (or, more formally, the probability) of stress scenarios is not difficult, at least in principle, and one straightforward way to do so is suggested by Breuer and Krenn (2000, p. 16). If we identify our stress scenario in terms of ann-dimensional vector of stress factorsrstress, this vector and the factor variance–covariance matrixΣdefine
Box 10.2 A Coherent Framework for Stress Testing
A risk manager typically faces two separate types of risk estimate — probabilistic estimates such as VaR or ETL, and the loss estimates produced by stress tests — with no obvious way of combining them. So how can we combine a probabilistic risk estimate with an estimate that such-and-such a loss will occur if such-and-such happens? The traditional answer is that we can’t: we have to work with these estimates separately, and the best we can do is use one set of estimates to look for possible problems with the other.
Berkowitz (2000a) suggests a solution to this problem: he suggests that we integrate stress testing into formal risk modelling by assigning probabilities to stress test scenarios. The resulting risk estimates then incorporate both traditional market risk estimates and the outcomes of stress tests, as well as the probabilities of each, and so give risk managers a single, integrated set of risk estimates to work with. This suggests the following four-step risk modelling process:
r
We go through our stress testing in the traditional way, and the outputs of this process will be a set of realised profits/losses associated with each scenario.r
Once we have gone through our scenarios and established their P/L outcomes, we go through a second, judgemental, process and assign probabilities to each of our scenarios.r
We then go through a formal risk modelling process of the traditional kind, and model our risks using appropriate risk measurement techniques. We can think of the outcome of this process as a set of P/L figures and their associated probabilities.r
We now have all the information we need, so we bring together our two sets of P/L figures and two sets of associated probabilities, and carry out an integrated risk estimation.Naturally, these estimates are dependent on the judgemental factors that go into stress testing and into the evaluation of scenario probabilities, but there is a good argument that it is better to incorporate our judgements of stress test events into risk modelling than to ignore them completely.
It is better to be approximate and probably right in our risk assessments, than to be precise and probably wrong.