A5.4 RECOMMENDED READING
10.2 SCENARIO ANALYSIS
Box 10.2 A Coherent Framework for Stress Testing
A risk manager typically faces two separate types of risk estimate — probabilistic estimates such as VaR or ETL, and the loss estimates produced by stress tests — with no obvious way of combining them. So how can we combine a probabilistic risk estimate with an estimate that such-and-such a loss will occur if such-and-such happens? The traditional answer is that we can’t: we have to work with these estimates separately, and the best we can do is use one set of estimates to look for possible problems with the other.
Berkowitz (2000a) suggests a solution to this problem: he suggests that we integrate stress testing into formal risk modelling by assigning probabilities to stress test scenarios. The resulting risk estimates then incorporate both traditional market risk estimates and the outcomes of stress tests, as well as the probabilities of each, and so give risk managers a single, integrated set of risk estimates to work with. This suggests the following four-step risk modelling process:
r
We go through our stress testing in the traditional way, and the outputs of this process will be a set of realised profits/losses associated with each scenario.r
Once we have gone through our scenarios and established their P/L outcomes, we go through a second, judgemental, process and assign probabilities to each of our scenarios.r
We then go through a formal risk modelling process of the traditional kind, and model our risks using appropriate risk measurement techniques. We can think of the outcome of this process as a set of P/L figures and their associated probabilities.r
We now have all the information we need, so we bring together our two sets of P/L figures and two sets of associated probabilities, and carry out an integrated risk estimation.Naturally, these estimates are dependent on the judgemental factors that go into stress testing and into the evaluation of scenario probabilities, but there is a good argument that it is better to incorporate our judgements of stress test events into risk modelling than to ignore them completely.
It is better to be approximate and probably right in our risk assessments, than to be precise and probably wrong.
the historical standard deviation of the relevant price). Some possible scenarios have been suggested by the Derivatives Policy Group (1995), and include parallel yield curve shifts of plus or minus 100 basis points, yield curve shifts of plus or minus 25 basis points, stock index changes of plus or minus 10%, currency changes of plus or minus 6%, and volatility changes of plus or minus 20%. If the institution is concerned about more extreme events, it might also want to consider such relatively rare events as 5- or 10-standard deviation changes in the relevant underlying price. We might also want to consider the impact of other factors too, such as changes in the slope or shape of the yield curve, a change in correlations, and a change in credit spreads (e.g., a jump or fall in the TED spread).
Stylised scenarios have been used for a long time in asset–liability management, where they are suited to handling portfolios that are exposed to a small number of risk factors. The usual idea is to imagine hypothetical changes in the value of each risk factor and then use pricing equations (e.g., simple linear equations for straightforward positions, duration or duration–convexity approx- imations for bonds, or delta or delta–gamma approximations for options) to determine the change in the portfolio value resulting from the market factor change. We might assume that the exchange rate rises by x%, interest rates fall by y%, and so on. Each particular combination of risk factor movements leads to a particular new portfolio value and hence a particular profit or loss. If we can combine the analysis with some assessment of the likelihood of the changes, even an informal one, these computations can give a good picture of the risks confronting our portfolio. However, the main limitation of this approach is that it easily becomes unmanageable when there is more than a small number of risk factors. If there are too many risk factors or too many different scenarios for each factor, then the risk manager can easily end up with thousands of loss figures, each for a different combination of risk factor movements. The information can be overwhelming, and the risk manager can have great difficulty in getting any overall sense of portfolio risk.
10.2.1.2 Actual Historical Events
We can also choose our scenarios from actual historical events. Historical scenarios can be based on relatively moderate market changes, which presumably have a reasonable chance of repeating themselves, or more extreme market changes, which are much less likely but more significant if they do, and they can also be based on bootstrap exercises from historical data. Historical scenarios have two advantages relative to other scenarios:
r
The fact that historical scenarios have actually occurred reduces their arbitrariness and gives them a certain plausibility that other scenarios lack. It is also hard(er) to dismiss historical scenarios on the grounds that they couldn’t happen.r
They are readily understood. A statement like ‘the firm would lose $X million if there were a repeat tomorrow of the October 1987 stock market crash’ is easy to understand, and this type of clarity is very useful in communicating risk information effectively.Whilst the precise choice of historical scenario — the data period used, the prices or price indices considered, whether and how to bootstrap, etc. — is inevitably subjective, we can make the selection process a little more systematic by using a well-produced scenario catalogue, rather than just a handful of ad hoc scenarios pulled out of thin air.6Such a catalogue might include:
r
Moderate market scenarios, such as bootstrapped recent market return scenarios, changes in market volatility, a bond market squeeze due to fiscal surpluses, changes in the euro, a widening or falling TED spread, and others from recent market experience.6A good example of such a catalogue is provided by Algorithmics’ Mark-to-Future system, which provides a wide selection of historical and other scenarios.
r
More extreme market scenarios, such as repeats of major stock market crises (e.g., the 23% fall in the Dow-Jones on October 19, 1987, the 48% fall in the Nikkei over 1990, etc.) or exchange rate crises (e.g., the ERM devaluations in September 1992, the fall in the peso in December 1994, the East Asian devaluations in 1997, the 40% fall in the rouble in August 1998, etc.), a bond market crash (e.g., the near doubling of US interest rates in 1994), major country shocks (e.g., the Latin American crisis in 1995, the Asian crisis in 1997, Russia in August 1998, and Brazil in 1999), or the failure or near failure of a large institution (e.g., LTCM in 1998, Enron in 2001).A good guide is to choose scenarios of much the same order of magnitude as the worst-case events in our historical (or bootstrapped historical) data sets. In doing so, we should obviously keep in mind that these events are notoriously difficult to predict in advance. We should also keep in mind that market experience suggests that maximum price falls vary enormously from one market to another and, within any given market, are often very much bigger than the next largest price fall.7
Box 10.3 Points to Watch for in Scenario Analysis
Many firms could improve their stress testing by looking out for the following points:8
r
Where maximum losses are associated with large changes in risk factors, it is important to allow for large changes in risk factors: in other words, stress situations should be fairly stressful.r
We need to take proper account of the speed and duration of stress events.r
We should identify key assumptions, and gauge our vulnerability to them. Unless they are made explicit, important assumptions often remain hidden.r
We should take account of linkages between risk factors, particularly in crises: we must account for connections between market, credit and liquidity risks, and so forth.r
Stress tests should be done reasonably frequently, so that results are up-to-date and relevant to the firm’s current situation.Successful stress testing also requires that the firm avoid or at least mitigate a number of common pitfalls:9
r
Senior management might not buy into the stress test exercise, and so ignore stress test results.r
Managers might fail to conduct adequate ad hoc tests because the results of standard historical and mechanical stress tests indicate that the portfolio is safe.r
Results might be evaluated by managers who lack the authority to take remedial action.r
Stress testers and managers might develop a ‘what if?’ mentality and rely too much on stress tests, or they might develop an excessively reactive mentality and rely too much on VaR or ETL.r
Stress tests can rely too much on historical scenarios, and not enough on plausible scenarios that are not reflected in the historical record.7A potential drawback with historical scenarios is that a firm can easily become over-reliant on them, and such over- reliance can make it excessively backward-looking and oblivious to new dangers. The solution is to strike an appropriate balance between historical and other possible scenarios — which is of course easier said than done. A second drawback is that it is hard to apply historical scenarios to new products or new markets, or to risk factors that are known to have changed significantly in the recent past. However, we can deal with this drawback, to some extent at least, by using suitable proxies or scenarios from comparable markets.
8For more on these points, see Wee and Lee (1999, pp. 16–17).
9For more on these, see Blanco (1999b).
r
Stress tests only capture a limited number of extreme scenarios, and stress testers and managers need to keep in mind that real extreme losses could be substantially higher.r
Stress testing can become politicised: stress tests can become a political weapon to be used or ignored by interested parties within the firm in the pursuit of other objectives, thus compromising the integrity and credibility of the stress testing process.Institutions can try to avoid these pitfalls by ensuring that senior managers buy into stress testing exercises, that all interested parties are involved in selecting scenarios, that there is a balance between purely hypothetical and historical scenarios, and that results are reported to interested parties in appropriate detail.
10.2.1.3 Hypothetical One-off Events
Scenarios can also come from plausible hypothetical scenarios that have no direct historical prece- dents. These scenarios would not be replays of past historical events, as such, although they would have some similarity with past events. These scenarios might be natural (e.g., a major earthquake in California), political (e.g., the outbreak of a war or a sovereign default), legal (e.g., a ruling on the legality of a derivatives contract), economic or financial (e.g., the default of a major financial institution or a new financial crisis), credit-related (e.g., the downgrading of a major counterparty), or liquidity-related (e.g., a major hike in credit spreads). We can often — though not always — formulate such scenarios by looking at historical experience and asking what might have been.
We can also look to the historical record to give us an indication of what such an event might look like. A good example highlighted in the recent BIS report on stress testing is a ‘flight to quality’ (Bank for International Settlements (2000, p. 12)). Such scenarios involve shocks to credit spreads, such as a jump in the TED spread. However, the flight to quality experienced in late 1998 also involved close connections between credit spreads and liquidity factors, so the flight-to-quality scenarios now used by financial institutions have been refined further to take more account of liquidity considerations, with more emphasis on liquidity-related changes in spreads, such as changes in the spread between on- and off-the-run US Treasury instruments.10
10.2.2 Evaluating the Effects of Scenarios
Having specified each scenario as fully as we can, we need to consider the effect of each scenario on the prices of all instruments in our portfolio. The key task is to get an idea of the sensitivities of our various positions to the underlying risk factors whose hypothetical changes we are considering.
This is very easy for some positions. Thus, a straight FX position changes one-for-one in value with changes in the exchange rate, and the value of a diversified stock portfolio changes (roughly) one-for-one with changes in the stock market index. Many other positions also change one-for-one (or thereabouts) with changes in the underlying market risk factor. Some other positions have less straightforward sensitivities, but we can usually handle them by using approximations. For example,
10Some authors also treat EVT or stressed VaR approaches as forms of stress test as well. An EVT approach gives us the VaR or ETL associated with an extreme event; this is very useful, but is perhaps best regarded as a form of parametric risk measurement rather than a stress test as such. However, this is a semantic issue and there is no denying that EVT is very useful for estimating extreme scenario risks. A stressed VaR exercise involves changing risk factors (e.g., volatilities or correlations) and seeing the impact on VaR. This is a stress test by anyone’s definition — and a very useful one at that — but we should keep in mind that it gives us the change in VaR (or ETL) rather than a loss as such. Stressed VaR exercises are very useful tools for helping us manage our risks, but the results of these exercises are measures of the riskiness of our risk measures, rather than ‘straight’ risk measures as such. For more on stressed VaR, see Box 10.4.
we could obtain the approximate sensitivities of option prices to changes in underlying risk factors from estimates of their deltas, gammas, vegas and other risk parameters, all of which should be readily available; and where bonds are concerned, we might proxy their sensitivities to changes in market interest rates by taking duration or duration–convexity approximations.
Once we have determined the effect of each scenario on all relevant prices, we can infer the effect of each scenario on the portfolio value as a whole. The portfolio loss is then found by subtracting the portfolio’s existing value from its post-scenario value.
In evaluating the effects of scenarios on our portfolio, we should also consider the impact of our hypothesised events on the markets in which we operate. In particular, it is very unwise to assume that markets will continue to function ‘normally’ when subjected to extreme stress. To illustrate, under normal stock market conditions we could expect to see sell orders executed within a matter of minutes; yet, on October 19, 1987, stock markets were so overwhelmed that it could take hours to get orders executed. Sell orders either expired because of time or price limits or else were executed at much lower prices than the sellers had expected. Market liquidity consequently dried up just when sellers were most dependent on it. Firms whose risk management strategies are based on dynamic hedging or an assumed ability to rebalance portfolios quickly should therefore pay considerable attention to the impact of extreme events on market liquidity. They should also watch out that volatility and correlation assumptions that may appear reasonable in ‘normal’ times do not break down when markets are stressed and leave them with much bigger exposures than they thought they might have.
Companies that use futures contracts to hedge illiquid positions should also take into account the funding implications of their hedge positions. Gains or losses in futures positions must be settled on a daily basis, while changes in other positions (e.g., forward ones) will not be settled until the position is finally closed out. Hence, even otherwise well-designed hedges can lead to mismatches between the timing of receipts and the timing of the payments that theoretically cover them. If the hedges are large, these interim funding requirements can also be large. Indeed, it was the failure to consider just this point that played a key factor in bringing the German industrial giant Metallgesellschaft to its knees in 1993–4.
Box 10.4 Stress Testing in a VaR Framework
A major problem with traditional stress testing is that it throws away valuable information, particularly about volatilities and correlations. To remedy this drawback, Kupiec (1999) proposes a new approach — a form of conditional stress testing or stress VaR approach — that seeks to make use of this information in stress testing. Suppose we have a set of risk factors that are, say, normally distributed. We partition these into two sets — a set ofkfactors,R˜1t, that are to be stressed to take valuesR1t, and those that are not,R˜2t. If the variance–covariance matrixΣis unaltered in the stress test,11the unstressed factorsR˜2t are conditionally distributed as:
R˜2t|R˜1t=R1t ∼N(µc,Σc)
whereµc=21−111R1t,Σc=Σ22−(Σ21Σ−111Σ12), and theΣ11and so forth are the partitioning sub-matrices of, given the valuesR1t of the stressed factors. Given this conditional density
11This assumption and the earlier assumption of normality are only made for convenience: they are not essential, and we can relax them if we are prepared to make the analysis a little more difficult.
function, the stress scenario change in portfolio value is a normally distributed random variable with meanX1tR1t+X2tµcand varianceX2tΣcXT2t, whereX1t andX2t are the position vectors corresponding to the two types of risk factors. Once the joint distribution of the risk factors is taken into account, our stress test thus produces a distribution of scenario loss values, not just a single loss value. If we wish, we can then focus on one of the percentile points of the scenario loss distribution, in which case our output can be interpreted as a stress VaR: the likely worst outcome at a chosen confidence level.
Alternatively, we can focus on the mean of the conditional loss distribution, in which case our stress loss isX1tR1t+X2tΣ21Σ−111R1t. This expected loss differs from the expected loss we get under traditional stress testing because it takes account of the correlations between different risk factors: under a traditional stress test, we would stress the stressed risk factors and take the other risk factors to be unaltered, and therefore get an expected loss ofX1tR1t. The results reported by Kupiec (1999, p. 12) indicate that this traditional loss measure performs very poorly in backtesting exercises, and that his proposed new expected loss measure fares better.12