Forecasting Expected Returns in the Financial Markets

(1)

(2)

Forecasting Expected Returns in the Financial Markets

(3)

Quantitative Finance Series

Aims and Objectives

• Books based on the work of financial market practitioners and academics

• Presenting cutting-edge research to the professional/practitioner market

• Combining intellectual rigor and practical application

• Covering the interaction between mathematical theory and financial practice

• To improve portfolio performance, risk management and trading book performance

• Covering quantitative techniques

Market

Brokers/Traders; Actuaries; Consultants; Asset Managers; Fund Managers; Regulators; Central Bankers; Treasury Officials; Technical Analysis; and Academics for Masters in Finance and MBA market.

Series Titles

Derivative Instruments; theory, valuation, analysis Managing Downside Risk in Financial Markets Economics for Financial Markets

Performance Measurement in Finance Real R&D Options

Advanced Trading Rules, Second Edition

Advances in Portfolio Construction and Implementation Computational Finance

Linear Factor Models in Finance Initial Public Offerings

Funds of Hedge Funds Venture Capital in Europe

Forecasting Volatility in the Financial Markets, Third Edition International Mergers and Acquisitions Activity Since 1990

Corporate Governance and Regulatory Impact on Mergers and Acquisitions Forecasting Expected Returns in the Financial Markets

Series Editor Dr Stephen Satchell

Dr Satchell is the Reader in Financial Econometrics at Trinity College, Cambridge; Visiting Professor at Birkbeck College, City University Business School and University of Technology, Sydney. He also works in a consultative capacity to many firms, and edits the journal Derivatives: use, trading and regulations and the Journal of Asset Management.

(4)

Forecasting Expected Returns in the Financial Markets

Edited by

Stephen Satchell

AMSTERDAM •BOSTON •HEIDELBERG •LONDON NEW YORK •OXFORD •PARIS •SAN DIEGO SAN FRANCISCO •SINGAPORE •SYDNEY •TOKYO

Academic Press is an imprint of Elsevier

(5)

Academic Press is an imprint of Elsevier 84 Theobald’s Road, London WC1X 8RR, UK

30 Corporate Drive, Suite 400, Burlington, MA 01803, USA 525B Street, Suite 1900, San Diego, California 92101-4495, USA First edition 2007

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone (+44) (0) 1865 843830; fax (+44) (0) 1865 853333;

email: [email protected]. Alternatively you can submit your request online by visiting the Elsevier web site at http://elsevier.com/locate/permissions, and selecting Obtaining permission to use Elsevier material

Notice

No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein.

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library Library of Congress Cataloging in Publication Data

A catalog record for this book is available from the Library of Congress ISBN: 978-0-7506-8321-0

For information on all Academic Press publications visit our web site at http://books.elsevier.com

Printed and bound in Great Britain

07 08 09 10 11 10 9 8 7 6 5 4 3 2 1

Working together to grow libraries in developing countries

www.elsevier.com | www.bookaid.org | www.sabre.org

(6)

Contents

List of contributors ix

Introduction xi

1 Market efficiency and forecasting 1

Wayne Ferson

1.1 Introduction 1

1.2 A modern view of market efficiency and predictability 2

1.3 Weak-form predictability 3

1.4 Semi-strong form predictability 5

1.5 Methodological issues 8

1.6 Perspective 10

1.7 Conclusion 12

References 12

2 A step-by-step guide to the Black–Litterman model 17 Thomas Idzorek

2.1 Introduction 17

2.2 Expected returns 18

2.3 The Black–Litterman model 21

2.4 A new method for incorporating user-specified confidence levels 32

2.5 Conclusion 36

References 37

3 A demystification of the Black–Litterman model: managing quantitative

and traditional portfolio construction 39

Stephen Satchell and Alan Scowcroft

3.1 Introduction 39

3.2 Workings of the model 40

3.3 Examples 42

3.4 Alternative formulations 46

3.5 Conclusion 50

Appendix 50

References 53

4 Optimal portfolios from ordering information 55 Robert Almgren and Neil Chriss

4.1 Introduction 55

4.2 Efficient portfolios 58

4.3 Optimal portfolios 70

(7)

vi Contents

4.4 A variety of sorts 77

4.5 Empirical tests 82

4.6 Conclusion 95

Appendix A 96

Appendix B 97

References 99

5 Some choices in forecast construction 101

Stephen Wright and Stephen Satchell

5.1 Introduction 101

5.2 Linear factor models 104

5.3 Approximating risk with a mixture of normals 106 5.4 Practical problems in the model-building process 108 5.5 Optimization with non-normal return expectations 112

5.6 Conclusion 115

References 115

6 Bayesian analysis of the Black–Scholes option price 117 Theo Darsinos and Stephen Satchell

6.2 Derivation of the prior and posterior densities 119

6.3 Numerical evaluation 131

6.4 Results 134

6.5 Concluding remarks and issues for further research 140

Appendix 142

References 148

7 Bayesian forecasting of options prices: a natural framework for

pooling historical and implied volatility information 151 Theo Darsinos and Stephen Satchell

7.2 A classical framework for option pricing 155

7.3 A Bayesian framework for option pricing 156

7.4 Empirical implementation 163

7.5 Conclusion 172

Appendix 173

References 174

8 Robust optimization for utilizing forecasted returns

in institutional investment 177

Christos Koutsoyannis and Stephen Satchell

8.2 Notions of robustness 178

8.3 Case study: an implementation of robustness via forecast

errors and quadratic constraints 182

8.4 Extensions to the theory 184

8.5 Conclusion 187

References 188

(8)

Contents vii

9 Cross-sectional stock returns in the UK market: the role of liquidity risk 191 Soosung Hwang and Chensheng Lu

9.2 Hypotheses and calculating factors 193

9.3 Empirical results 196

9.4 Conclusions 211

References 212

10 The information horizon – optimal holding period, strategy aggression

and model combination in a multi-horizon framework 215 Edward Fishwick

10.1 The information coefficient and information decay 215 10.2 Returns and information decay in the single model case 217

10.3 Model combination 221

10.4 Information decay in models 222

10.5 Models – optimal horizon, aggression and model combination 224

Reference 226

11 Optimal forecasting horizon for skilled investors 227 Stephen Satchell and Oliver Williams

11.2 Analysis of the single model problem 228

11.3 Closed-form solutions 232

11.4 Multi-model horizon framework 236

11.5 An alternative formulation of the multi-model problem 241

11.6 Conclusions 243

Appendix A 244

Appendix B 246

References 250

12 Investments as bets in the binomial asset pricing model 251 David Johnstone

12.2 Actual versus risk-neutral probabilities 252

12.3 Replicating investments with bets 255

12.4 Log optimal (Kelly) betting 256

12.5 Replicating Kelly bets with puts and calls 258

12.6 Options on Kelly bets 259

12.7 Conclusion 260

References 261

13 The hidden binomial economy and the role of forecasts

in determining prices 265

Stephen Satchell and Oliver Williams

13.2 General set-up 266

13.3 Power utility 271

(9)

viii Contents

13.4 Exponential utility, loss aversion and mixed equilibria 276

13.5 Conclusions 277

Appendix 278

References 278

Index 281

(10)

List of contributors

Robert Almgren

Departments of Mathematics and Computer Science, University of Toronto, Toronto, ON M5S, Canada

Neil Chriss

Department of Mathematics and Center for Financial Mathematics, University of Chicago, Chicago, IL 60637, USA

Theo Darsinos

Faculty of Economics, University of Cambridge, Cambridge CB2 1TQ, UK Wayne Ferson

Marshall School of Business, University of Southern California, 701 Exposition Blvd, Los Angeles, CA

Edward Fishwick

Managing Director, Head of Equity Risk & Quantitative Analysis, BlackRock, 33 King William Street, London, EC4R 9AS, UK

Soosung Hwang

Reader in Finance, Cass Business School, 106 Bunhill Row, London EC1Y 8TZ Thomas Idzorek

Director of Research, Ibbotson Associates, 225 North Michigan Avenue, Chicago, IL, USA

David Johnstone

School of Business, University of Sydney, NS 2006, Australia Christos Koutsoyannis

Deputy Head of Quant Research, Old Mutual Asset Managers (UK) Ltd, 2 Lambeth Hill, London EC4P 4WR

Chensheng Lu

Faculty of Finance, Cass Business School, 106 Bunhill Row, London EC1Y9TZ, UK Stephen Satchell

Reader in Financial Econometrics, Faculty of Economics, University of Cambridge, Cambridge CB2 1TQ, UK

(11)

x List of contributors

Alan Scowcroft

Head of Equities Quantitive Research at UBS Warburg Oliver Williams

PhD student, King’s College, University of Cambridge, Cambridge, UK Steve Wright

Fixed Income Department, UBS Investment Bank

(12)

Introduction

This book about forecasting expected returns covers a great deal of established topics in the forecasting literature, but also looks at a number of new ones. Wayne Ferson contributes a chapter on market efficiency and forecasting returns. Thomas Idzorek and Alan Scowcroft provide analyses of the Black–Litterman model; a methodology of great importance to practitioners in that it allows for the combination of analysts’ views with equilibrium modelling.

Several chapters concern ranked returns. Neil Chriss and Robert Almgren provide an original and stimulating analysis of the use of rankings in portfolio construction. On the same topic, Steve Wright shows how ranking forecasts can be implemented in practice.

Theo Darsinos has contributed two chapters concerned with forecasting option prices;

this is an under-studied area.

Christos Koutsoyannis investigates how robust optimization can be used to improve forecasting returns. He also provides a thorough overview of the robust optimization literature. Soosung Hwang looks at the role of liquidity as an explanatory variable in forecasting equity returns.

Edward Fishwick and Oliver Williams contribute two chapters on the analysis of forecasting horizons – again, an area where very little work has been published.

Finally, David Johnstone and Oliver Williams present analyses of forecasting in a bino

mial world. Although this is a very simple framework, it nevertheless leads to important insights into such issues as to how forecasts can move markets.

Having put all this material together, I’m conscious that there is more that has not been included. I hope to rectify this at some later point.

Stephen Satchell

(13)

This page intentionally left blank

(14)

1 Market efficiency and forecasting

Wayne Ferson

1.1 Introduction

The interest in predicting stock prices or returns is probably as old as the markets themselves, and the literature on the subject is enormous. Fama (1970) reviews early work and provides some organizing principles. This chapter concentrates selectively on developments following Fama’s review. In that review, Fama describes increasingly fine information sets in a way that is useful in organizing the discussion. Weak-form pre

dictability uses the information in past stock prices. Semi-strong form predictability uses variables that are obviously publicly available, and strong form uses anything else. While there is a literature characterizing strong-form predictability (e.g. analyzing the profitabil

ity of corporate insider’s trades), this chapter concentrates on the first two categories of information.

For a while, predicting the future price or value (price plus dividends) of a stock was thought to be easy. Early studies, reviewed by Fama (1970), concluded that a martingale or random walk was a good model for stock prices, values or their logarithms. Thus, the best forecast of the future price was the current price. However, predicting price or value changes, and thus rates of return, is more challenging and controversial. The current financial economics literature reflects two often-competing views about predictability in stock returns. The first argues that any predictability represents exploitable inefficiencies in the way capital markets function. The second view argues that predictability is a natural outcome of an efficient capital market.

The exploitable inefficiencies view of return predictability argues that, in an efficient market, traders would bid up the prices of stocks with predictably high returns, thus lowering their return and removing any predictability at the new price (see, for exam

ple, Friedman, 1953; Samuelson, 1965). However, market frictions or human imper

fections are assumed to impede such price-correcting, or ‘arbitrage’ trading. Predictable patterns can thus emerge when there are important market imperfections, like trading costs, taxes, or information costs, or important human imperfections in processing or responding to information, as studied in behavioural finance. These predictable patterns are thought to be exploitable, in the sense that an investor who could avoid the friction or cognitive imperfection could profit from the predictability at the expense of other traders.

The ‘efficient markets’ view of predictability was described by Fama (1970). According to this view, returns may be predictable if required expected returns vary over time in asso

ciation with changing interest rates, risk or investors’ risk-aversion. If required expected returns vary over time there may be no abnormal trading profits, and thus no incen

tive to exploit the predictability. Predictability may therefore be expected in an efficient

(15)

2 Forecasting Expected Returns in the Financial Markets

capital market. The return is written as R=ER+u, where is the information at the beginning of the period and u is the unexpected return. Since Eu= 0, the unexpected return cannot be predicted ahead of time. Thus, predictability, in the ‘efficient markets’

view, rests on systematic variation through time in the expected return. Modelling and testing for this variation is the focus of the conditional asset pricing literature (see reviews by Ferson, 1995; Cochrane, 2005).

While this chapter focuses on return predictability, not all of the predictability associ

ated with stock prices involves predicting the levels of returns. A large literature models predictable second moments of returns (e.g. using ARCH and GARCH-type models (see Engle, 2004) or other stochastic volatility models). Predictability studies have also exam

ined the third moments (see, for example, Harvey and Siddique, 2001).

1.2 A modern view of market efficiency and predictability

As described by Fama (1970), any empirical analysis of stock return predictability or the market’s informational efficiency involves a ‘joint hypothesis’. There must be an hypothesis about the model for equilibrium expected returns, and also an hypothesis about the informational efficiency of the markets. These can be easily described using a modern representation for asset pricing models.

Most of the asset pricing models of financial economics can be described as versions of equation (1.1):

Em_t1 +rt_t−1=1 (1.1)

where _t−1 is the information set of economic agents at the beginning of the period and r_tis the rate of return of a financial asset. The scalar random variable m_tis the stochastic discount factor. Different models imply different stochastic discount factors, and the stochastic discount factor should price all of the assets in the model through equation (1.1).

The joint hypothesis of stock return predictability and market efficiency tests may now be described. The assumption of a model of market equilibrium amounts to a specification for m_t. For example, the Capital Asset Pricing Model of Sharpe (1964) implies that m_tis a linear function of the market portfolio return (see, for example, Dybvig and Ingersoll, 1982). Assume that the analyst uses the lagged variables, Z_t−1, to predict stock returns. The hypothesis of informational efficiency is simply the statement that Z_t−1is contained in _t−1. For example, weak-form efficiency says that past stock prices are in _t₋₁, while semi-strong form efficiency says that other publicly available variables are in _t−1.

The martingale model for stock values follows as a special case of this modern view. If we assume that m_tis a constant over time (as implied by risk neutral agents with fixed time discounting), then equation (1.1) implies that Er_t_t−1 is a constant over time.

Since r_t=Er_t_t−1+u_t, and u_tis unpredictable, it follows that the returns r_tcannot be predicted by any information in _t−1.

(16)

1.3 Weak-form predictability

Much of the literature on weak-form predictability can be characterized through an autoregression. Let R_tbe the continuously compounded rate of return over the shortest measurement interval ending at time t. Let rt t+H=_{j=1 H}R_t+j. Then,

rt t+H = a_H+_Hrt−H t+t t+H (1.2)

is the autoregression and H is the return horizon. Studies can be grouped according to the return horizon.¹

Many studies measure small but statistically significant serial dependence in daily or intra-daily stock return data. Serial dependence in daily returns can arise from end-of-day price quotes that fluctuate between bid and ask (Roll, 1984), or from non-synchronous trading of the stocks in an index (see, for example, Fisher, 1966; Scholes and Williams, 1977). These effects do not represent predictability that can be exploited with any feasible trading strategy. Spurious predictability due to such data problems should clearly not be attributed to time-variation in the expected discount rate for stocks. On the other hand, much of the literature on predictability allows that high frequency serial dependence may reflect changing conditional means. For example, Lo and MacKinlay (1988) and Conrad and Kaul (1988) model expected returns within the month as following an autoregressive process.

Conrad and Kaul (1988, 1989) studied serial dependence in weekly stock returns. They point out that if the expected returns, ER, follow an autoregressive process, the actual returns would be described by the sum of an autoregressive process and a white noise, and thus follow an ARMA process. The autoregressive and moving average coefficients would be expected to have the opposite signs: If current expected returns increase, it may signal that future expected returns are higher, but stock prices may fall in the short run because the future cash flows are discounted at a new, higher rate. The two effects, offset and returns, could have small autocorrelations. Estimating ARMA models, they found that the autoregressive coefficient for weekly returns on stock portfolios are positive, near 0.5, and can explain up to 25% of the variation in the returns on a portfolio of small-firm stocks.

Even with weekly returns, however, some of the measured predictability can reflect non-synchronous trading effects. Lo and MacKinlay (1990) and Muthaswamy (1988) use statistical models that attempt to separate out the various effects in measured portfolio returns. Boudoukh et al. (1994) use stock index futures contracts, which are not subject to non-synchronous trading, and find little evidence for predictability at a weekly frequency.

Much of the literature on weak-form predictability studies broad stock market indexes or portfolios of stocks, grouped according to the market capitalization (size) or other characteristics of the firms. However, another significant stream of the literature studies relative predictability. Stocks have relative predictability if the future returns of one group of stocks are predictably higher than the returns of another group. Thus, if a trader could

1An alternative to the autoregression is the Variance Ratio statistic, Varrt t+H/HVarR_t, proposed by Working (1949) and studied for stock returns by Lo and MacKinlay (1988, 1989) and others. Cochrane (1988) shows that the variance ratio is a function of the autocorrelation in returns. Kaul (1996) provides an analysis of various statistics that have been used to evaluate weak-form predictability, showing how they can be viewed as combinations of autocorrelations at different lags, with different weights assigned to the lags.

(17)

buy the stocks in the high-return group and sell short the stocks in the low-return group, the trader could profit even if both groups were to go up (or down). In a weak-form version of relative predictability, past stock prices or returns are used to form the groups.

If past winner (loser) stocks have predictably higher (lower) returns, we have continuation or ‘momentum’. If past winner stocks can be predicted to have lower future returns, we have ‘reversals’. Relative predictability can be evaluated by viewing equation (1.2) as a cross-sectional regression – an approach taken by Jegadeesh (1990). Lehman (1990) finds some evidence for reversals in the weekly returns of US stocks.

Monthly returns are commonly used in the literature that tests asset pricing models.

At this frequency, the evidence on weak-form predictability is relatively sparse. Jegadeesh (1990) finds some evidence for reversals at a monthly return frequency. Ferson et al.

(2005) make indirect inferences about the time-variation in monthly expected stock returns by comparing the unconditional sample variances of monthly returns with esti

mates of expected conditional variances. The key is a sum-of-squares decomposition:

VarR=EVarR+VarER, where E and Var are the conditional mean and variance, and Var and E , without the conditioning notation, are the uncondi

tional moments. The interesting term is VarER; that is, the amount of variation through time in conditionally expected stock returns. This quantity is inferred by subtract

ing estimates of the expected conditional variance, EVarR=ER−ER², from estimates of the unconditional variance. The expected conditional variance is estimated following Merton (1980), who showed that while the mean of a stock return is hard to estimate, it is almost irrelevant for estimating the conditional variance, when the time between observations is short. Using high-frequency returns to estimate the conditional variance for each month, then subtracting its average from the monthly unconditional variance, the difference – according to the decomposition – is the variance of the monthly conditional mean.

Ferson et al. (2005) find that while historical data prior to 1962 suggests economi

cally significant weak-form predictability in monthly stock market returns, there is little evidence of weak-form predictability for monthly returns in modern data. In particular, the evidence for the period after 1992 suggests that any weak-form predictability in the stock market as a whole has vanished. At the same time, a simulation study shows that the indirect tests have the power to detect even modest amounts of predictability.

Jegadeesh and Titman (1993) find that relatively high-past-return stocks tend to repeat their performance over 3- to 12-month horizons. They study US data for 1927–1989, but focus on the 1965–89 period. The magnitude of the effect is striking. The top 20%

winner stocks over the last 6 months can outperform the loser stocks by about 1% per month for the next 6 months. This momentum effect has spawned a huge subsequent literature which is largely supportive of the momentum effect, but which has not reached a consensus about its causes. The efficient markets view of predictability suggests that momentum trading strategies should be subject to greater risk exposures which justify their high returns. Most efforts at explaining the effect by risk adjustments have failed.²

The momentum effect has inspired a number of behavioural models, suggesting that momentum may occur because markets under-react to news in the pricing of stocks.

2There are some partial successes. For example, Ang et al. (2001) associate some of the momentum strategy profits with high exposure to ‘downside risk’ – that is, the covariance with market returns when the market return is negative.

(18)

For example, one argument (Daniel et al., 1998) is that traders have ‘biased self attribu

tion’, meaning that they think their private information is better than it really is. As a result they do not react fully to public news about the value of stocks, so the news takes time to get impounded in market prices, resulting in momentum. In another argument, traders suffer a ‘disposition effect’, implying that they tend to hold on to their losing stocks longer than they should, which can lead to momentum (Grinblatt and Han, 2003).

These arguments suggest that traders who can avoid these cognitive biases may profit from momentum trading strategies. However, Lesmond et al. (2004) and Korajczyk and Sadka (2004) measure the trading costs of momentum strategies and conclude that the apparent excess returns to the strategies are consumed by trading costs.

Perhaps the most controversial evidence of weak-form predictability involves long- horizon returns. Fama and French (1988) use autoregressions like equation (1.2) to study predictability in portfolio returns, measured over 1-month to multi-year horizons. They find U-shaped patterns in the autocorrelations as a function of the horizon, with negative serial dependence, or mean reversion, at 4- to 5-year horizons. Mean reversion can be consistent with either view of predictability. If expected returns are stationary (reverting to a constant unconditional mean) but time-varying, mean reversion can occur in an efficient market. Mean reversion would also be expected if stock values depart temporarily from the fundamental, or correct, prices, but are drawn back to that level. The evidence for weak-form predictability in long-horizon returns is subject to a number of criticisms on statistical grounds, as described below.

DeBondt and Thaler (1985) find that past high-return stocks perform poorly over the next 5 years, and vice versa, a form of relative predictability. They interpret reversals in long-horizon relative returns as evidence that the market over-reacts to news about stock values, and then eventually corrects the mistake. The reversal effect was shown to occur mainly in the month of January, by Zarowin (1990) and Grinblatt and Moskowitz (2003), which is interpreted as related to ‘tax loss selling’. In this story, investors sell loser stocks at the end of the year for tax reasons, thus depressing their prices, and buy them back in the new year, subsequently raising their prices. McLean (2006) finds that reversals are concentrated in stocks with high idiosyncratic risks, which is thought to present a deterrent to arbitrage traders who might otherwise correct temporary errors in the market prices.

Like momentum, behavioural models attempt to explain reversals as the result of cognitive biases. Models of Barberis et al. (1998), Daniel et al. (1998) and Hong and Stein (1999) argue that both short-run momentum and long-term reversals can reflect biases in under- and over-reacting to news about stock values. Research in this area continues, and it’s fair to say that the jury is still out on the issue of weak-form predictability in long-horizon returns.

1.4 Semi-strong form predictability

Studies of semi-strong form predictability can be described with the regression:

rt t+H = _H+_HZ_t+vt t+H (1.3)

where Z_tis a vector of variables that are publicly available by time t. Many predictor variables have been analyzed in published studies, and it is useful to group them into

(19)

categories. The first category of predictor variables comprises ‘valuation ratios’, which are measures of cash flows divided by the stock price. Keim and Stambaugh (1986) use a constant numerator in the ratio and ‘detrend’ the price. Rozeff (1984), Campbell and Shiller (1988) and Fama and French (1989) use dividend/price ratios, Pontiff and Schall (1998) and Kothari and Shanken (1997) use the book value of equity divided by price. Boudoukh et al. (2004) and Lei (2006) add share repurchases and other non-cash payouts, respectively, to the dividend measure. Lettau and Ludvigson (2001) propose a macroeconomic variation on the valuation ratio: aggregate consumption divided by a measure of aggregate wealth. All of these studies find the regression coefficients _Hto be significant.

Malkiel (2004) reviews a valuation ratio approach that he calls the ‘Federal Reserve Model.’ Here, the market price/earnings ratio is empirically modelled as a function of producer prices, Treasury yields and other variables, and the difference between the model’s output and the ratios observed in the market are used to predict the market’s direction. (Malkiel finds that the model does not outperform a buy-and-hold strategy.)

Rozeff (1984) and Berk (1995) argue that valuation ratios should generally predict stock returns. Consider the simplest model of a stock price, P, as the discounted value of a fixed flow of expected future cash flows or dividends: P =c/R, where c is the expected cash flow and R is the expected rate of return. Then, R = c/P, and the dividend price ratio is the expected return of the stock. If predictability is attributed to the expected return, as in the efficient markets view, then a valuation ratio should be a good predictor variable.

Predictability of stock returns with valuation ratios is also related to the expected growth rates of future dividends or cash flows. Consider the Gordon (1962) constant- growth model for a stock price: P = c/R−g, where g is the future growth rate. Then, c/P = R−g. This suggests that if dividend/price ratios vary, either across stocks or over time, then expected returns should vary, and/or expected cash flow growth rates should vary, and the dividend/price ratio should be able to predict one or the other. Campbell and Shiller (1988) show that the intuition from this example holds to a good approximation in more general discounted cash flow models, where the growth rates and expected returns are not held fixed over time. They find that market dividend/price ratios do not significantly predict future cash flow growth rates. Cochrane (2006) uses this result to re

evaluate the empirical evidence for stock return predictability using dividend/price ratios.

He essentially argues that if you know that the dividend/price ratio does not forecast future cash flow growth, then it must forecast future stock returns.

Studies of semi-strong form predictability in stock index returns typically report regres

sions with small R-squares, as the fraction of the variance in returns that can be predicted with the lagged variables is small – say 10–15% or less for monthly to annual return horizons. The R-squares are larger for longer-horizon returns – up to 40% or more for 4- to 5-year horizons. This is interpreted as the result of expected returns that are more persistent than returns themselves, as would be expected if returns are expected returns plus noise. Thus, the variance of the sum of the expected returns accumulates with longer horizons faster than the variance of the sum of the returns, and the R-squares increase with the horizon (see, for example, Fama and French, 1989). However, small R-squares can mask economically important variation in the expected returns.

Stocks are long ‘duration’ assets, so a small change in the expected return can lead to a large change in the asset value. To illustrate, consider another example using the Gordon

(20)

model, where the dividend is c =kE E is the earnings and k is the dividend payout ratio.

The price/earnings ratio P/E =15, the payout ratio k=0 6, and the expected growth rate g = 3%. The expected return R = 7%. Suppose there is a shock to the expected return, ceteris paribus. A change of 1% in R leads to approximately a 20% change in the asset value. Of course, this overstates the effect, to the extent that a shock that changes the required return also changes the expected future cash flows.

As the example suggests, small changes in expected returns can produce economically significant changes in asset values. Consistent with this argument, studies such as Kandel and Stambaugh (1996), Campbell and Viceira (2002) and Fleming et al. (2001) show that optimal portfolio decisions can be affected to an economically significant degree by return predictability, even when the amount of predictability, as measured by R-squared, is small.

The second category of semi-strong form predictor variables for stock returns includes calendar and seasonal effects. The list of effects that have been related to stock returns and the list of studies are too long to cite here (see Haugen and Lakonishok (1988) and Schwert (2003) for reviews). Some examples include the season (winter versus summer), the month of the year (especially, high returns in January), the time of the month (first versus last half), holidays, the day of the week (low returns on Mondays), the time of the day, the amount of sunlight (as in seasonal affective disorder) and even the frequency of geomagnetic storms.

The third category of predictor variables in equation (1.3) is a catch-all, ‘other’ vari

ables. Prominent among these are bond yields and yield spreads. Fama and Schwert (1977) were among the first to observe that the level of short-term Treasury yields predicts returns in equation (1.3) with a negative coefficient. They interpreted the short-term yield as a measure of expected inflation. Ferson (1989) argues that the regressions imply that the systematic risk of stocks that determines the expected returns must vary over time with changes in interest rates. Keim and Stambaugh (1986) study the yield spreads of low-quality over high-quality bonds, and find predictive ability for stock returns, and Campbell (1987) studies a number of yield spreads in shorter-term Treasury securities.

Fama and French (1989) assemble a list of variables from studies in the 1980s and describe their relations with US business cycles.

Another interesting set of predictor variables in equation (1.3) includes measures of the conditional variance or volatility of stock returns. Merton (1980) shows that a sim

ple version of his Intertemporal Asset Pricing Model implies that expected returns on the aggregate stock market should be positively related to the conditional variance of the market returns; that is, there should be a positive risk-return tradeoff for the market as a whole. Sharpe’s (1964) Capital Asset Pricing Model also makes this prediction. Early studies that tried to predict market returns with predetermined market volatility measures found mixed results – weak or even negative _Hcoefficients (e.g. French et al., 1987;

Breen et al., 1993). Scruggs (1998) showed that if additional risk factors as suggested by Merton’s (1973) model were included, the partial coefficients became positive as predicted by the theory.

Of course, many other semi-strong form predictor variables have been proposed and more will doubtless be proposed in the future. Some recent variables include the fraction of equity issues in new issues of corporate securities (Baker and Wurgler, 2000), firms’

investment plans (Lamont, 2000), the average ‘idiosyncratic’ or firm-specific component of past return volatility (Goyal and Santa Clara, 2003), the level of corporate cash holdings

(21)

(Greenwood, 2004), the aggregate rate of dividend initiation (Baker and Wurgler, 2000), share issuance (Pontiff and Woodgate, 2006), and the political party currently in office (Santa Clara and Valkanov, 2003).

1.5 Methodological issues

Even though the regressions in equations (1.2) and (1.3) seem pretty straightforward, interpreting the predictability evidence for stock returns based on these regressions is not.

It can be argued that one of the greatest contributions of the literature on stock return predictability is the methodological lessons it has taught researchers in the field. Perhaps the most difficult issues involve selection bias and data mining. Additional issues that have been addressed in the literature include small sample biases in the coefficients, stan

dard error estimation, multiple comparisons, efficient estimation, regime shifts, spurious regression and the interactions among these effects.

Selection bias and data mining are serious concerns. Data mining refers to sifting through the data in search of predictive or associative patterns. There are two kinds of data mining. Sophisticated data mining accounts for the number of searches undertaken when evaluating the statistical significance of the finding (see, for example, White, 2000).

This is important, because if 100 independent variables are examined, we expect to find five that are ‘significant’ at the 5% level, even if there is no predictive relation.

Naive data mining does not account for the number of searches. The big problem, given the strong interest in predicting stock returns among academics and practitioners, and the many studies using the same data, is that it is difficult to account for the number of searches. There probably have been at least as many regressions run using the Center for Research in Security Prices (CRSP) database as there are numbers in the database. Compounding this problem are various selection biases. Perhaps most difficult is the fact that only ‘significant’ results are circulated and published in academic papers. No one knows how many insignificant regressions were run before those results were found.

A reasonable response to these concerns is to see if the predictive relations hold out-of-sample. This kind of evidence is mixed. Some studies find support for predictability in step-ahead or out-of-sample exercises (for example, Fama and French, 1989; Pesaran and Timmerman, 1995). Semi-strong form variables show some ability to predict returns outside of the US data where they were originally studied (for example, Harvey, 1991;

Ferson and Harvey, 1993, 1999; Solnik, 1993). Other studies conclude that predictabil

ity using many of the semi-strong form variables does not hold outside of the original samples (for example, Goyal and Welch, 2003, 2004; Simin, 2006). Even this evidence is difficult to interpret, because a variable could have real predictive power yet still fail to outperform a naive benchmark when predicting out of sample (Campbell and Thompson, 2005; Hjalmarsson, 2006).

A large literature has addressed statistical issues in predictive regressions. Boudoukh and Richardson (1994) provide an insightful review. From the perspective of testing the null hypothesis of no predictability, we are interested in whether _His zero. The coeffi

cient is proportional to the covariance, Covrt t+H Z_t. The literature has employed three basic approaches to estimating the covariance. The first is to run the regres

sion in equation (1.3) with overlapping data on rt t +H, observed each period t,

(22)

usually 1 month (see, for example, Fama and French, 1989). Given overlapping data, this approach uses all of the data, as opposed to a sampling scheme, which measures non-overlapping returns every H months, and should therefore be more efficient than the sampling scheme. However, the overlap induces autocorrelation in the error terms in the form of an H −1 order moving average process. To conduct inference about _H, it is necessary to estimate the coefficients and the standard errors without bias in the presence of the moving average error terms. This is complicated by the evidence that stock return data are conditionally heteroscedastic.

When H >1 and several horizons are examined together, the issue of multiple compar

isons arises. If 20 independent horizons are examined, it is expected that there will be one

‘significant’ t-statistic found at the 5% level when the null hypothesis of no predictability is true. The slope coefficients for the different horizons are correlated, which complicates the inference. Richardson (1993) shows this implies that the U-shaped patterns in the autocorrelations across return horizons, observed by Fama and French (1988), are likely to be observed by chance. Boudoukh et al. (2005) argue that high correlation of test statistics across the return horizons renders much of the evidence for semi-strong form long-horizon predictability suspect.

Even when H = 1 and there is no overlap, correlation in the error terms can lead to finite sample biases in estimates of the slope. Stambaugh (1999) studies one such bias that arises because the regressor is stochastic and its future values are correlated with the error term in the regression. For example, if Z_tis a dividend/price ratio, then shocks to the dividend price ratio at time t+1 are related to stock returns at time t+1 through the stock price. Stambaugh provides corrections for this bias, which he shows is related to the bias in a sample autocorrelation coefficient, as derived by Kendal (1954). Amihud and Horvitz (2004) explore solutions for this bias in a multiple regression setting.

A second approach to estimating Covrt t+H Z_t = Cov_{j=1 H}R_t+j Z is _t to recognize that, if the variables are covariance stationary, then Cov_{j=1 H}R_t+j Z_t = CovR_t _{j=1 H}Z_t−j. This suggests using the horizon H = 1 on the left-hand side of equation (1.3), and replacing Z_ton the right-hand side with _{j=1 H}Z_t−j. This is the approach taken by Hodrick (1992) and Cochrane (1988). In this approach, the error terms in the regression are not overlapping and there is no induced moving average structure to account for. There may be efficiency gains compared with the first approach, but these depend on the stochastic process that drives the variables.

A third approach to estimating the predictive regression coefficient is to model the single period data R_t Z_t−1 using a vector autoregression, and then infer the value of the long- horizon coefficient _Hfrom the autoregression parameters. This is the approach taken by Kandel and Stambaugh (1990), Campbell (1993), and others. This can be efficient if the vector autoregression is correctly specified, but is subject to error if the autoregression is not correctly specified.

A potential issue with all of the approaches to predictive regressions is spurious regres

sion bias. Spurious regression is studied by Yule (1926) and Granger and Newbold (1974), who warn that empirical relations may be found between the levels of trending time series that are actually independent. For example, given two independent random walks, it is likely that a regression of one on the other will produce a ‘significant’ slope coefficient, evaluated by the usual t-statistics. In equation (1.3) the dependent variables are stock returns, which are not highly persistent. However, recall that the returns may be consid

ered the sum of the expected return, plus unpredictable noise. If the expected returns are

(23)

persistent, even if stationary time series, there is still a risk of spurious regression in finite samples. Because the unpredictable noise represents a substantial portion of the variance of stock returns, spurious regression effects, will differ for stock returns, from those in the classical setting of Granger and Newbold. This version of the spurious regression problem has received some attention in recent econometric studies, but more attention to the problem is probably due.

The interaction among the various statistical issues with predictive regressions has received relatively little research to date, and I would expect this to be an active field in the future. Ferson et al. (2003) study the interaction between data mining and spurious regression effects. They find that data mining for predictor variables interacts with spuri

ous regression bias in equation (1.3). The two effects reinforce each other, because more highly persistent series are more likely to be found significant in the search for predictor variables. Simulations suggest that many of the regressions in the literature, based on individual predictor variables, may be spurious. Powell et al. (2006) extend the analysis of the interaction between data mining and spurious regression to conclude that recent stud

ies of presidential regimes (Santa-Clara and Valkanov, 2003), the ‘Halloween Indicator’

(Bouman and Jacobsen, 2002) and business cycle effects in momentum (Chordia and Shivakumar, 2002) appear insignificant in view of their combined effects.

1.6 Perspective

Does the current body of evidence lead to the conclusion that there actually is predictabil

ity in stock returns? I think there are good reasons to be sceptical of predictability and good reasons to believe in predictability.

Why should we be sceptical? First, the logic of the old random walk = efficient markets literature is compelling to many. In that view, traders would bid up the prices of stocks with predictably high returns, thus lowering their returns and removing any predictability at the new price. Furthermore, data mining and selection bias conspire to make us see predictable patterns where none may exist. There are many choices for the return horizon, H. It need not be the same on both sides of the regression in equation (1.2).

Jegadeesh (1990) provides a statistical analysis of the choice of horizons in this context.

However, it is the number of choices that leads to scepticism.

For another example, studies of relative predictability, such as momentum, have sliced and diced common stocks into portfolios based on many characteristics of the data, at which point the effect often retreats into subsets of stocks, subperiods of time, phases of the business cycle or other parts of the data. Studies find momentum to be concentrated according to industries, the size of the firm (more momentum in large stocks), the price of the share (more when the price is above $5 per share), etc. The effect does not appear prior to 1940, appears stronger after 1968, and appears stronger during economic expansions than contractions.

The evidence for long-term return reversals has a similar problem. Reversals have been found to be concentrated in small stocks, low-priced stocks, the month of January, high idiosyncratic risk stocks, and to be more pronounced in earlier samples than in more recent data. For each approach to slicing and dicing the data there is a clever story. That is not bad. By digging into subsamples it should be possible in principle, to isolate what

(24)

is driving an effect. But many of the patterns, especially those documented in the weak- form predictability literature, appear to be sample-specific. Finding results that vary with the sample period stimulates research featuring structural breaks and regime shifts. This reader is left more with concerns about naive data mining, improper multiple comparisons and statistical issues than with an understanding of why and where these weak-form effects occur systematically.

Predictive regressions are subject to a host of statistical issues. There are finite sample biases and problems associated with structural breaks and regime shifts. It is hard to get reliable standard errors for the regressions. There are potential spurious regression problems. And, as we are now beginning to understand, these effects can interact with each other. If semi-strong form predictability is spurious, as a result of statistical bias and naive data mining, we would expect predictor variables to appear in the empirical literature, then fail to work with fresh data. To some extent, the literature has evolved in this way.

There are also many good reasons to believe that stock returns are predictable. First of all, theory suggests that some amount of predictability is likely. If expected returns vary over time with some degree of persistence, predictability is expected. Most people find it easy to believe that expected stock returns and risks might be different coming out of a recession, for example, from going into one, and the predictability evidence tends to confirm such commonsense patterns. Studies find that predictability using lagged variables is largely explained by asset pricing models with multiple risk factors, if they allow the premiums associated with those risks to vary over time (see, for example, Ferson and Harvey, 1991; Ferson and Korajczyk, 1995; Avramov and Chordia, 2006).

The behavioural models of predictability are compelling to many, as we see ourselves making the same cognitive errors as made by the agents in those models.

The momentum effect has been found to hold ‘out of sample’, relative to the original study of Jegadeesh and Titman (1993). Jegadeesh and Titman (2001) find momentum in data for 1990–1998. Momentum is found in stock markets outside of the US by Rouwehorst (1998) and Chui et al. (2000), among others.

The evidence of semi-strong form predictability has also survived a number of out-of

sample tests, working in other countries and over different time periods. Many of the variables identified as predictors for stock returns also seem to have some predictive power for other types of securities, such as bonds and futures, and also for the growth rates in

‘fundamental’ macroeconomic data. The evidence for semi-strong form predictability has survived corrections for a host of statistical and data problems.

Some studies that find semi-strong form stock market predictability, measured directly using lagged variables, has weakened in recent samples. It may be that the predictability was never really there, or that it was ‘real’ when first publicized, but diminished as traders attempted to exploit it. Ferson et al. (2005) examine semi-strong form predictability by regressing individual stocks on firm-specific predictors, then measuring the average covariances of the fitted values. It can be shown that the variance of the expected return on a large portfolio is approximately the average covariance. They find no evidence that predictability, measured in this way, is weaker in recent subperiods. As the firm-specific predictors have not been examined extensively in the literature, they may be less subject to naive data mining biases. Campbell and Thompson (2005), using step-ahead tests, also find that semi-strong form predictability holds up in recent data.

(25)

1.7 Conclusion

The issue of predictability in stock returns has important and broad economic implica

tions. For example, it relates to the efficiency of capital markets in allocating resources to their highest valued uses. However, the interpretation of predictability, and the evi

dence for its very existence, remains controversial. This review of the literature finds the evidence for weak-form predictability (using the information in past stock prices) to be more fragile and less compelling than the evidence for semi-strong form predictability (using publicly available information more generally).

For the field of financial economics and asset pricing in particular, allowing for pre

dictability through time-variation in expected returns, risk measures and volatility has been one of the most significant developments of the past two decades. Such conditional asset pricing models have provided a rich setting for the study of the dynamic behaviour of asset markets. For example, Conditional Performance Evaluation is the application of these models to the problem of evaluating the performance of portfolio managers.

Models that allow for time-varying conditional moments produce different inferences about performance than do the traditional measures that do not allow for predictability (e.g. Ferson and Schadt, 1996), and they have influenced both academic views and profes

sional investment practice. Research on predictability has stimulated numerous advances in the statistical and econometric methods of financial economics. The interactions between statistical biases and data mining in stock return studies should be a fruitful area for future research. Research on predictability in asset markets is likely to continue, and remain both useful and controversial for some time.

Acknowledgements

The author acknowledges financial support from the Collins Chair in Finance at Boston College.

References

Amihud, Y. and Horvitz, C. (2004). Predictive regressions: a reduced-bias estimation method. Journal of Financial and Quantitative Analysis, 39:813–842.

Ang, A., Chen, J. and Xing, Y. (2001). Downside risk and the momentum effect. NBER Working Paper No. 8643.

Avramov, D. and Chordia, T. (2006). Asset pricing models and financial market anomalies. Review of Financial Studies, 19:1001–1040.

Baker, M. and Wurgler, J. (2000). The equity share in new issues and aggregate stock returns. Journal of Finance, 55:2219–2257.

Barberis, N., Shleifer, A. and Vishny, R. (1998). A model of investor sentiment. Journal of Financial Economics, 49:307–343.

Berk, J. (1995). A critique of size-related anomalies. Review of Financial Studies, 8:275–286.

Boudoukh, J. and Richardson, M. (1994). The statistics of long-horizon regressions revisited. Mathematical Finance, 4:103–120.

Boudoukh, J., Richardson, M. and Whitelaw, R. (1994). A tale of three schools: insights on the autocorrelations of short-horizon returns. Review of Financial Studies, 7:539–573.

Boudoukh, J., Richardson, M. and Whitelaw, R. (2004). The myth of long-horizon predictability. NBER Working Paper No. 11 841.

(26)

Bouman, S. and Jacobsen, B. (2002). The Halloween indicator: sell in May and go away: another puzzle.

American Economic Review, 92:1618–1635.

Breen, W., Glosten, L. and Jagannathan, R. (1993). On the relation between the volatility and the expected value of the nominal excess returns on stocks. Journal of Finance, XLVII:1779–1801.

Campbell, J. Y. (1993). Intertemporal asset pricing without consumption data. American Economic Review, 83:487–512.

Campbell, J. Y. and Shiller, R. (1988). The dividend price ratio and expectations of future dividends and discount factors. Review of Financial Studies, 1:195–228.

Campbell, J. Y and Thompson, S. (2005). Predicting the equity premium out of sample: can anything beat the historical average? NBER Working Paper No. 11 468.

Campbell, J. Y. and Viceira, L. (2002). Strategic Asset Allocation: Portfolio Choice for Long-term Investors.

New York, NY: Oxford University Press.

Chordia, T. and Shivakumar, L. (2002). Momentum, business cycle and time-varying expected returns. Journal of Finance, 57:985–1019.

Chui, A., Titman, S. and Wei, K. C. J. (2000). Momentum, ownership structure and financial crises: an analysis of Asian stock markets. Working Paper, University of Texas at Austin.

Cochrane, J. H. (2005). Asset Pricing. Princeton, NJ: Princeton University Press.

Cochrane, J. H. (2006). The dog that did not bark: a defense of return predictability. NBER Working Paper No. 12 026.

Cochrane, J. Y. (1988). How big is the random walk in GNP? Journal of Political Economy, 96: 893–920.

Conrad, J. and Kaul, G. (1988). Time-variation in expected returns. Journal of Business, 61:409–425.

Conrad, J. and Kaul, G. (1989). Mean reversion in short-horizon expected returns. Review of Financial Studies, 2:225–240.

Daniel, K., Hirshleifer, D. and Subrahmanyam, A. (1998). Investor psychology and security market under and over-reaction. Journal of Finance, 53:1839–1885.

DeBondt, W. and Thaler, R. (1985). Does the stock market overreact? Journal of Finance, 40:793–805.

Dybvig, P. H. and Ingersoll, J. (1982). Mean variance theory in complete markets. Journal of Business, 55:233–252.

Engle, R. (2004). Risk and volatility: econometric modelling and financial market practice. American Economic Review, 94:405–420.

Fama, E. F. (1970). Efficient capital markets: a review of theory and empirical work. Journal of Finance, 25:383–417.

Fama, E. and French, K. (1988). Permanent and temporary components of stock prices. Journal of Political Economy, 96:246–273.

Fama, E. and French, K. (1989). Business conditions and expected returns on stocks and bonds. Journal of Financial Economics, 25:23–49.

Fama, E. F. and Schwert, G. W. (1977). Asset returns and inflation. Journal of Financial Economics, 5:115–146.

Ferson, W. E. (1989). Changes in expected security returns, risk and the level of interest rates. Journal of Finance, 44:1191–1217.

Ferson, W. E. (1995). Theory and empirical testing of asset pricing models. In: R. Jarrow, V. Maksimovic and B. Ziemba (eds), Finance, Handbooks in Operations Research and Management Science. Amsterdam:

Elsevier, pp. 145–200.

Ferson, W. E. and Harvey, C. R. (1991). The variation of economic risk premiums. Journal of Political Economy, 99:385–415.

Ferson, W. E. and Harvey, C. R. (1993). The risk and predictability of international equity returns. Review of Financial Studies, 6:527–566.

Ferson, W. E. and Harvey, C. R. (1999). Conditioning variables and cross-section of stock returns. Journal of Finance, 54:1325–1360.

Ferson, W. and Korajczyk, R. (1995). Do arbitrage pricing models explain the predictability of stock returns?

Journal of Business, 68:309–349.

Ferson, W. E. and Schadt, R. W. (1996). Measuring fund strategy and performance in changing economic conditions. Journal of Finance, 51:425–462.

Ferson, W. E., Heuson, A. and Su, T. (2005). Weak and semi-strong form stock return predictability revisited.

Management Science, 51:1582–1592.

Ferson, W. E., Sarkissian, S. and Simin, T. (2003). Spurious regressions in financial economics? NBER Working Paper No. W9143.

Fisher, L. (1966). Some new stock market indexes. Journal of Business, 39:191–225.

(27)

Fleming, J., Kirby, C. and Ostdiek, B. (2001). The economic value of volatility timing. Journal of Finance, 56:329–352.

Foster, D., Smith, T. and Whaley, R. (1997). Assessing goodness-of-fit of asset pricing models: the distribution of the maximal R-squared. Journal of Finance, 52:591–607.

French, K., Schwert, G. W. and Stambaugh, R. (1987). Expected stock returns and volatility. Journal of Financial Economics, 19:3–29.

Friedman, M. (1953). The case for flexible exchange rates. In: Essays in Positive Economics. Chicago, IL:

University of Chicago Press, pp. 157–203.

Gordon, M. J. (1962). The Investment, Financing and Valuation of the Corporation. Homewood, IL: Irwin.

Goyal, A. and Santa Clara, P. (2003) Idiosyncratic risk matters. Journal of Finance, 43:975–1007.

Goyal, A. and Welch, I. (2003). Predicting the equity premium with dividend ratios. Management Science, 49:639–654.

<