is an approach which has limitations beyond the philosophical. Comorbid presentations are very common in the general population. For example, an epidemiological survey in the UK (Office of Population Censuses and Surveys 1995) found that mixed anxiety and depression was much more common than depression alone in the general population. In addition, some diagnoses appear to ‘cluster’, for example people with generalised anxiety disorder were more likely to have an additional diagnosis of obsessional compulsive disorder or depression, and there was a significant association between a diagnosis of obsessional compulsive disorder and a co-occurring diagnosis of depression, phobia and generalised anxiety disorder (a pattern likely to be recognised by most clinicians). The challenge for researchers would be to conduct a trial which examined outcomes in relation to patients with these commonly occurring clusters of comorbid presentations as main effects (rather than through post hoc analysis), while maintaining appropriate levels of internal validity. Such trials would have utility because they would cast some light on a common clinical problem arising from comorbidity. For example, treating a patient with mixed depression and anxiety requires some thought about titrating research information about treatment outcomes in depression and in anxiety, in the absence of robust scientific information about how this is best done.
For some conditions applying research findings is complicated by the simple fact that there is not enough research available to draw reliable conclusions. An inspection of the Cochrane database of RCTs shows, for example, that while there are many good quality trials looking at outcomes in depression or panic disorder, there are very few RCTs examining outcomes in anorexia nervosa. To some degree this may reflect a very practical problem. Individuals who are depressed are, unsurprisingly, relatively easy to identify and to recruit. However, certain presentations are relatively uncommon or are usually seen in specialist settings, and this may limit the opportunity for research. The relative paucity of research in some key areas limits our capacity to make reliable statements about expected outcomes, because reviews become based on a limited number of trials. This leads to individual studies achieving an inappropriate weighting in the evaluation of efficacy, with the result that conclusions about efficacy are potentially unstable, the addition of just a few more trials can shift the balance of our conclusions markedly.
Beyond diagnosis, other factors routinely encountered by clinicians also tend to be under-represented in research. There is for example little direct exploration of the impact of social factors on outcome, even though material deprivation and the consequences of social exclusion will inevitably impact on the management of many patients seen in public health settings. It is also worth noting that most clients in research trials are white; as a consequence questions about the applicability of psychological therapies across ethnic groups and cultures are very hard to answer.
The majority of trials ask questions about comparative outcome from ‘brand names’ of therapy. This focus has led to the almost universal use of manualisation (in one form or another) in most research trials;
therapists are restricted to a repertoire of specified techniques, and their ability to adhere to them and deliver therapy in line with requirements is monitored carefully. In contrast, much clinical practice is an admixture of theoretical and technical eclecticism, with the consequence that knowing that a technique has proved effective under research conditions does not guarantee that it will work in the hand of clinicians.
Outcomes from research trials are usually better than those obtained in clinical practice (Weiss and Weisz 1990), and it is sensible to wonder whether at least part of the explanation for this lies in the greater rigour and focus of therapeutic practice within research. However, this level of prescription is usually seen as inappropriate in clinical practice, where the emphasis on tailoring practice to the patient is seen as a virtue, and linked to notions of clinical freedom. How far this is a reasoned approach could be questioned, but it is the case that while evidence for the benefits of adapting technique to the individual is not compelling, there are also reasons for being cautious about the virtues of manualisation. For example, exclusion criteria for many research trials can select out the very complexities that force clinicians to become more adaptive in
142 EVIDENCE IN MENTAL HEALTH CARE
their approach, which implies that there may be limits to the appropriateness or feasibility of a ‘cookbook’
approach. Indeed there is some evidence that therapists judged as more competent are able to deviate appropriately from technical recommendations with more difficult patients (Rounsaville et al. 1988), and that these individuals produce greater improvement in their patients (O’Malley et al. 1988).
While questions of outcome are important to clinicians, in daily practice they will be more concerned with questions of mechanism, i.e. what are the processes that underpin outcomes? Unfortunately much of what we know about many process-outcome relationships rests on post hoc analysis of studies originally designed to ask questions about outcome from specific therapies. Not all studies permit sensible analysis, but some trials have been multiply re-examined for information about process factors. The National Institute of Mental Health multi-centre trial for depression (well described in Elkin 1994) is a good example of this.
While initially analysed in relation to outcomes from the therapies employed in the trial, numerous papers on process issues have now appeared; indeed some are cited in this chapter because of the quality of the information they contain. Because of its size, information from this trial is likely to influence thinking about therapy process. Some caution about extrapolation is required however, since most process variables have fairly small effect sizes, and without a broad research base there is a considerable risk of mistaking sample variation for real effects. A recent review (McKay and Barkham 1998, commissioned as part of the evidence base for a clinical guideline) was explicitly briefed to report on a variety of potentially significant process issues. Not only was there relatively little data on which to base comments, but much of the process information that was obtained rested on multiple examination of a relatively small pool of studies, reducing the power and hence the quality of the evidence.
The consequence of post hoc analysis is that we know relatively little about some important topics. For example, what is the impact of therapist factors such as their skilfulness? Does training make a difference to outcome? These are not easy questions to answer; there is evidence that more skilful application of a therapy is associated with better outcomes, though the degree to which training confers this skill is less clear (Roth and Fonagy 1996). Which patient factors are pertinent to outcome? In outline we know that motivation, readiness for change, and psychological mindedness are likely to improve outcome (Orlinsky et al. 1994), but more specific predictions are harder to make. What is clear is that a positive therapeutic alliance is a significant and necessary underpinning to any therapy, and numerous studies (Horvarth and Symonds 1991) and analyses of major outcome trials (Krupnick et al. 1996; Castonguay et al. 1996) have now demonstrated a reliable relationship between alliance level and outcome. This description does not of course account for mechanism, though (through very different methodologies) there is increasing interest in exploring the therapeutic manoeuvres that form and sustain the alliance (see, for example, Bennett 1999).
Even where there is direct concern to collect process information, the methodologies we currently employ may not be appropriately sensitive. Stiles and Shapiro (1994) pointed out that treating process ingredients as additive and static variables may be profoundly misleading, either because the behaviour of therapists (and indeed patients) is likely to be mutually responsive to that of the other, or because therapists may deliver certain important ingredients at high levels regardless of patient characteristics or outcomes. Thus therapists may be warm towards their clients in a rather non-specific manner, with the consequence that it might be quite hard to demonstrate an association between warmth and outcome using statistics which assume linear associations. In addition, some process relationships may be quite complex, requiring some sophistication both to discover and to interpret. An example of this would be Blatt et al.’s (1996) finding that patient perfectionism impacts on the capacity to form a therapeutic alliance, but only in those patients moderate in their level of perfectionism did this link to outcome.
The search for the ‘effective ingredients’ of therapy remains an important, though an elusive challenge. All therapies appear to have a number of common factors, unsurprising given that the roots of most approaches
lie in engaging patients in the task of self-examination, in the context of a mutually trusting and reasonably honest interaction. Beyond this many therapists make strong claims for the specific benefits of their techniques; indeed the development of new models and their therapeutic applications continues apace. In practice, however, there is surprisingly little evidence for the differential efficacy of one therapy over another (Wampold et al. 1997). Process studies of therapy also suggest that the overlap between techniques may be as significant as their differences. In an analysis of the National Institute of Mental Health study of psychological therapy for depression, referred to above, Ablon and Jones (1999) used a Q-sort technique to examine which types of interactions were most characteristic of the two therapies employed: cognitive behaviour therapy and interpersonal psychotherapy. While the therapies have very different rationales, in practice there was a large degree of overlap in terms of what therapists actually did, and the ways in which patients responded. Of course, some differences were also detected between therapies, congruent with their different theoretical emphases. While it is possible to argue that the effective ingredients of each therapy lie with these differences in technique, it seems more reasonable to suggest that most effective therapies will demonstrate some overlap, and that there should be some caution in making too strong a claim about the uniqueness of one approach over another.
There have been studies which ‘dismantle’ the ingredients of therapy, removing or modifying elements which are thought to be mutative and exploring the resulting outcomes. In some areas this can be an important approach, especially where strong claims are made for the specificity of a particular technique.
For example, advocates of eye movement and desensitisation therapy initially maintained that its success in helping patients with post-traumatic stress disorder was related to the therapist tracking their hands across the patient’s field of vision while the patient speaks about (and hence re-experiences) their traumatising experience. However, it is evident that such hand movements may not be necessary (Renfey and Spates 1994), and while this finding may not be fatal to advocates of this model as a whole, it does relocate eye movement and desensitisation therapy closer to more conventional therapies in terms of the mechanism underpinning its efficacy. However, there are obvious limitations to the strategy of dismantling. For example, by design most pain-management programmes include a number of components, including input from medical staff, from psychologists and from physiotherapists. Overall it is clear that cognitive behaviour therapy is an effective technique for patients with chronic pain (Morley et al. 1999), but any desire to separate out the successful elements of the package may be detrimental if, as is likely, each component may bring about change in more than one outcome, and components interact to produce the overall effect.
Much of the above discussion suggests that information from outcome and process-outcome research translates uneasily to standard practice. An obvious strategy for improving matters would be to conduct more research on the effectiveness of therapy as conducted in routine settings. Unfortunately interpretation of data collected in the field is often limited by considerations of internal validity. Ideally information about effectiveness would be derived from large field trials across a number of settings, in order to overcome the usual statistical problems associated with small-scale open trials or case series. In practice this requires the use of a common metric across clinical settings, and by implication the co-operation of clinicians of disparate orientations and opinions. For now, it remains true that studies of clinical effectiveness within the published literature are fairly rare (Shadish et al. 1997).
Notwithstanding the above enumeration of difficulties, there are some conclusions to be drawn from the literature. Certainly there is good evidence for the efficacy of psychological therapies. The evidence relates to outcomes in relation to major mental health conditions, and this suggests that psychological therapies should be considered as a standard part of the repertoire of treatments. It remains true that some therapies are better researched than others, and this places some limits on comments about the comparative efficacy of the therapies. None the less, where good quality research exists, evidence for differential efficacy of
144 EVIDENCE IN MENTAL HEALTH CARE
therapies is not strong (for example, in depression there is good evidence for the efficacy of cognitive behaviour therapy, interpersonal psychotherapy and of short-term psychodynamic methods). However, there is some evidence for differential efficacy in relation to some disorders, and where this exists it should be noted (for example, the strength of the evidence suggests that obsessional compulsive disorder is best treated using behavioural and cognitive-behavioural methods).
As a generalisation, outcomes from therapy are likely to be moderate, though clinically significant, across a range of conditions. As is to be expected, treatment targets will reflect the nature of the condition being treated; in some conditions (such as some of the anxiety disorders) a complete remission can be expected, while in others improvements in functioning may be more realistic (as is the case for many individuals with schizophrenia). The likelihood of remission-relapse cycles is also an important consideration in thinking about outcomes. For example, individuals with a history of depression are likely to relapse over time if not given further treatment, and this suggests that follow-up maintenance treatments (whose efficacy has been demonstrated, for example Frank et al. 1991) might well be considered as an alternative either to brief interventions or to long-term treatment. ‘Brand names’ of therapy are only partially predictive of outcome, and it is worth recognising that the therapeutic alliance is important to successful treatment. Clearly therapists are not all the same: what works in the hands of one therapist might not work in the hands of another. Equally, patients are not all the same: what works for one patient might not work for another. All of this leads to a cautionary statement—‘evidence-based practice’ is more than the direct application of psychotherapy research. In the US the American Psychological Association has attempted to identify
‘empirically supported therapies’ (Chambless and Hollon 1998). This model is superficially attractive because it appears to resolve some of the ambiguity surrounding our current state of knowledge. However, for all the reasons given above, this certainty is more apparent than real, and the empirically supported therapy approach has created an acrimonious and, at points, divisive debate between researchers, clinicians and purchasers of health care. In the UK a rather different approach has been taken, with a recognition that clinical guidelines are needed to help to weigh evidence, and in some cases to weigh the absence of evidence. The role of professional and clinical judgement in titrating research findings against experience is critical here; without some attempt to test guidelines against conditions found in the field they will have little credence or respect among the majority of clinicians. Guidelines are recommendations, not prescriptions, and can and should be varied by clinicians, not at whim but on the basis of clinical judgement. Of course the possibility remains that even the most carefully written guideline prove ineffective for the most basic of reasons, because no one reads them or uses them. Clearly the dissemination and implementation of guidelines is a critical process, and there needs to be an audit of impact incorporated into the process of distribution. In this way the evidence base can be applied in a manner which takes into account the strengths and the limitations of psychotherapy research, and meets the needs of clinicians for easy interpretation of a very large literature. Ultimately, of course, it should also meet the needs of patients who need the treatment that is, as far as can be ascertained, the most likely to benefit them. And this, of course, is the most compelling reason why we should be concerned with applying the evidence.