An example of the complexity of outcome assessment in neurology is found in epilepsy. Clinical trials in epilepsy pres-ent a wide range of challenges. Four main types of outcome measures are used in clinical trials of epilepsy, for example, seizure frequency, seizure severity, adverse effects, and QOL.
Seizure frequency is the most commonly used. Although superficially simple, measuring this outcome is not straightfor-ward. First, few studies have explored the validity of seizure diaries, the standard method of capturing this outcome.
Second, there is no consensus about the optimum specific
seizure measure. Most pharmaceutical trials focus on some measure of improvement in seizure frequency (e.g.50%), for which no notion of a minimum clinically important change exists, as opposed to seizure freedom, which is always clinically meaningful. Third, comparing mean seizure counts is an alternative; however, the clinical interpretation of group means and medians is problematic (see above).
Finally, the overall objective of any intervention in epilepsy is to improve the patients’ QOL. Numerous scales have been designed to assess QOL in epilepsy, their reliability, validity, and responsiveness have been established [28], and thresh-olds that constitute minimum clinically important change in individual instruments have been reported [19]. Yet, because of regulatory and licensing requirements, few stud-ies emphasize or adequately assess the impact of interven-tions on QOL.
Interpreting outcomes in epilepsy surgery
Clinicians at the University of Western Ontario wished to know if they should refer a patient with temporal-lobe epilepsy (TLE) for surgery. After searching the literature for the best evidence, they came across a study that compared surgery and medical treatment in patients with TLE [29].
The study was a RCT that assessed the safety and efficacy of temporal lobe surgery as compared to medical treatment. The primary outcome was freedom from seizures. Secondary out-comes were frequency and severity of seizures, QOL, disability, and death. Clinicians calculated the CER (medical therapy), EER (surgical therapy), ARD, NNTs, and their respective confi-dence intervals using a simple, 2 2 table, and produced a critically appraised topic (Appendix) (http://www.uwo.ca/
clinns/ebn).
References
1 Biller J, Bogousslavsky J. Clinical Trials in Neurologic Practice.
Butterworth-Heinemann, Woburn, MA, 2001.
2 Nunnally JCJ. Introduction to Psychological Measurement. McGraw-Hill, New York, 1970.
3 Temple RJ. A regulatory authority’s opinion about surrogate end-points. In: Nimmo WS & Tucker GT, eds., Clinical Measurement in Drug Evaluation, John Wiley & Sons, New York, 1995.
4 Bucher H, Guyatt G, Cook DJ, Holbrook A, McAlister F. Therapy and applying the results: surrogate outcomes. In: Guyatt G &
Rennie D, eds., Users’ Guides to the Medical Literature. American Medical Association, Chicago, 2002: pp. 393–413.
5 Valori RM. Outcome measures. In: McGovern DPB, Summerskill WSM, Valori RM, Levi M & McManus RJ, eds., Evidence-Based Medicine in General Practice. BIOS, Oxford, 2001.
6 Jaeschke R, Singer J, Guyatt GH. Measurement of health status.
Ascertaining the minimal clinically important difference. Contr.
Clin. Trials 1989; 10(4): 407–15.
7 Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology, 2nd edn. Little Brown, London, 1991.
20 Part 1: Introduction
8 Sackett DL. Bias in analytic research. J. Chronic Dis. 1979;
32(1–2): 51–63.
9 Parsons HM. What happened at Hawthorne? Science 1974; 183:
922–32.
10 Burneo JG, Montori VM, Faught E. Magnitude of the placebo effect in randomized trials of antiepileptic agents. Epilepsy Behav.
2002; 3(6): 532–4.
11 Macklin R. The ethical problems with sham surgery in clinical research. N. Engl. J. Med. 1999; 34(13): 992–6.
12 Eiser C, Morse R. Can parents rate their child’s health-related quality of life? Results of a systematic review. Qual. Life Res.
2001; 10(4): 347–57.
13 Gordon K, MacSween J, Dooley J, Camfield C, Camfield P, Smith B. Families are content to discontinue antiepileptic drugs at dif-ferent risks than their physicians. Epilepsia 1996; 37(6): 557–62.
14 Rothwell PM, McDowell Z, Wong CK, Dorman PJ. Doctors and patients don’t agree: cross sectional study of patients’ and doc-tors’ perceptions and assessments of disability in multiple scler-osis. BMJ 1997; 314(7094): 1580–3.
15 Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N. Engl. J.
Med. 1988; 318(26): 1728–33.
16 Bussiere M, Wiebe S. Measuring the benefit of therapies for neurological disorders. Can. J. Neurol. Sci. 2005; 32: 419–24.
17 Sackett DL, Strauss SE, Richardson WS, Rosenberg W, Haynes B.
Evidence-Based Medicine. How to Practice and Teach EBM, 2nd edn.
Churchill Livingstone, London, 2000.
18 Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS.
Interpreting treatment effects in randomised trials. BMJ 1998;
316(7132): 690–3.
19 Wiebe S, Matijevic S, Eliasziw M, Derry PA. Clinically important change in quality of life in epilepsy. J. Neurol. Neurosurg.
Psychiatr. 2002; 73(2): 116–20.
20 Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med. Care 2003; 41(5): 582–92.
21 Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life. Ann. Intern. Med. 1993; 118(8): 622–9.
22 Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc. Sci. Med. 2005; 60(4): 833–43.
23 Gilliam FG, Fessler AJ, Baker G, Vahle V, Carter J, Attarian H.
Systematic screening allows reduction of adverse antiepileptic drug effects: a randomized trial. Neurology 2004; 62(1): 23–7.
24 Wagner AK, Ehrenberg BL, Tran TA, Bungay KM, Cynn DJ, Rogers WH. Patient-based health status measurement in clinical practice: a study of its impact on epilepsy patients’ care. Qual.
Life Res. 1997; 6(4): 329–41.
25 Wiebe S, Guyatt G, Weaver B, Matijevic S, Sidwell C.
Comparative responsiveness of generic and specific quality-of-life instruments. J. Clin. Epidemiol. 2003; 56(1): 52–60.
26 Walker MC, Sander JW, Shorvon SD. Epilepsy: ethics, outcome variables and clinical scales. In: Guiloff RJ, ed., Clinical Trials in Neurology, Springer-Verlag, London, 2001.
27 Thrombolysis for acute ischaemic stroke (Cochrane Review).
The Cochrane Library, 2003. (Accessed on July 23rd, 2006).
28 Cramer JA. Principles of health-related quality of life: assess-ment in clinical trials. Epilepsia 2002; 43(9): 1084–95.
29 Wiebe S, Blume WT, Girvin JP, Eliasziw M. A randomized, con-trolled trial of surgery for temporal-lobe epilepsy. N. Engl. J. Med.
2001; 345(5): 311–8.
Appendix
In patients with TLE, surgery was superior to medical ther-apy at 1 year. The NNT was 3 to render patients free from all seizures, and to render them free of seizures impairing awareness.
Clinical Problem: The patient is a 36-year-old male with long-standing history of medically refractory TLE and mesial tem-poral sclerosis on MRI.
Clinical Question: What is the efficacy and safety of temporal lobe surgery in patients with TLE?
Search Strategy: PubMed: Search ‘TLE’ and ‘surgery’, All Fields, Limit to RCT, Human. Yielded eight citations. The article chosen was cited first.
Clinical bottom lines:
1 With temporal lobe surgery, the NNT for freedom from all seizures at 1 year was 3 (95% CI 2, 5), while it was 2 (95% CI 1.5, 3) for freedom from seizures impairing awareness.
2 The proportion of patients that remained free of seizures impairing awareness at 1 year was 58% in the surgical group and 8% in the medical group (P 0.001).
3 The proportion of patients that remained free of all seizures was 38% in the surgical group and 3% the medical group (P 0.001).
4 The QOL was better among the patients in the surgical group than among those in the medical group (P 0.001), and it improved over time in both groups (P 0.003).
5 Temporal lobe surgery was safe in patients with TLE (see data interpretation below).
The evidence:
This parallel group RCT assessed the safety and efficacy of temporal lobe surgery as compared to optimum medical treatment at 1 year in 80 patients with TLE. Optimal medical therapy and primary outcomes were assessed by blinded epileptologists. Analysis was by intention to treat. The pri-mary outcome was freedom from seizures that impair awareness. Secondary outcomes were frequency and sever-ity of seizures, QOL, disabilsever-ity, and death.
Chapter 3: Outcome and adverse effect measures in neurology 21
Data and interpretation:
1 Freedom of seizures impairing awareness at 1 year
Seizure free Seizures Total
Surgery 23 17 40
Medical therapy 3 37 40
Total 26 54
CER 0.08, EER 0.58, ARR 0.5, NNT 2 (95% CI 1.5, 3).
2 Freedom of all seizures at 1 year
Seizure free Seizures Total
Surgery 15 25 40
Medical therapy 1 39 40
Total 16 64
CER 0.025, EER 0.375, ARR 0.35, NNT 3 (95% CI 2, 5).
3 Risk and harm
ARI (%) NNH (1/ARI) 95% CI (ARI)
Surgery
Total risks (surgical
complications) 11 9
Thalamic infarct 2.5 40 0.06, 13
Wound infection 2.5 40 0.06, 13
Decreased verbal
memory 5 20
Asymptomatic superior
quadrantanopsia 61 1.5
Medical therapy
Death 2.5 40 0.06, 13
ARI: absolute risk increase; NNH: number needed to harm.
Comments about the evidence:
1 Although two blinded epileptologists assessed the patient’s seizure diaries, one wonders whether patients may have had seizures that impaired awareness and thus were not recorded. However, this is the only feasible manner of measuring seizure outcome, and is standard for all medical or surgical epilepsy trials.
2 No clear description was given specifically about how adverse events were collected.
3 The long-term outcome of surgery was not addressed by this RCT. However, long-term outcomes are unlikely to be derived from RCTs, which are typically of limited duration.
22 Part 1: Introduction
Introduction
Obtaining an accurate diagnosis is essential in clinical prac-tice. Almost all clinical actions are in one way or the other related to the diagnosis. When confronted with a patient with a medical problem, the first effort goes – either impli-citly or expliimpli-citly – into making a diagnosis. This should trans-late into an expected prognosis and guide further action, either immediate therapeutic intervention or further diagnos-tic testing. An accurate diagnosis is crucial in choosing the right therapy and, vice versa, it is often only possible to prop-erly evaluate the results of any treatment within the context of a certain diagnosis.
Despite the pivotal role of diagnosis in clinical practice, it has been the focus of much less research than has the develop-ment and evaluation of treatdevelop-ments. As in football, scoring the goal draws more attention than giving the assist. Consequently the methodological development of therapeutic research is more advanced. Randomized controlled trials (RCTs) are now widespread and standardized approaches to evaluating the quality of these trials have been developed. Moreover, given that sometimes the omissions are not in the execution of research but in the reporting of the findings, guidelines have been developed to improve the reporting of RCTs and thus facilitating the assessment of the validity of the results [1]. This so-called CONSORT statement is now accepted and applied by the major medical journals.
Progress is now being made in the field of diagnostic testing, such as the STARD initiative [2]. Like the CONSORT, STARD is intended to improve the quality of reporting of studies to the advantage of clinicians, reviewers and others. However, since this is a relatively recent development, most published studies on diagnostic tests that are included in present reviews may suffer from a lack of information on key elements of design, conduct and analysis [3].