In summary, validity can have different meanings depending on the context in which it is used. It is applied to the measurement of concepts, called construct validity, as in the case of ‘validating psy- chiatric diagnoses’, and to the relationship between operational measures, called theinternal and exter- nal validity of a presumed causal relationship. As applied to construct validity, it is an unending pro- cess in which one attempts to measure a concept of interest as accurately as possible. Validity involves a theoretical understanding of the concept as well as an empirical assessment of the criteria chosen to operationalise the concept. This chapter discusses
three basic ways in which validity is assessed: content validity, criteria validity and construct validity. Con- tent and criterion validity can be thought of as part of the process of assessing construct validity. One study cannot wholly validate a measure of a concept. It requires a pattern of consistent findings across studies involving different samples and different settings. An ideal example of this is how the field has approached identifying psychiatric diagnoses from the inception of the Diagnostic and Statistical Manual of Men- tal Disorders to current notions of understanding psychiatric diagnoses by incorporating aetiological information in future versions of how we opera- tionalise our diagnostic classifications.
The other way in which validity has been discussed in this chapter refers to the ‘internal and external validity’ of empirical relationships between opera- tional measures of the concepts of interest.Internal validityrefers to the extent that a statistically signifi- cant relationship is a causal one. There are a number of ways in which causal plausibility is assessed, for example the five criteria of judgement used in epidemiological studies [78]. In addition, causal plau- sibility is dependent on the type of study design employed. As discussed, quasi-experimental designs are open to a number of threats to internal validity, including regression artefacts, history and selection effects. Experimental study designs, in which one manipulates the time order of variables and controls for confounding factors, are less vulnerable to threats to internal validity. Finally,external validityrefers to the extent that one can generalise the study findings to and across persons, time periods and settings. To minimise threats to external validity, multiple stud- ies are needed in which the study populations, the historical time periods and the setting are varied.
Acknowledgements
This chapter was written while Dr. Goldstein was supported by NIMH RO1 MH56956 and NIMH-ORWH P50 MH082679. Drs. Goldstein and Cherkerzian are also supported by the Connors Center for Women’s Health and Gender Biology at Brigham & Women’s Hospital. The authors would like to thank Lisa Cushman-Daly for help in manuscript preparation.
References
[1] Carmines, E.G. and Zeller, R.A. (1979)Reliability and Validity Assessment. Series Quantitative Applications in the Social Sciences, Sage University Press, Beverly Hills, CA.
[2] Cronbach, L.J. and Meehl, P.E. (1955) Construct validity in psychological tests.Psychol. Bull.,52(4), 281–302.
[3] Anastasi, A. (1976)Psychological Testing, Macmillan, London.
[4] Nunnally, J.C. (1978) Psychometric Theory, McGraw-Hill, New York.
[5] Cronbach, L.J. (1971) Educational measurement, in Test Validation(ed. R.L. Thorndike), American Coun- cil on Education, Washington, D.C.
[6] Robins, L.N., Helzer, J.E., Croughan, J. and Ratcliff, K.S. (1981) The NIMH diagnostic interview schedule:
its history, characteristics, and validity.Arch. Gen.
Psychiatry,38, 381–389.
[7] Endicott, J. and Spitzer, R.L. (1978) A diagnostic interview: the schedule for affective disorders and schizophrenia.Arch. Gen. Psychiatry,35, 837–844.
[8] Streiner, D.L. (1993) A checklist for evaluating the usefulness of rating scales. Can. J. Psychiatry, 38, 140–148.
[9] Schwartz, C.C., Myers, J.K. and Astrachan, B.M.
(1975) Concordance of multiple assessments of out- come in schizophrenia: on defining the dependent variables in outcome studies.Arch. Gen. Psychiatry, 32, 1221–1227.
[10] Cloninger, C.R. (1987) A systematic method for clinical description and classification of personal- ity variants. A proposal.Arch. Gen. Psychiatry,44, 573–588.
[11] Takeuchi, M., Yoshino, A., Kato, M., Ono, Y. and Kitamura, T. (1993) Reliability and validity of the Japanese version of the tridimensional personality questionnaire among university students. Compr. Psychiatry, 34, 273–279.
[12] Kim, J.O. and Mueller, C.W. (1978)Factor Analysis:
Statistical Methods and Practical Issues, Sage Univer- sity Paper Series on Quantitative Applications in the Social Sciences, series no. 07-014, Sage Publications, Inc., Beverly Hills, CA.
[13] Addington, D., Addington, J. and Maticka-Tyndale, E. (1993) Rating depression in schizophrenia: a com- parison of a self-report and an observer scale.J. Nerv.
Ment. Dis.,181, 561–565.
[14] Woolson, R.F. (1987) Statistical Methods for the Analysis of Biomedical Data, John Wiley & Sons, Inc., New York.
[15] Simpson, J.C. (1982) Amino acid levels in schizophre- nia and celiac disease: another look.Biol. Psychiatry, 17, 1353–1357.
[16] Somervell, P.D., Beals, J., Kinzie, J.D., Boehnlein, J., Leung, P. and Manson, S.M. (1993) Criterion validity of the center for epidemiologic studies depression scale in a population sample from an American Indian village.Psychiatry Res.,47, 255–266.
[17] Radloff, L.S. (1977) The CES-D scale: a self-report depression scale for research in the general population.
Appl. Psychol. Meas.,1, 385–401.
[18] American Psychiatric Association (1987)DSM-III-R:
Diagnostic and Statistical Manual of Mental Disor- ders, 3rd edition revised. American Psychiatric Press, Washington, DC.
[19] Murphy, J.M., Berwick, D.M., Weinstein, M.C., Borus, J.F., Budman, S.H. and Klerman, G.L. (1987) Performance of screening and diagnostic tests: appli- cation of receiver operating characteristics analysis.
Arch. Gen. Psychiatry,44, 550–555.
[20] Baldessarini, R.J., Finkelstein, S. and Arana, G.W.
(1983) The predictive power of diagnostic tests and the effect of prevalence of illness.Arch. Gen. Psychiatry, 40, 569–573.
[21] Glaros, A.G. and Kline, R.B. (1988) Understanding the accuracy of tests with cutting scores: the sensi- tivity, specificity, and predictive value model.J. Clin.
Psychol.,44(6), 1013–1023.
[22] Robins, E. and Guze, S.B. (1970) Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia.Am. J. Psychiatry,126, 983–987.
[23] American Psychiatric Association (2000)Diagnostic and Statistical Manual of Mental Disorders, 4th edi- tion, Text Revision. American Psychiatric Association, Inc., Washington, DC.
[24] Hyman, S.E. (2003) Foreword, inAdvancing DSM:
Dilemmas in Psychiatric Diagnosis(eds K.A. Phillips, M.B. First and H.A. Pincus), American Psychiatric Association, Washington, DC, pp. xi–xxi.
[25] Kendler, K.S. (2006) Reflections on the relationship between psychiatric genetics and psychiatric nosology.
Am. J. Psychiatry,163(7), 1138–1146.
[26] Szatmari, P., Maziade, M., Zwaigenbaum, L. et al.
(2007) Informative phenotypes for genetic studies of psychiatric disorders.Am. J. Med. Genet. B Neuropsy- chiatr. Genet.,144B, 581–588.
[27] Charney, D.S. (2002) Foundation for the NIMH strategic plan for mood disorders research.Biol. Psy- chol.,52, 455–456.
[28] Lecrubier, Y. (2008) Refinement of diagnosis and dis- ease classification in psychiatry.Eur. Arch. Psychiatry Clin. Neurosci.,258(Suppl. 1), 6–11.
[29] Hyman, S.E. (2007) Can neuroscience be integrated into the DSM-5?Nat. Rev. Neurosci.,8, 725–732.
[30] Gottesman, I.I. and Gould, T.D. (2005) The endophe- notype concept in psychiatry, inResearch Advances in Genetics and Genomics: Implications for Psychiatry, (ed. N.C. Andreasen), American Psychiatric Publish- ing, Inc., Washington, DC, pp. 63–84.
[31] Allardyce, J., Suppes, T. and van Os, J. (2007) Dimen- sions and the psychosis phenotype. Int. J. Methods Psychiatr. Res.,16(Suppl. 1), S34–S40.
[32] Guth, C., Jones, P. and Murray, R.M. (1993) Familial psychiatric illness and obstetric complications in early- onset affective disorder. A case–control study.Br. J.
Psychiatry,163, 492–498.
[33] Cannon, M., Jones, P.B. and Murray, R.M. (2002) Obstetric complications and schizophrenia: historical and meta-analytic review.Am. J. Psychiatry,159(7), 1080–1092.
[34] Buka, S.L., Lipsitt, L.P. and Murray, R. (1993) Preg- nancy/delivery complications and psychiatric diagno- sis: a prospective study.Arch. Gen. Psychiatry,50(2), 151–156.
[35] Dalman, C., Allebeck, P., Cullberg, J., Grunewald, C.
and K ¨oster, M. (1999) Obstetric complications and the risk of schizophrenia: a longitudinal study of a national birth cohort.Arch. Gen. Psychiatry,56(3), 234–240.
[36] Buka, S.L., Cannon, T.D., Torrey, E.F., Yolken and R.H. Collaborative Research Group. (2008) Maternal exposure to herpes simplex virus and risk of psy- chosis among adult offspring.Biol. Psychol.,63(8), 809–815.
[37] Machon, R.A., Mednick, S.A. and Huttunen, M.O.
(1997) Adult major affective disorder after prena- tal exposure to an influenza epidemic. Arch. Gen.
Psychiatry,54(4), 322–328.
[38] Brown, A.S., Begg, M.D., Gravenstein, S.et al. (2004) Serologic evidence of prenatal influenze in the etiol- ogy of schizophrenia.Arch. Gen. Psychiatry,61(8), 774–780.
[39] Buka, S.L., Tsuang, M.T., Torrey, E.F., Klebanoff, M.A., Bernstein, D. and Yolken, R.H. (2001) Mater- nal infections and subsequent psychosis among off- spring: a forty year prospective study. Arch. Gen.
Psychiatry,58, 1032–1037.
[40] Brown, A.S., van Os, J., Driessens, E., Hoek, H.W.
and Susser, E.S. (2000) Further evidence of relation between prenatal famine and major affective disorder.
Am. J. Psychiatry,157(2), 190–195.
[41] Susser, E.S., St. Clair, D. and He, L.. (2008) Latent effects of prenatal malnutrition on adult health: the example of schizophrenia. Ann. N. Y. Acad. Sci., 1136, 185–192.
[42] Blue, I. and Harpham, T. (1996) Urbanization and mental health in developing countries. Curr. Issues Public Health,2(4), 181–185.
[43] Pedersen, C.B. and Mortensen, P.B. (2001) Evidence of a dose-response relationship between urbanicity during upbringing and schizophrenia risk.Arch. Gen.
Psychiatry,58(11), 1039–1046.
[44] Selten, J.P., van Os, J. and Nolen, W.A. (2003) First admission for mood disorders in immigrants to the Netherlands.Soc. Psychiatry Psychiatr. Epidemiol., 38, 547–550.
[45] Cantor-Graae, E. (2007) The contribution of social factors to the development of schizophrenia: a review of recent findings.Can. J. Psychiatry,53(5), 277–286.
[46] Rwegellera, G.G. (1977) Psychiatric morbidity among West Africans and West Indians living in London.
Psychol. Med.,7(2), 317–329.
[47] Corcoran, C., Perrin, M., Harlap, S. et al. (2009) Incidence of schizophrenia among second-generation immigrants in the Jerusalen Perinatal Cohort.
Schizophr. Bull.,35(3), 596–602.
[48] Caspi, A., Moffitt, T.E., Cannon, M. et al. (2005) Moderation of the effect of adolescent-onset cannabis use on adult psychosis by a functional polymorphism in the catechol-O-methyltransferase gene: longitudinal evidence of a gene×environment interaction.Biol.
Psychiatry,57(10), 1117–1127.
[49] Caspi, A., Sugden, K., Moffitt, T.E. et al. (2003) Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science, 301 (5631), 386–389.
[50] Gottesman, I.I. and Shields, J. (1972)Schizophrenia and Genetics: A Twin Study Vantage Point, Academic Press, New York.
[51] Bearden, C.E. and Freimer, N.B. (2006) Endopheno- types for psychiatric disorders: Ready for primetime?
Trends Genet.,22(6), 306–313.
[52] Bilder, R.M. (2008) Phenomics: Building scaffolds for biological hypotheses in the the post-genomic era.
Biol. Psychiatry,63, 439–440.
[53] Skuse, D.H. (2001) Endophenotypes and child psy- chiatry.Br. J. Psychiatry,178, 395–396.
[54] Doyle, A.E., Faraone, S.V., Deidman, L.J.et al. (2005) Are endophenotypes based on measures of execu- tive functions useful for molecular genetic studies of ADHD?J. Child Psychol. Psychiatry Allied Discip., 46(7), 774–803.
[55] Waldman, I.D. (2005) Statistical approaches to complex phenotypes: evaluating neuropsychologi- cal endophenotypes for attention-deficit/hyperactivity disorder.Biol. Psychiatry,57, 1347–1356.
[56] Frances, A.J., First, M.B., Widiger, T.A.et al. (1991) An A to Z guide to DSM-IV conundrums.J. Abnorm.
Psychol.,100(3), 407–412.
[57] Sabb, F.W., Bearden, C.E., Glahn, D.C.et al. (2008) A collaborative knowledge base for cognitive phe- nomics.Mol. Psychiatry,13(4), 350–360.
[58] Wittchen, H.U., Beesdo, K., Bittner, A.et al. (2003) Depressive episodes – evidence for a causal role of primary anxiety disorders?Eur. Psychiatry, 18 (8), 384–393.
[59] Tsuang, M.T., Stone, W.S., Tarbox, S.I. et al.
(2003) Insights from neuroscience for the concept of schizotaxia and the diagnosis of schizophrenia, in Advancing DSM: Dilemmas in Psychiatric Diagno- sis(eds K.A. Phillips, M.B. First and H.A. Pincus), American Psychiatric Association, Washington, DC, pp. 105–127.
[60] Maziade, M., Bissonnette, L., Rouillard, E. et al.
(1997) 6p24-22 region and major psychoses in the Eastern Quebec population. Le Groupe IREP.Am. J.
Med. Gen.,74(3), 1726–1733.
[61] Wildenauer, D.B., Hallmayer, J., Schwab, S.G.
et al. (1996) Searching for susceptibility genes in schizophrenia by genetic linkage analysis.Cold Spring Harb. Symp. Quant. Biol.,61, 845–850.
[62] Berrettini, W. (2003) Bipolar disorder and schizophre- nia: not so distant relatives?World Psychiatry,2(2), 68–72.
[63] Badner, J.A. and Gershaon, E.S. (2002) Meta- analysis of whole-genome linkage scans of bipolar disorder and schizophrenia. Mol. Psychiatry, 7(4), 405–411.
[64] Maziade, M., Chagnon, Y.C., Roy, M.A.et al. (2009) Chromosome 13q13-q14 locus overlaps mood and psychotic disorders: the relevance for redefining phe- notype.Eur. J. Hum. Genet.[Epub ahead of print (doi: 10.1038/ejhg.2008.268)].
[65] Christoforou, A., Le Hellard, S., Thomson, P.A.
et al. (2007) Association analysis of the chromosome 4p15-p16 candidate region for bipolar disorder and schizophrenia.Mol. Psychiatry,12(11), 1011–1025.
[66] Schwab, S.G., Hallmayer, J., Lerer, B.et al. (1998) Support for a chromosome 18p locus conferring sus- ceptibility to functional psychoses in families with schizophrenia, by association and linkage analysis.
Am. J. Hum. Genet.,63(4), 1139–1152.
[67] Bearden, C.E., Jasinska, A.J. and Freimer, N.B. (2009) Methodological issues in molecular genetic studies of mental disorders.Annu. Rev. Clin. Psychol., 5, 49–69.
[68] Meehl, P. (1962) Schizotaxia, schizotypy, and schizophrenia.Am. Psychol.,17, 827–838.
[69] Tsuang, M.T., Stone, W.S., Seidman, L.J.et al. (1999) Treatment of nonpsychotic relatives of patients with schizophrenia: Four case studies.Biol. Psychiatry,41, 1412–1418.
[70] Tsuang, M.T., Stone, W.S., Gamma, F.et al. (2003) Schizotaxia: current status and future directions.Curr.
Psychiatry Rep.,5, 128–134.
[71] Faraone, S.V., Seidman, L.J., Kremen, W.S. et al.
(1995) Neuropsychological functioning among the
nonpsychotic relatives of schizophrenic patients: a diagnostic efficiency analysis. J. Abnorm. Psychol., 104, 286–304.
[72] Faraone, S.V., Kremen, W.S., Lyons, M.J.et al. (1995) Diagnostic accuracy and linkage analysis: how use- ful are schizophrenia spectrum phenotypes?.Am. J.
Psychiatry,152, 1286–1290.
[73] Battaglia, M. and Torgersen, S. (1996) Schizotypal disorder: at the crossroads of genetics and nosology.
Acta Psychiatr. Scand.,94, 303–310.
[74] Tsuang, M.T., Stone, W.S., Gamma, F.et al. (2000) Towards the prevention of schizophrenia.Biol. Psy- chiatry,48, 349–356.
[75] Stone, W.S., Faraone, S.V., Seidman, L.Jet al. (2001) Concurrent validation of schizotaxia: a pilot study.
Biol. Psychiatry,50, 400–434.
[76] Cook, T.D. and Campbell, D.T. (1979) Quasi- Experimentation Design and Analysis Issues for Field Settings, Rand McNally College Publishing Company, Chicago.
[77] MacMahon, B. and Pugh, T.F. (1970)Epidemiology:
Principles and Methods, Little, Brown, and Company, Boston.
[78] Susser, M. (1973) Causal Thinking in the Health Sciences. Concepts and Strategies of Epidemiology, Oxford University Press, London.
[79] Campbell, D.T. and Elebacher, A. (1970) How regres- sion artifacts in quasiexperimental evaluations can mistakenly make compensatory education look harm- ful, inCompensatory Education: A National Debate, Disadvantaged Child, Vol.3(ed. J. Helmuth), Brun- ner/Mazel, New York, pp. 185–210.
8 Use of register data
for psychiatric epidemiology in the Nordic countries
Jouko Miettunen,
1Jaana Suvisaari,
2Jari Haukka
2,3and Matti Isohanni
11Department of Psychiatry, University of Oulu, Finland
2Department of Mental Health and Substance Abuse Services, National Institute for Health and Welfare, Helsinki, Finland
3Department of Public Health, University of Helsinki, Finland