T
he next section reviews how we positive psychologists evaluate whether or not interventions have worked from both a qualitative and quantitative perspective.Do mental wellbeing interventions work? Well yes, actually, and quite significantly!
Sin and Lyubomirsky’s (2009) meta-analysis demonstrated that in a sample of 4000+, both normal and depressed populations benefited from participation in PPIs. There were, however, several important factors for impact. Depression status influenced results, with those reporting higher levels of depression at baseline reporting the greatest impact. Also, as found in other studies, people who self-selected for the studies had higher levels of improved wellbeing. The age of participants was also influential, with younger ones experiencing more benefits. Whether the PPIs were delivered in individual or group therapy or self-administered had an impact, with individual therapy having greatest impact. Finally, PPIs that were longer than four weeks, but shorter than 12 weeks tended to have better results.
time out
Qualitatively evaluating PPIs
Evaluating PPIs is a very complex, time-consuming and expensive endeavour. There is also a very large difference between evaluating a large-scale versus small-scale study.
In a perfect world, all interventions would use proper assessment tools and neat research designs. They would recruit appropriate sample sizes and conduct proper statistical analyses with everyone on the research team having the relevant education.
In reality, however, we may not have access to hundreds of participants or to data- analysis software (such as SPSS). We may not have access to or knowledge of statistics or the money to buy in people to analyse statistics. However we still want to know if it worked, by how much, for whom, what worked best and what didn’t.
One major criticism of positive psychology, from a European perspective, is that it has fixated itself on the ‘scientific method’ and less on using
methodologies that enable researchers to access the entire human participant.
Positive psychological interventions 159
Accessing the person as a whole falls under the realm of qualitative methodology. Conducting this type of research is an important piece of the jigsaw that can give us information we would never obtain from quantitative inquiry alone. Researchers and students can use structured, semi-structured or unstructured interviews to collect data on the experiences of the PPIs. It is best to use qualitative enquiry for small sample sizes as it focuses on in-depth, exploratory analysis of interventions (Robson, 2004). Questions should be open, non-directive and focus on one concept at a time. Try not to use double negatives, use ‘what’ or ‘how’ instead. Examples of intervention evaluation questions could be:
What was your experience of the programme?
What was your favourite part of the programme? Least favourite?
What would you change about it?
Would you recommend this programme to others? (Why or why not?)
Furthermore, you can use a data-collection technique called the focus group.
Focus groups access people whose ‘voices’ are not traditionally heard and researchers therefore evaluate the findings through the eyes of the people that the research is about. A quote from Fine and Gordon sums up the benefits of a focus group situation:
If you really want to know either of us, do not put us in a laboratory, or hand us a survey, or even interview us separately alone in our homes.
Watch me with my women friends, my son, his father, my niece, or my mother and you will see what feels most authentic to me.
– (Fine and Gordon, 1989, as cited in Wilkinson, 1998)
If you decide that this is the most appropriate method for you, make sure that you allow two hours – you don’t want to be tied for time. You need to think about whether or not you will use one group once or several focus groups on several different occasions, which allows you to get a range of viewpoints. In terms of size, you need to be wary of too many participants versus too little. Focus groups tend to range from 2–12, although the norm tends to be 4–8. One of the biggest issues with this type of data collection is the breach in confidentiality, since people will be talking about their experiences, in front of others. Moderators need to ensure that all individuals are aware of what confidentiality means and that the participants agree not to break it.
To start the focus group, the moderator should explain the purpose and objectives of the session. They are allowed to prompt discussion by posing questions from an already created and edited list. Moderators must try to enable participation by all members and appear non-judgemental.
Analysing the data
There are several types of analysis you can use with your data set. One in particular is content analysis, which commences with a list of predetermined themes or key words (objectives). The researcher then searches for matching words and records frequency, with a higher number of instances equalling a higher importance. This technique can be used to work with qualitative or quantitative data.
Another type of analysis is thematic analysis (TA). An umbrella term until 2006, TA is the most common method of analysis used by novices. One of the reasons students and researchers like this approach is that it does not stem from an underlying philosophical theory and is therefore used a lot by researchers with no real affinity to qualitative methods; although if the researcher does identify with a theory, this needs to be stated clearly and followed.
Thematic analysis is the analysis of textual material, which looks for major themes, beyond surface-level description. It attempts to organize text into coherent sections. By following three major steps – transcription, theme identification and analytic effort – you can ensure that you undertake a proper qualitative approach (Braun and Clarke, 2006).
The transcription process ranges from superficial/simple transcription to more in-depth (literal or Jefferson). You must try to do the transcription yourself as this will get you closer to the data.
When beginning the analysis, make sure you read through the scripts and become familiar with the text. You can highlight and code based on very detailed line-by-line coding, to see broader overarching themes. Ultimately, the further you take the analysis, the better the analysis – it’s all about
refinement. The key in the write-up is to (1) be transparent, (2) clearly display results and (3) write the report demonstrating these themes and appropriate quotes.
There are several issues with TA. Accounts on ‘how to’ are scarce and actual methods tend to be neglected in report write-ups. Furthermore, it does not demand intensely close detailed analysis. Finally, you must make sure you go beyond simple labelling – themes need to cover the entire data set.
For more information on qualitative inquiry (for example, interpretative phenomenological analysis, conversational analysis, grounded theory) please see:
Willig, C. (2008) Introducing Qualitative Research in Psychology. Philadelphia:
Open University Press.
Smith, J., Flowers, P. and Larkin, M. (2009) Interpretative Phenomenological Analysis: Theory, Method and Research. London: Sage Publications.
Birth of the first PPI 161