Being able to state a research question clearly and spell out a research goal with precision are necessary but not sufficient conditions for the produc- tion of knowledge. In fact, they are worthless unless you are able to design and execute the research projects that will test the questions and move you closer to the goal. The methods explained below and summarized in Table 4.1 are all used in psychology of religion research, and each one is ideal for answering a question posed a certain way. Yet, they differ greatly among themselves. This means that they are complementary, each one offering its own unique sort of knowledge to the whole. Thus, when the same general question is tested using several methods, the outcome is a set of data that provides a particularly rich, possibly robust, picture of the phenomenon under study.
During the past 25 years the type of research approach used has expanded. When the first chapter on the psychology of religion was pub- lished in the Annual Review of Psychology (Gorsuch, 1988), correlational procedures were overwhelmingly the most frequently used (see also Hun- sberger, 1991). By the time the second Annual Review chapter came out (Emmons & Paloutzian, 2003), experimental methods began to get a foot- hold. About a quarter century ago, a debate flourished about whether psy- chologists could or should conduct manipulative experiments (see Batson, 1977, 1979, 1986; Gorsuch, 1982; Yeatts & Asher, 1979). Some research- ers seemed to think the pure laboratory experiment was the ideal model of how to do science and that the psychology of religion had to use it or be second rate. That debate is history. Researchers and theorists understand that it is against the abstract model of the perfect experiment that claims of public knowledge acquired by any method are evaluated. The ideal goal is knowledge of causal relationships, and experiments are more likely to get us closer to this goal than one-shot correlational studies. The idea of the pure experiment is a logical prototype of the process of knowledge produc- tion. This is because scientific knowledge is knowledge of differences. Let us get a taste of how to do good research in the psychology of religion by the correct application of a number of interesting methods.
126 TABLE 4.1. Summary of Psychological Research Methods: Defining Features, Strengths, and Weaknesses Research methodDefining featuresStrengthsWeaknesses Lab experimentsManipulation of independent variable(s) to examine potential effects on dependent variable(s).
Allows researcher to control the situation. Permits researcher to identify cause and effect, and to differentiate between placebo and treatment effects.
Situation is artificial, and results may not generalize well to the real world. Sometimes impossible to eliminate experimenter effects. Field experimentsManipulation of independent variable(s) to examine potential effects on dependent variable(s) in the context of the “real world” as opposed to the lab setting.
Closer approximation to “real-world” behavior than experiments conducted in a lab. Allows researcher to control the situation. Permits researcher to identify cause and effect, and to differentiate between placebo and treatment effects.
Sometimes impossible to eliminate experimenter effects. Quasi- experimentsLack of random assignment sets quasi-experimental designs apart from “true” experiments.
Easier to set up than pure experiments. Reduces threats to external validity.Internal validity may suffer without rando assignment. Naturalistic observationThe research scenario is in the context of everyday life, not the artificial setting of a lab.
Results relate directly to real-world occurrences; can study visible behavior.Allows researcher little or no control of situation. Observations may be biased. Does not allow firm conclusions about cau and effect. Field observationNo participation necessary on part of researcher; purely observation and recording of behavior(s).
With permission: No deception necessary.With permission: Observer effect is more l potential loss of accurate, “true” behavio Allows researcher little or no control of situation. Observations may be biased. Does not allow firm conclusions about cau and effect.
127 Participant observationResearcher is an active participant in the group that he or she is studying.
With permission: No deception necessary.With permission: Observer effect is more lik potential loss of accurate, “true” behavior. Observation of unusual groupsDone with naturalistic observation methods, but with a concentration on a particular group that is not considered part of the “normal” population.
Enables researchers to comprehensively learn about that population.Possibility for overgeneralization. Correlational studiesApproach that evaluates a potential relationship between two or more variables.
Allows for general predictions.Does not permit identification of cause and effect. SurveysUse of a questionnaire to collect data about internal states (opinions, thoughts, beliefs, etc.)
Ability to collect large amounts of data with relative ease.Self-report bias. Content analysisApproach used to quantifiably score, or code, qualitative information. This includes the use of interviews and written texts.
Reveals qualitative information; qualitative information can easily be turned into quantitative data for statistical analysis.
Can only provide descriptions of patterns found in the data, not any potential reasons why those patterns exist. PhenomenologyIntrospection from an individual’s first-person perspective, allowing for deeper understanding of one’s beliefs and motives.
Rich, deep information can be revealed. Useful when examining complex beliefs and behaviors.
Subjective nature allows for smaller sample sizes and is more time-consuming to implem in a study. Neurophysio- logical studiesStudy of the activity of the central and peripheral nervous systems with the use of physiological observations and measurements.
Reveals neurological correlates of behavior.Expensive; time-consuming; typically small sample size for these reasons.
Experiments in the Lab
The most precise form of research design is the laboratory experiment. This procedure allows you to come closest to a cause– effect conclusion of the form “changes in variable A reliably lead to changes in variable B.” In the psychology of religion, such a statement might be “adopting a transcen- dental belief system reduces people’s fear of death.” The laboratory experi- ment is the logical prototype for all empirical psychological research. It is the model of precision and clarity against which any research procedure or conclusion, including those from correlational procedures, is compared.
In a true experiment the investigator manipulates one variable, called the independent variable, to see whether changes in that variable bring about changes in a consequence variable called the dependent variable. Suppose, for example, you wanted to find out whether thinking about religious ideas causes people to feel emotional. You could show a list of religious words to one group of subjects (the experimental group) and a list of neutral words to another group of subjects (the control group). The independent variable in this case would be the religious versus neutral content of the word list.
The dependent variable could be the increase, decrease, or constancy in skin conductance due to perspiration in the palms of the hands, one sign of emotional response. You would determine whether people who are shown religious words respond more, or less, emotionally (as measured by skin conductance) than people who see neutral words. If the results show that the religious- word subjects have greater skin conductance responses than the neutral- word subjects, then one piece of evidence has been offered for the general proposition that seeing religious words or thinking religious thoughts causes emotional arousal.
It is difficult to do controlled experiments in the psychology of religion, and not long ago they were few and far between. Warren (1976) found that from 1950 to 1970 only 2% of the studies were true experiments. Although we want to treat some aspect of religiousness as an independent variable, we often cannot manipulate that variable ourselves. For example, you may be interested in the effects of conversion; but it is impossible to randomly assign people to the condition of being converts or nonconverts (nor would it be ethical to do so if it were possible). Fortunately, the recent past has witnessed a rise in the use of experiments, in both laboratory and field set- tings, in the psychology of religion (Hood & Belzen, 2005, 2013; Park &
Paloutzian, 2013).
A clever application of the experimental method involved the use of isolation tanks in which participants dressed in waterproof suits were sub- merged under water, floating slightly below the surface (Hood & Mor- ris, 1981). Isolation tanks are dark and silent and control body tempera- ture, which allows for the subjects’ thoughts to be the only salient focus of attention. Intrinsically and extrinsically religious participants were asked
to imagine either religious or cartoon figures while floating in the tank.
Results showed that the intrinsics reported more religious imagery under the religious imagination condition than did extrinsics. In another isolation tank experiment (Hood, Morris, & Watson, 1990), intrinsics reported reli- gious experiences whether prompted or not, in contrast to extrinsics, who did not. Experiments of this sort are discussed further in Chapter 8 in con- nection with research on factors that trigger or facilitate experiences inter- preted as religious or spiritual. In the years since these experiments were conducted, researchers have applied experimental methods to psychology of religion questions including but not limited to those from cognitive psy- chology (Farias & Barrett, 2013; Wenger, 2004, 2007), social psychology (Blaine & Nguyen, 2002; Ladd & Borshuk, 2013), neuroscience (Azari, 2006; McNamara & Butler, 2013; Schjødt, Stødkilde- Jørgensen, Geertz, &
Roepstorff, 2011), behavior in natural settings (LaBouff, Rowatt, Johnson,
& Finkle, 2012), and performance of mental tasks as hypothesized from ethnographic interview data (Luhrmann, Nusbaum, & Thisted, 2013).
In the Wild: Field Experiments
Field experiments allow the researcher some degree of control over the independent variable, although this control is often not as precise as that of a laboratory experiment. However, field experiments do allow you to give a treatment to some people but not to others, and consequently assess the effects of that treatment in both groups. An example of this type of experiment was done by Pahnke (1970). The subjects were theology stu- dents attending a Good Friday service. Some of them were given the hal- lucinogenic drug psilocybin before the service, while others were given a mild control drug with no “mind- expanding” properties. Those who took psilocybin were significantly more likely to have a mystical religious experi- ence at the Good Friday service, compared to the control group. The key things that make a field experiment desirable are that (1) the investiga- tor can exercise some control over the independent variable treatment and avoid too much confounding due to extraneous variables, and (2) the data are obtained in the ordinary context in which the research participants behave, so the results are generalizable to that “real life” context. There is no concern about the “artificiality” of the laboratory.
The results also maintain the internal validity of the experiment while maximizing contextual realism, which makes them applicable to real set- tings (Hood & Belzen, 2005, 2013). It is precisely because of this contex- tual realism that field experiments can reveal different patterns of results than experiments conducted in laboratories. In light of the MIP, both types of study are necessary and useful because they keep us from overinterpret- ing the meaning of either one (Hood & Belzen, 2005, 2013).
QuasiExperimental Methods
Natural quasi- experiments share the features of field experiments, with one exception: in a natural experiment you do not manipulate the independent variable. Nature manipulates it for you and you record its effects. Natu- ral experiments take advantage of natural conditions as a “manipulation.”
Good applications of this approach can be read in articles by Hood (1977, 1978), Chilcott and Paloutzian (2016), and Xygalatas, Konvalinka, Roep- storff, and Bulbulia (2011). Hood was aware that people sometimes report a religious or mystical experience when they spend time in the wilderness, where they become more acutely aware of nature. He also had theoretical reasons to expect that the amount of stress someone felt might affect how likely it would be that the person would have such an experience. In order to explore this question he took advantage of an excursion in the wilderness that was required of boys in a private school. He was able to measure the amount of stress the boys felt in anticipation of doing potentially dangerous activities such as whitewater rafting or rock climbing. He found that boys who scored lower in anticipatory stress for such high- stress activities also had higher scores on a measure of mysticism.
In general, in order to use the natural quasi- experiment method you must be prepared to exploit the research possibilities embedded in circum- stances beyond your control. As you can imagine, there are lots of possibili- ties for quasi- experimental studies in field situations and for using antici- pated natural events as experimental “manipulations.” However, doing these studies can be time- consuming, expensive, and impractical. When circumstances do not allow for the use of the experimental method, cor- relational and survey research can be a good option.
Naturalistic Observation
In naturalistic observational studies, the researcher takes advantage of nat- urally occurring circumstances in the real world. Rather than artificially constructing a situation to test some hypothesis, as would be done with for- mal experimental procedures, the researcher uses the “natural laboratory”
of real life. One advantage of naturalistic observation is that the results speak directly back to real-world events. Another advantage is that with naturalistic methods we can study actual, overt behavior directly, rather than measuring only opinions or other mental dimensions as is the case with questionnaires.
In one type of naturalistic method called field observation, the inves- tigator develops coding categories appropriate to the behaviors under study and then observes and scores the subjects’ behavior without interfering in any way. An example would be if you wanted to find out how religious denominations differ in the degree of vocalizing done by the congregation
during church services. An outside, nonparticipating observer would visit the worship services of the various groups and record how often people in the audience spoke, what they said, whether or not the minister, imam, or priest encouraged it, and so on.
Another type of naturalistic method called participant observation is used when it is necessary for the investigator to appear to be actively involved in the group whose behavior is being studied. In this case the investigator “joins” the group and acts like an average participant. A good participant observer takes care to be inconspicuous in order to avoid draw- ing personal attention and distorting the situation. The necessary data are thus gathered from inside the group, resulting in unique findings not obtainable by other methods. A classic example of the participant observa- tion method appears in When Prophecy Fails by Festinger et al. (1956). The authors infiltrated a “doomsday group” that predicted the end of the world.
Their study of the formation, maintenance, and dissolution of the group, and of the activity of the group members after the world was not destroyed on the expected date, became a milestone in the study of social forces oper- ating in small religious groups.
Both field and participant observation can be done either with or with- out permission. There are advantages and disadvantages either way. The obvious advantage of getting permission to observe and record what others are doing is that it is honest and transparent. No deception or later debrief- ing is required. The important problem with observation with permission, however, is that those who are being observed know they are being watched and that their behavior is recorded. This often creates reactive effects, that is, the behavior that is being studied is changed because of the very process of being studied, so that the behavior that is observed differs from what it normally is (which is what one wishes to study). The solution to the prob- lem of reactive effects is to conduct the observations without permission.
The problem with that, however, is that it can be seen as unethical and, except for behavior in the public domain (e.g., walking down the sidewalk), as a violation of privacy. Because of the issues illustrated by these examples, professional associations and governmental agencies have published guide- lines for scientific research involving human subjects. Researchers in the psychology of religion may need to resolve such issues in their research protocols.
Studying Unusual Groups
The approach of studying unusual groups has been used in conjunction with the methods listed above. It involves studying specialized groups or individuals and generalizing the findings to the ordinary population as far as is valid. Studying unusual groups has both positive and negative aspects. A positive aspect of this approach is that it enables you to learn
much about one special population, as is illustrated by the Festinger et al.
study mentioned above. It could also illustrate especially potent forms of religiousness. But these good aspects set the stage for a possible negative side effect— overgeneralization. This is a difficulty for any approach that involves studying only a unique group and then drawing conclusions about the population as a whole. Freud, for example, studied neurotic patients and then applied his theory to normal individuals. Albert Ellis (1962) made the logical error of concluding that all religion is bad for mental health by basing his conclusions primarily on observations of his clients (who came to him because they had problems). Even James (1902/1985; see comments in Figures 2.2 and 2.3, in Chapter 2) based his insights on the analysis of cases of extreme religious experience. In fact, many important contri- butions to understanding religiousness have come from studying extreme cases. Therefore, there is an important place for this approach. At the same time, it would be a mistake to rely exclusively on this approach because global generalizations about religiousness based solely on highly special- ized populations might not be valid. Instead, we should balance the study of specialized cases with its opposite, the study of religiousness in normal populations, from which we then generalize to and predict the deviations.
Correlational and Survey Studies
The most common form of research in the contemporary psychology of religion is the questionnaire and survey. The word survey refers to the pro- cedure for sampling the subjects for a particular study; questionnaire refers to the instrument itself.
Questionnaire survey data can be collected from one person at a time, but a big advantage of this method is that data can be collected from many subjects at once. Subject sampling procedure can be random, representa- tive, or biased, with several specialized techniques available for determin- ing who should be in the sample. Questionnaires do not measure overt behavior. Instead, they ask a series of structured questions about people’s opinions or judgments on issues, their memory of past behavior, and guesses as to how they might behave in a particular situation. For example, a ques- tionnaire might be able to assess whether people claim that they actively participate in church work, but you cannot measure actual church- working behavior with a questionnaire.
A psychometric issue concerning the use of behavioral versus ques- tionnaire measures of religiousness is applicable here. Whether a study is experimental or correlational,2 it is highly desirable to include behavioral
2 In a correlational study, the investigator assesses only the degree of association between the variables. In an experimental study, the investigator independently manipulates a variable(s) to see whether that manipulation produces a change in the other variable.