Martín Sánchez-Jankowski
Sociologists and anthropologists have had a long and prominent history of using participation-observation methodology in addressing a full range of research questions. The primary concern of most of the participant-observation studies was to document the everyday activities of the societies, or sub-societies, they were observing. Endemic within this approach was the act of representation because the societies and sub-societies studied had to be for-mally represented to both other researchers and the general public. How this was to be done was left in the hands of the researcher(s) who were directly on the scene. Given that a significant amount of money and effort was required to undertake this research, there were generally few opportunities to replicate the study with the same group of people. Therefore, researchers using participant-observation as a method were entrusted with a consider-able amount of responsibility. Thus, there is not a participant-observation study that has not involved the concepts of ‘representation’ and ‘responsi-bility’. This chapter, in a very quick and skeletal way, identifies some of the tensions that exist for the researcher when dealing with representation and responsibility, the methodological problems that such tensions provide for the researcher in carrying out their study, the subsequent problems associ-ated with the evaluation of evidence emerging from these studies, and some of the cues that should be utilized to determine the level of internal and external validity present in the study.
Before continuing, I want to make clear that this chapter is not about ethnography; it is about research that utilizes participant-observation methodology. I have avoided the word ethnography because the word has come to engulf a number of different kinds of studies, many of which have not done and are not interested in using participant-observation methodo-logy (Gans 1999). Herbert Gans (1999) has observed that many researchers have utilized the word ‘ethnography’ to include a variety of conceptual and methodological approaches. For example, some position ethnography as an expression of writing (Clifford and Marcus 1986), or as part of the post-modern effort to deconstruct abstract categories by constructing alternative abstractions (Tyler 1986), or as a political activity (Marcus 1988). This chapter
on the other hand is about the methodology that has data gathered by the researcher being present, and participating in the activities of the subjects under investigation; directly observing them and the other social pheno-mena relevant to the research question, i.e. participant-observation.
O
OBBSSEERRVVAATTIIOONN,, RREEPPRREESSEENNTTAATTIIOONN AANNDD VVAALLIIDDIITTYY
Participation-observation has the advantage of being able to directly observe the behaviour of those who the researcher is interested in studying. It is pre-cisely this ability that has often allowed ethnographers to believe that inter-nal and exterinter-nal validity are integrated into the method of data collection itself.1That is to say, the data analysis is occurring at the same time that the data gathering is taking place, giving the participant-observer ‘face validity’.
However, this apparent advantage is not something that can be automati-cally taken for granted because ‘face validity’ is dependent on a number of contingent factors related to the observation of phenomena and its represen-tation. It is the relationship between observation and representation that I first want to turn my attention to. In this regard, I will be addressing the broad question of how observed events will be represented?
The relationship between observation and representation seems to be quite straightforward. A field researcher observes and then records what he or she sees, thereby representing a particular object of study. It is this directness that is the basis for the claim that participant-observation has ‘face validity’ built into it. Yet the simple sequence of observation and recording are filled with critical conceptual and analytic junctures that require close scrutiny on the part of the researcher. In observing a certain object (a setting, person, thing) there exists certain points where interaction forms what I have called a ‘crit-ical conceptual and analytic juncture’.
The first of these critical junctures has to do with the interaction of the observer and the observed, and the filtering system employed to gather the data. This filtering system will determine not simply what will be seen and not seen, but how it will be seen. Just how the filtering system will represent the observed phenomena will depend on the content of the system’s appa-ratus itself. For all participant-observers the filtering system is composed of the eyes, ears and smell associated with the sensory system of the body, as well as the knowledge-bank that is used to identify, catalogue and cate-gorize. This knowledge-bank is of course the information and the apriority conceptualized schema that individuals have accumulated and developed to make sense of the constant stream of everyday information that must be quickly (sometimes instantly) arranged in a manner that allows a field researcher (actually any person) to decide what the most appropriate and/or necessary behaviours for a particular situation are. For any person (and in particular the subjects of research) it is the development of a given list of appropriate and necessary behaviours for various situations that provides the methodological problem for participant-observation researchers because
they are dependent on their library of behaviours to determine what should, and should not, be included in the field notes. The fact that the researcher’s technical capabilities (i.e. knowledge base and conceptual propensity) are one aspect of the first critical juncture of representation in participant-observation research is not intended to minimize the issue of values in research. The fact-value issue is one that confronts all social research, but it presents an ever-present danger for the participant-observer because he or she is the data-gathering instrument. Thus, participant-observers must remind them-selves while they are observing and recording the data, as well as remind the reader once they are reporting their findings, that they were aware of this methodological issue and took steps to reduce errors associated with it.
Without a public acknowledgement of the problem and some discussion as to the strategy employed to deal with it, the findings are subject to questions concerning the probability of ‘type I and type II’ errors.
The second obstacle associated with the researcher’s filtering system has to do with the researcher’s library of cues that help him or her determine what is going on in a particular social environment or interaction. The more cues researchers have in their library, the higher the probability that they will recognize important conditions in the social environment that are vital in answering their research questions. However, the library of cues not only must be extensive, but also must be varied. By varied, I am suggesting that the cues accumulated must not be of the same kind. That is to say, they must not come from simply one experience, because if they do, there will be the tendency to miss either important data or interpret the observations they have made inaccurately. This is not to suggest that it is only possible for someone who has experienced the same environment through the same lenses can study that environment. However, if a person has no experiential knowledge of the environment it increases the probability that the data col-lected may, and the emphasis here is on ‘may’, increase the possibility of committing a type I or type II error.2
The types of experiences, and the concomitant cues that emerge from them, can be related to any number of categories. The most basic examples are associated with class, gender and race. What I am suggesting is that just because a scholar is trained and has familiarity with what the literature says about a particular area of study, there is no guarantee that he or she will avoid committing errors of commission and/or omission. This is because the participant-observation researcher is the instrument for data-gathering, and thus the responsibility associated with gathering all relevant data is placed solely on the researcher. Therefore, like any instrument (e.g. a survey ques-tionnaire), the researcher’s personal and field-research background is very important because the information accumulated during this process will have provided him or her with the cues to more accurately decipher what is important data to record and what is not (Jackson 1987: 107). As I have said, there are no guarantees of course, and it is not my intent to argue that schol-ars can study only the group from which they have come, but rather to draw attention to the fact that researchers who study a group in which they have
spent a good deal of time with, and therefore have become increasingly familiar with, there is a higher probability for them to see and record data
‘relevant’ to the research question than those with less familiarity (Emerson et al. 1995: 108–112). More specifically, knowing what it is that one is seeing is terribly important in determining whether such an event should be included in the notes and how it should be coded. The accuracy of such decisions will increase proportional to the amount of time spent by the researcher interacting with, and formally observing the subjects of study (Emerson et al. 1995).
The act of representation presents three rather major obstacles for the researcher. First, are all the actors, if actors are the primary observational objects, acting in a way that accurately represents: who they are personally;
who they are in the particular situation under investigation; or who they are as a proxy for a particular general group? This problem always presents itself in the beginning of the fieldwork because the researcher is, to varying degrees, an alien in the environment. There is the tendency on the part of the subjects to act in a way that minimizes who they think they are. That is to say, to minimize what they believe are their negative attributes and accentuate their perceived positive ones. Or they may simply try to be, or avoid being, someone they think the researcher believes they are. Thus, in studies where the researcher has spent very little time in the field, there is a higher proba-bility that the researcher has committed a type I or type II error (Leidner 1993).
The second major obstacle is to determine who saw what, when and how.
Sometimes this is associated with research projects in which there have been more than one field-researcher gathering data. When there are two or more independent observations and there is conflict in the content of the field observations there is the need to reconcile the conflict because determining how to represent an object is dependent on a consistent pattern. The problem is to decide whether there is a substantive difference in observed behaviour, or a difference in each researcher’s filtering system (May et al. 2000). If the differences remain random (i.e. no pattern) then it is very difficult to identify the exact origin of the problem. On the other hand, one might assume that if the differences were constant that it would be clear that a problem resonated with how the researchers were ‘observing’ people, places and events. Yet such an assumption would be premature because it is possible that each of the researchers were observing substantively different phenomena, or pheno-mena that presented itself in substantively different ways (Goffman 1959).
This presents a dilemma as to how the object of study will be represented because what observations will take precedence and why determines the substance of representation; but it does not necessarily produce an accurate representation of the substance.
The sequencing of observations is also a factor in how the object of study will be represented. Here, I am referring to researchers who are both inter-acting, and thus observing and conversing, with one or more of the subjects under investigation. On the one hand, this is the substance of participant-observation research; on the other, it presents the researcher with another
dilemma. Suppose the researcher is talking to one or more subjects, who recount an event that just occurred to them. The researcher observes their present state and has confirmation from all, or most, of those involved in a particular event that the event indeed occurred. Recording the data requires that the researcher note down what one or more of those involved in the event observed occurring. This event will be composed of both verbal and physical behaviour, and in listening to the subjects recount the event, there is a trail of verbal reflections on who said and did what, where and how. The substance of how this event will be represented is predicated on the obser-vations of the participants and not that of the researcher since the researcher was not present at this time. In this instance, the validity of the representa-tion can be increased if the researcher is using the news-media protocol of having three independent sources verify the action, but it cannot be authen-ticated. For example, recently there have been a number of investigative journalists who have written books representing the lives of African Americans living in poor and high-crime neighbourhoods. When reading these books, it is easy to be gripped by the events as they are recounted, but there are times when one wonders, ‘how does the author/researcher know that?’ This is particularly true when an event is being recounted and dia-logue is present (Kotlowitz 1991: 127; Dash 1997: 26–27). In those cases, the representation of the event, or the event’s representation of a larger object or condition, is predicated on unobserved phenomena, or at least unobserved by the researcher. Thus, the question of the observational accuracy, or its inexactness, remains an embedded feature in the researcher’s representation of the participants’ representations.
The means, or manner, of data collection is another factor in the represen-tational process. It matters how notes are taken. There is no paucity of books describing the ‘how to’ of note taking, but the exact mechanism utilized has not been rationalized to the point that it is uniformed across research pro-jects and disciplines (Jackson 1987; Sanjek 1990; Emerson et al. 1995). The question of what is the proper way to take notes is most likely to produce a vague answer like ‘whatever you find useful for both documenting what you observed; and writing up the results at a later time’ (Emerson et al. 1995:
13–16). An answer like this will allow researchers a great deal of flexibility for unique situations, but unlike other methods (e.g. survey questionnaires), it presents an irrational system full of ambiguity for systematic comparability and replication.
There are a variety of simple questions that have an impact on how sys-tematically comparable and replicable the data will be such as: Are notes taken by longhand, shorthand, or tape recording? Do the notes that are taken need to be reinterpreted and/or rewritten at some later point in time?
Are they stored in notebooks, on tapes, or computer files? Obviously, to address each of these questions adequately and consider their implications would require a separate chapter; however, to mention them raises the issue of whether the use of a computer file with its own structure and logic, might provide a means to log, classify, code and scale data in a way that
allows some rationality to pervade the note-taking process (Dohan and Sánchez-Jankowski 1998). For example, if data (field notes) were made avail-able in a systematically uniformed way, it would help to determine whether the account used in a paper, report or monograph was based on a single observation or some longitudinal pattern (Johnson and Johnson 1990).
Without such a logical system, anarchistic data collection procedures will create ambiguous findings, which in turn will create strains in evaluating validity and reliability of representations within and across studies. In par-ticular, there is the evaluative question of whether an author’s sociological (or anthropological, economic and political) insight is based on a single observation, albeit interesting, or patterned behaviour (i.e. multiple obser-vations). Obviously the difference is substantively significant because there is a greater chance of a type II error in the first instance than there is in the second.
M
MEEAANNIINNGG AANNDD RREEPPRREESSEENNTTAATTIIOONN
The representation of phenomena presents another challenge and that has to do with establishing the ‘meaning’ of an observed event or pattern of events.
This constitutes the second critical juncture for the researcher. There are two broad aspects to the concept of ‘meaning’ that have implications for the participant-observer. The first has to do with establishing, where appropriate to the research, the meaning of certain actions to the subjects observed. This can take the form of determining what meaning the subjects gave to their own action, or it can be the meaning they gave to other actors’ activities. In both cases, the primary effort is to understand how the subjects interpret their own and others’ motives and behaviours. This is a problem that anthropo-logy has confronted, and continues to confront, when studying divergent peoples and cultures (Geertz 1983). In essence, to penetrate accurately the subjects’ understandings of reality, these researchers must put themselves in the minds of the subjects they are studying. Is this possible? Must a researcher become the subject they are researching? In Geertz’s (1983: 58) judgement this is neither required nor possible, but for many other investi-gators the research question requires precisely this. In fact, participant-observers employing the ethnomethodological approach to their field of study believe not only that the researcher should be engaged in such ques-tions (subjective meaning for the subjects), but also that with sustained train-ing it is possible (Garfinkel 1967). Within this phenomenological-oriented tradition, the effort is to become the actor in order to go beyond empathy and establish understanding of the meanings that the actors of a given encounter gave to the encounter while it was occurring (Cicourel 1974;
Schutz 1982). Thus, representation of the ‘native’s point of view’ involved the interpretation of the native’s interpretation of the physical, metaphysical, and symbolic world (Geertz 1974; Wacquant 1995). This reincarnates the basic question of whether it is possible to interpret the words and linguistic
symbols of communicative behaviour as an accurate representation of an actor(s) meaning of an event(s). It also presents significant challenges to vali-dating the meaning of the representations. If a follow-up study is not able to determine that the same meanings were in use as were found in a previous study, does this invalidate the meanings that were recorded in the prior study? Does it nullify the meanings found in the present study? Regardless of the epistemological orientation, with such a situation, the clouds of both a type I and type II error remain settled into an evaluative inversion. It is for this reason that ethnomethodology relinquished the quest for an answer to such a question and argued that social meaning is time, place and situation dependent. It is not something that is reproducible. Thus, representation in this context has qualities that are both dynamic and static in nature. The irony is that what the researcher sees is not what the reader gets, but rather what the reader gets is what the researcher sees through the act of transfer-ence that accompanies their (the researcher’s) penetration of the subject’s subjective vision and its meaning (Emerson 1970).
Similar to the concerns of the ethnomethodologists, the ethnographic studies that emerge from a symbolic interactionist conceptual framework also attempt to represent the meanings that individuals bring to their social world (Blumer 1969; Cicourel 1971). Here, however, the empirical represen-tations are products of the researcher’s understanding of the meanings that the subjects bring to, and get from, their social interactions (Morris 1962).
This would also include two rather important factors. The first being recog-nizing and interpreting the symbols through which meaning is communi-cated, while the second has to do with determining the significance of the researcher (him/herself) to the subjects within their (the subjects’) social milieu. Therefore, the representations generated from this research are built on the social meanings of behaviour as defined by the actors in relation to their interactions with each other and the researcher. Thus, instead of con-sidering the researcher as an object of bias, as a participant-observer doing ethnography from a logical positivist tradition might, the symbolic inter-actionist simply considers it a part of the social environment that must be considered when trying to understand the social world (i.e. society) of the subjects (Horowitz 1995). In both the ethnomethodologist and symbolic-interactionist tradition, the representation of everyday life becomes first and foremost interpretively descriptive. Representing the ‘common sense’
through which people play out their ordinariness is the product of this research. Because there are so many individuals involved in the study, each with their own meanings for the interactional situations they find them-selves, the researcher ‘interprets’ what he or she believes is the subject’s own interpretive meaning for specific situations (Fine 1996, 1998). This pro-duces ‘rules’ (Goffman 1963) or ‘interpretative processes’ (Blumer 1969) through which actors navigate their social world. The result of these approaches is that what the reader gets is not what the researcher sees, but rather, what the reader gets is what the researcher sees the subject(s) see (Wellman 1988).
R
REESSPPOONNSSIIBBIILLIITTYY AANNDD RREEPPRREESSEENNTTAATTIIOONN
The concerns about representation are not simply limited to how social, politi-cal and economic phenomena will be depicted, but how they should be repre-sented. It is here that the issues surrounding the concept of responsibility penetrate participant-observation studies. Stated more simply: once the data have been gathered and analysed, what is the proper way that they should be represented? It is at this juncture that the fact–value issue arises again and it does so in a way that Weber (1949) did not write on. In the instance that I am now referring to, the fact–value dilemma presents itself in ethical garb; how-ever, it is the researcher’s values that are standing nakedly before the reader.
There is no place where values plays a more prominent role than in the determination of what to include in the public version of a report. Although many considerations are present, there are three that are quite prominent.
First, there are considerations as to whether the researcher ought to expose the faults of those he or she studies (Scheper-Hughes 1979). If they do, is it a violation of the ethics surrounding the social agreement made between researchers and subjects through the use of a formal protocol, or through the process of the researcher presenting him or herself as a close friend and con-fidant? Of course, this is a key issue to all research, but it is particularly pre-sent in participant-observation research because of the closeness (both physically and emotionally) that must be obtained between researchers and subjects. No doubt, there will be differences of opinion concerning the proper ethical position to be assumed, but the question itself conjures up the issue of ‘responsibility’ and its relation to ‘representation’. For example, at one time the dominant position among many researchers studying the poor or underprivileged in a particular country was that the final report should not include negative, or self-destructive beliefs and/or behaviours that were observed during the research. The arguments for this were centred on two separate propositions that were often working simultaneously with each other. The first of these was that since the condition of the poor was assumed to be caused by structural problems, researchers who did expose negative beliefs and behaviours about socio-economically vulnerable populations were in effect blaming the victim for their present condition (Ryan 1971). The work of anthropologist Oscar Lewis (1961, 1964, 1966, 1975) was a primary target for this criticism because he described many negative personal attri-butes of the poor he studied.
The second proposition was that even if the researcher was not blaming the victim for their present condition, the evidence presented in the work could be used by politically sinister parties to carry out, or rationalize, actions that would hurt the population depicted in the study. This criticism was often utilized against scholars doing studies on racialized minorities (Frazier 1957; Horowitz 1984; Anderson 1990). Although the criticisms were not always issued in print, they existed in both the formal and informal con-versations at professional conferences. The quality of the work was not the primary focus, even though some did try to undermine the arguments using
methodological criticisms. The primary impetus for such criticisms was that the work could be detrimental to certain racial/ethnic minorities that were already the object of socio-economic and political discrimination.
Reflecting on such criticism there was a tendency to see that the responsi-ble thing to do under such circumstances was to avoid describing the nega-tive behaviour that was observed, even if this was a contributing factor to the subjects’ place in life. Hence, responsibility for some researchers seemed to dictate that what they publicly reported was only part of what they saw.
Thus, what the broader research community confronted was partial evi-dence with very high probabilities of type I and type II errors.
Yet, there is another fear faced by researchers that affects the ‘quality’
of the data that are ultimately presented. This fear is again associated with the subjects, but its primary focus is on what the subjects think of what the researcher has observed. In a sense it is the fear that what they (the researchers) have seen will, when described in the report, be offensive to the subjects. The full extent of this fear is that it will be offensive to subjects who have been very cooperative in letting the researcher carry out his or her work. This fear is based on some level of guilt that the researcher broke the social contract that had been entered into with the subjects. In fact, there may well be an even greater feeling of guilt centred on the question of whether the research has exploited the subjects by merely treating them as tools for the researcher’s professional advancement (Scheper-Hughes 2000).
A far more intense fear is that if the subjects do not like what was said, they would go before the public and declare that what the researcher reported was not the truth. This occurred in a criticism of Whyte (1993 [1943]) and his Street Corner Society. In a follow-up study Boelen (1992) interviewed a number of the subjects that Whyte had identified in his study. She reported that most people refuted much of Whyte’s account of the area. This type of follow-up study cannot act as an adequate replication because the mere rejection of Whyte’s analysis on the basis of the subjects’ response to it is not valid. What would be necessary is for Boelen (or some other researcher) to study a sociological area that could be defined as a ‘slum’ and empirically demonstrate that Whyte’s findings about such areas were inaccurate. While Boelen’s approach was not valid, the use of negative evaluations of a participant-observation analysis by the subjects can have devastating effects on the validity of the entire research project, as well as the researcher’s career, so protecting against such an event is something that most researchers give some thought to.3 The upshot of this fear is to influence researchers to be very careful about what they report. Responsibility assumes a variety of characters in this context. One way that responsibility plays itself out is to have the researcher offer the subjects the opportunity to review what has been written about them. This strategy has the advantage of informing the subjects what the researcher has seen and thinks about the observations. The hope is that this will reduce the subject’s surprise, public embarrassment and humiliation, feelings of betrayal and outrage, and elim-inate retaliation. Thus, responsibility is associated with letting the subjects