All content analyses are ( or should be) guided by research questions. Sampling problems do not arise when analysts can answer their research questions by examining all texts of a particular population of texts, such as all of a given writer's works, all issues of a newspaper within a chosen period, all documents generated by a legal proceeding, the complete medical record of a patient, or all e-mails received and answered by a certain office, on a certain issue, and during a certain period in time. When researchers analyze a sample of texts in place of a larger population of texts, however, they need a sampling plan to ensure that the textual units sampled do not bias the answers to the research question.
Only when all sampling units are equally informative concerning a research question is sampling in content analysis the same as sampling in survey research. For such situations, statistical sampling theory offers three sampling techniques, summarily called probability sampling because they are designed to ensure that all sampling units have the same chance to be included in the sample. In the following subsections, I describe these sampling techniques first.
When sampling units are unequally informative, which is far more typical in content analysis than in survey research, the sampling of texts becomes a function of what is known about the distribution of information (content) within a textual universe. I describe below four sampling techniques that respond to this condition.
In addition to the distinction between equal and unequal informativeness of sampling units, there are situations in which researchers know their popu
lations of texts well enough to enumerate (assign numbers to) or comprehen
sively list the members of those populations. Regular publications, for example, have sequential dates of publication that are known before sam
pling. Many institutions keep accounts of texts in various forms, including library catalogs; Books in Print; professional guides to scholarly j ournals;
records of legal transactions; variously kept logs, diaries, chronicles, histories, and almanacs; reference works such as dictionaries and encyclopedias; and alphabetical directories. Among the many existing and widely used systems for enumerating texts are the ISBNs (International Standard Book Numbers) of books, URLs of Web pages on the Internet, telephone numbers, product numbers in catalogs-all the way to the page numbers of books. The first four sampling techniques reviewed below rely on systems of this kind.
(Enumeration systems may not be of uniform quality; for example, most URLs do not name active Web pages, and for some daily newspapers there may be publication gaps. )
A more challenging situation is one in which a population of text has a con
ceptual boundary but no enumerable members, for example, when a researcher is interested in information on a certain issue that could appear in a rather diverse population of texts. Cluster sampling, the fifth technique described below, is useful in situations where sampling units can be listed in larger chunks, or clusters. Cluster sampling also may be used in situations in which sampling units and recording units differ in kind and/or in number. Following the discus
sion of cluster sampling, I address three sampling techniques that deviate even further from the idea of selecting a representative subsample from a population.
And the final technique discussed, convenience sampling, contradicts the most important features of statistical sampling theory.
Random Sampl ing
To draw a simple random sample, a researcher must enumerate (or list) all sampling units to be included in or excluded from the analysis (issues of journals, authors, Web pages, speeches, turns at talk, sentences). The researcher then applies a randomization device-a device that grants each unit the same proba
bility of being included in the sample-to the enumerated units to determine which will be analyzed. Throwing dice is one way of selecting units at random, but a random number table is more versatile.
SAMPLI N G
1 1 5
Systematic Sampling
In systematic sampling, the researcher selects every kth unit from a list after determining the starting point of the procedure at random. In content analysis, systematic samples are favored when texts stem from regularly appearing publications, newspapers, television series, interpersonal interaction sequences, or other repetitive or continuous events. The interval k is a constant, so it will create a biased sample when it correlates with a natural " rhythm" in a list of units, such as seasonal variations or other cyclic regularities. For example, if a researcher examining issues of newspapers were to select every seventh day of the week, the New York Times science section, which is published every Tuesday, would be overrepresented if sampling commenced on a Tuesday and never included otherwise. For this reason, researchers should take care not to select every seventh issue of a daily publication or every even (as opposed to odd) turn at talk in two-person conversations. Hatch and Hatch's ( 1 947) study of marriage announcements in Sunday editions of the New York Times unwittingly demonstrated this kind of bias. The researchers systematically sampled all June issues between 1 932 and 1 942 and found an absence of announcements con
cerning marriages performed in Jewish synagogues; however, they failed to real
ize that all the issues sampled coincided with a period during which Jewish tradition prohibits marriages ( Cahnman, 1 94 8 ) .
Systematic sampling can b e applied t o any kind o f list; the units need not necessarily be consecutive events.
Stratified Sampling
Stratified sampling recognizes distinct subpopulations (strata) within a popu
lation. Each sampling unit belongs to only one stratum, and the researcher car
ries out random or systematic sampling for each stratum separately. Thus stratified samples represent all strata either in equal numbers (i.e., in proportion to their actual size) or according to any other a priori definition, whereas the properties within individual strata are sampled without a priori knowledge.
Newspapers, for example, may be stratified by geographic area of distribution, by frequency of publication, by size of readership, or by audience composition as obtained from readership surveys.
For many years, Gerbner and his colleagues analyzed a "typical week of U.S. television programming" each year, constructing that typical week through stratified sampling from the entire year's programming by the three major TV networks ( see, e.g., Gerbner, Gross, Morgan, & Signorielli, 1 995 ) . The strata were the networks' programming slots, much as they are listed in TV Guide. For each year, the researchers obtained a "typical week" by randomly selecting 1 out of the 52 programs aired over the year for each programming slot of each week
day. This "week" had no empty periods or duplications, and the sampling
method granted each program aired on the networks the same probability of inclusion.
6.2�4. Varying Probability Sampling
Varying probability sampling recognizes that textual units are unequally informative about the answers to analysts' research questions and so assigns to each sampling unit an individual probability of contributing to any one answer.
In pursuit of answers to research questions about public opinion, for example, analysts may sample newspapers according to their circulation figures. In such a sample, large-circulation newspapers, which presumably affect more people, would have to be overrepresented relative to low-circulation newspapers in order for their contents to relate to public opinion variables. Thus when Maccoby, Sabghir, and Cushing ( 1 950) were interested in the information that newspaper readers were exposed to, they listed all dailies within each of nine census districts (strata) in descending order of their circulation figures and assigned a probabil
ity to each newspaper according to its share in total circulation. Here, readership determined the likelihood that any given newspaper would be included in the sample.
Analysts may not find it easy to assign probabilities to sources of text in terms of their importance, influence, or informativeness. One strategy that has been used in such cases is to have experts rank the sources. In surveying psychological literature, for instance, Bruner and Allport ( 1 940) enlisted professional psychol
ogists to rank publications in order of their importance to the field. In studying newspaper coverage, Stempel ( 1961 ) relied on journalists. Some other kinds of evaluative sources that analysts might consult when sampling with unequal prob
abilities include best-seller lists, reviews (of books, plays, films) in prestige jour
nals, book awards, and lists showing frequencies of citations.
Researchers may also use varying probability sampling to reverse certain known statistical biases in representations of reality. For example, the mass media are likely to air the voices of celebrities and to suppress unaffiliated voices that may not fit the media's own conceptions of the stories being reported. To infer what might exist outside of such selective reporting, an analyst might need to give the rare occasion of normally silenced views more weight than unin
formed reiterations of mainstream ideas.
Cluster Sampling
Cluster sampling is the technique of choice when analysts cannot enumerate all units of analysis but find lists of larger groups of such units, or clusters.
Analysts start by listing available clusters, then select among them randomly, systematically, or stratificationally and bring all units of analysis contained in
SAMPLI NG
1 1 7
those chosen into the analysis. In fact, wherever sampling units and recording units (see Chapter 5) differ, cluster sampling is taking place. Because the units that are contained in the sampled clusters are unknown, not only in kind but also in number, the probability that a particular unit will be included in an analysis depends on the size of the chosen cluster. In content analysis, cluster sampling is used far more often than many realize.
Since the early days of quantitative newspaper analysis, communication researchers have sampled among issues of newspapers but then measured, coded, and analyzed every article, paragraph, or proposition contained in the chosen issues. If such sampling is done correctly, every issue will have the same chance of being included in the sample. And if the sample is large enough, it should also accurately represent the population of newspapers from which the sample was drawn, but it will not represent the population of units contained in the news
papers, because the probability of part�ular units' inclusion in the analysis depends on such factors as where the newspaper is printed, which newspapers publish which kinds of articles, and which tend to reflect which kinds of perspectives, discourses, or attitudes. In content analysis, cluster sampling is con
venient because text tends to be organized in relatively large units-j ournals con
taining articles, television shows featuring casts of characters, news broadcasts presenting issues, conversations occurring among participants-that address dif
ferent topics. Analysts handle these large units (each of which consists of mater
ial that was printed, recorded, or aired in one piece) as wholes; they give the units names or label them by dates, keywords, headlines, author names, or genres and catalog them for easy retrieval. The text's constitutive elements, usually the pri
mary focus of an analysis, thereby become secondary or implied by the way the large textual units, the clusters, are handled.
From the perspective of statistical sampling theory, the variance within clus
ter samples is likely to be exaggerated and sampling error remains uncontrolled.
In content analysis, where researchers choose texts according to the texts' likely ability to contribute to decisions on rather specific research questions, sampling by clusters is more economical than sampling from a list of all available record
ing units. If the recording units are very unevenly distributed across the sampled clusters, the researcher will find it difficult to justify statistical generalizations about these units. However, because generalization is not a very important issue in content analysis, it is usually sufficient for a researcher to take precautions to prevent the uneven distribution of recording units.
Snowball Sampling
Snowball sampling is a multistage technique. Analysts start with an initial sample of units to which they repeatedly apply a given set of sampling criteria.
This recursion produces a sequence of additions of sampling units that cause the sample to grow in size until a termination criterion is reached. A good example
is the sampling of the literature on a particular subject. Researchers may start with a recent text, note its references, examine the cited works for their refer
ences, and so on. If the field examined is a close-knit one, the researchers will find themselves in a dense network of duplicate citations. Snowball sampling naturally terminates when the process generates no new references. In the case of a study of the content analysis literature, the trail stops with an obscure 1 690 dissertation referred to by a historian of Publizistic (German for newspaper science) named Otto Groth ( 1 94 8 ) . One could complement this snowball sam
pling criterion by adding the requirement that the term content analysis be used and thus get up to 1 941 (Waples & Berelson, 1 94 1 ) as probably the earliest use of the term. This example illustrates snowball sampling that relies on citations of one work in another. The Science Citation Index ( Garfield, 1 979) has expanded snowball sampling of scholarly literature into the other direction, by iteratively generating lists of published articles in which particular works are cited.
Underlying all snowball sampling is the idea of intertextuality, the notion that units of text are connected, that they form actual or virtual networks within nat
ural boundaries. The network of scientific references is j ust one example. The unfolding in time of a story in the news, which makes one news item dependent on a preceding one; the reproduction of information from news wire services to public conversations; networks of literary relationships within which ideas but also plagiarisms travel; hypertext links connecting one text to another and one Internet site to another-all of these may be used as bases for snowball sampling.
Sociologists have studied the effects of social networks, such as how the buddy system in an organization influences promotions, how a subject is able to get a message to a famous person via a chain of acquaintances, and how rumors spread.
Analysts could use all such intertextualities to sample relevant texts naturally.
Snowball sampling starts with an initial set of sampling units, as I have noted
and it is important that researchers choose these units wisely. Snowball sampling ends when it reaches natural boundaries, such as the complete literature on a subject. When it reaches its boundaries, the importance of the starting sample dimin
ishes in favor of the sampling criteria that recursively create the boundaries. (All rumors, for example, have origins, but their transmission quickly renders those ori
gins unimportant. The limits that rumors reach have much to do with the networks through which they travel and the needs they serve in a population. ) But snowball sampling can also explode growing sample sizes exponentially, like an avalanche, in which case the researchers need to accept some constraints (e.g., requiring that cho
sen samples conform to more stringent inclusion criteria-that citations be multiple, for instance, not casual---or that the sample not exceed a manageable size).
Relevance Sampling
I n the sampling techniques reviewed above, texts are sampled according to their sources, situations, time periods, genres, and intertextualities-all of these
SAMPLI N G
1 1 9
can be used without significant reading or analysis of the sampled texts.
RelevaIlc� �<l}l:lpling, in contrast, aims at selecting all rextuai llnits that contribute Jo �n��eriIlggiven research questions. Because the resulting sample is defined by tb.� analytical problem at hand, rekvance sampling is also called purposive
�ampling (see, e.g., Riffe, Lacy, & Fico, 1 998, p. 86).
It is important to remember that the use of random samples always entails the admission that one does not have a clue regarding what the population of interest looks like or where to find the needed information. In content analy
sis, this is rarely the case. Cluster sampling already acknowledges that the universe of texts is partitioned onto large clusters and makes use of this knowl
edge. Snowball sampling presumes knowledge of the networklike organization of this universe of texts. }yhen llsing relevance sampling, analysts proceed by actually examining the texts to be analyzed, even if only superficially, often in
_ a multistage process. Suppose researchers are interested in akoholism in the United States; more specifically, they want to find out what conceptions drive the use of alcohol on college campuses, what makes this a problem, and for whom. A random sample drawn from all that people read, write, and talk about would certainly contain answers to these research questions, but the task of sorting through the mostly irrelevant records in the sample would be a hope
less undertaking. Perhaps the researchers' first step in reducing the task would be to think about where they might find relevant documents and what those documents are likely to contain. When searching the Internet for alcohol, using the Google search engine, the researchers may find, say, 7,230,000 mentions of the word. They then narrow the search to find documents relevant to alcohol consumption, say, on campuses: "alcohol + students" yields 1 , 1 40,000 hits;
alcoholism, 658,000; "alcoholism + students, " 1 3 1 ,000; "alcoholism +
students + academic, " 40,000; "alcoholism + students + academic + rehabilita
tion, " 1 0,500; and so on. Thus the size of a universe of possible texts is reduced to a sample containing, ideally, a manageable number of relevant texts. Of course, relevance sampling is not limited to Internet searches, nor does it require electronic texts and their containing keywords as criteria for rel
evance. In the case of research into alcoholism on college campuses, possibly the most relevant data are recorded interviews of students by students, reports by student counselors, accounts of fraternity parties, and medical and police reports.
Relevance sampling is not probabilistic. In using this form of sampling, an a!1alyst proceeds by following a conceptual hierarchy, systematically lowering the number of units that need to be considered for an analysis. The resulting units of text are not meant to be representative of a population of texts; rather, they are the population of relevant texts, excluding the textual units that do not pos
ses:s r�levant information. Only when the exclusion criteria have exhausted their ability to shrink the population of relevant texts to a manageable size may the analyst apply other sampling techniques. Issues of accurate representation may arise at that point, but only relative to the relevant units from which the sample was drawn, not relative to the whole population of possible texts.
Relevance sampling is so natural that it is rarely discussed as a category of its own. It has motivated political scientists since Lasswell's ( 1 941 ) World Attention Survey, which compared the political climates of several countries; Lasswell restricted his analysis to the "prestige" newspapers in these countries (ignoring the "less influential" local papers). In a study of the coverage of foreign affairs during the 1 990 U.S. congressional campaign, Wells and King ( 1 994) used the same logic to limit their content analysis to the New York Times, the Washington Post, the Los Angeles Times, and the Chicago Tribune. They reasoned that these newspapers include extensive international coverage, have their own news
gathering abilities, and serve as the main channels of knowledge about other countries for U.S. political elites as well as other newspapers. Most researchers adopt some kind of relevance criteria for defining the populations from which they sample.
The problems associated with relevance sampling hav,e gained in importance with the increasing use of very large electronic text databases and the Internet, where irrelevant texts are vast in number. Relevance sampling selects relevant data in ways that statistical sampling theory has not yet addressed.
Census
A body of texts that includes all of its kind is called a census. Studying the collected works of a particular author requires no sampling. The analysts may have to exert some effort to get ahold of these works, but that is a clerical task;
the analysts do not make any choices concerning what to include or exclude. For another example, if content analysts want to know something about the press coverage of a given event and collect all newspaper articles pertaining to the event, that complete set of texts constitutes a census. Because it is complete, the analysts have no need to expand the number of texts by snowballing, and if the set of texts is manageable in size, they have no need to reduce it by using relevance or random sampling.
Convenience Sampling
A convenience sample is motivated by analytical interest in an available body of texts that is known not to include all texts of the population that the analysts are concerned with. Such a sample is convenient in the sense that the analysts do not care to make an effort or find it too difficult to sample from that population. By proceeding from available texts without any sampling effort, analysts leave the matter of how and why the data-and which data
get into the sample to circumstances out of their control, to the interests of the texts' channels or sources, whether or not the latter are aware of how their texts will be analyzed.