and Moderator on Response Quantity and Quality Author(s): Edward F. Fern
Source: Journal of Marketing Research, Vol. 19, No. 1 (Feb., 1982), pp. 1-13 Published by: American Marketing Association
Stable URL: http://www.jstor.org/stable/3151525 . Accessed: 16/04/2014 12:00
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].
.
American Marketing Association is collaborating with JSTOR to digitize, preserve and extend access to Journal of Marketing Research.
During the past 20 years many marketing researchers have reported the successful use of focus group inter- viewing to solve a variety of problems faced by mar- keting managers. However, focus groups have received little empirical scrutiny in the marketing literature and virtually no study in the literature of any other discipline.
We therefore undertook a study to test some widely held assumptions about focus group methodology. Empirical research reports on brainstorming, creative problem solving, and group psychotherapy suggest that some of the assumptions upon which the focus group method- ology is based are questionable. Evidence from these research reports is the basis for the research hypotheses which we tested empirically. Our study is an initial effort to assess a widely used research technique employed routinely in a variety of situations.
Of the 262 companies responding to a survey by Greenberg, Goldstucker, and Bellenger (1977), 47% in- dicated they had used focus groups. Consumer goods companies led the list with 81% reporting the use of fo- cus groups; marketing research and consulting firms were second with 61%. In comparison, 79% of the con-
sumer goods companies and 91% of the marketing re- search and consulting firms reported using personal in- terviews. The use of mail surveys was reported by 79%
of the consumer goods companies and 70% of the mar- keting research and consulting firms.
The current popularity of focus groups is not difficult to understand. Many reports on the use of this technique cite the relatively low cost as well as the speed with which a focus group report can be obtained (if transcripts are not required, it may take only a few days). Also, focus groups are apparently flexible because they have been used for:
-generating hypotheses,
-exploring opinions, attitudes, and attributes, -testing new product ideas,
-evaluating commercials, and
-identifying and pretesting questionnaire items.
Despite this widespread use, focus groups have not been tested empirically nor has a theory of focus group inter- viewing evolved. A wide range of notions about why focus groups work appears in the available literature.
However, agreement among authors on the subject is only at the most superficial level (e.g., participants should have something in common).
If a reliable methodology of focus group interviewing is to be developed, more fundamental issues need to be explored. First, some boundaries need to be set for use of the technique. Focus groups are interactive discussion groups. Therefore an individual's behavior in the group Assumptions about focus group interviewing were tested. Individual interviews generated more ideas than focus groups, eight-m-,nber groups generated signifi- cantly more ideas than four-member groups, no differences were found between focus groups and unnuderated discussion groups, and the effect of acquaintanceship
was not clearly determined.
The Use of Focus Groups for Idea Generation:
The Effects of Group Size, Acquaintanceship,
and Moderator
onResponse Quantity and
Quality
*Edward F. Fern is Assistant Professor, Department of Marketing, Virginia Polytechnic Institute and State University.
This research was supported by the Department of Defense. The author thanks an anonymous JMR reviewer for comments which re- sulted in substantial improvements in the article.
1
setting may be influenced by the presence of other group members. To the extent the presence of others has un- intended or unwanted effects on individual behavior, fo- cus groups may be an inappropriate data collection tech- nique. A second issue pertains to focus group methodology. Most researchers in this area have their own "tricks of the trade" for structuring group sessions, which within reason may be essential. However, some general guidelines should be applicable to a specific fo- cus group task regardless of the idiosyncracies of the moderator. A third issue is that naive theoretical expla- nations about why focus groups work have all too fre- quently gone unchallenged. Moreover, much of the trade jargon is rhetorical and provides little or no understand- ing of this phenomenon. Finally, there is no empirical evidence to support any of these explanations.
Common Assumptions of Focus Group Interviewing Four assumptions about focus group interviewing were explored: (1) the focus group's output is in some way better than the output of individual interviews, (2) moderators are crucial to the focus group process, (3) groups should be composed of eight to 12 members, and (4) group participants should be strangers. These as- sumptions were chosen for three reasons. First, although there is little agreement among focus group researchers on the exact methodology to be used, these four as- sumptions are more widely held than others. Second, the results of studies in the areas of group problem solving, brainstorming, and counseling psychology seem to con- tradict these assumptions. Because of the contradictory evidence, we believed the more widely held assumptions should receive primary emphasis in this initial study.
Third, we considered these assumptions to be repre- sentative of the "state of the art," and by choosing them could show that much of what is accepted on the basis of past focus group experience can be tested empirically.
The first assumption-that the group's output is in some way better than the output from individual inter- views-generally follows from the intuitively appealing idea that two heads are better than one and that groups benefit from synergy in generating ideas. Focus groups are purported to provide more information which is qual- itatively different from that obtained by summing the results of individual interviews (Goldman 1962) and a wider range of information, insight, and ideas than can be obtained from individuals (Hess 1968). Explanations for this notion include greater spontaneity and candor in groups (Goldman 1962), release of group members' in- hibitions (Hess 1968), and greater anonymity or security provided group members (Hess 1968).
A second assumption is that the focus group moder- ator's role is crucial in obtaining the desired information from focus groups. The moderator's expertise, person- ality, and the procedures used are viewed as critical in promoting group interaction. Expertise factors include ability, knowledge in social or clinical psychology, past focus group experience, and product or problem knowl-
edge. Moderator traits deemed important are sensitivity, outward personality, and a genuine interest in people.
Moreover, activities considered important include con- trolling dominant respondents, activating shy respon- dents, extending the range of the discussion, regulating interactions, coping with interruptions, and counteract- ing the leader effect.
A third assumption is that the ideal group size is eight to 12 members (Merton, Fiske, and Kendall 1956; Smith 1972). However, focus groups can consist of as few as five members (Sampson 1972) and as many as 20 (Hess 1968). According to Wells (1974), the ideal group size depends on the seating arrangements and the inter- viewer's personal style.
A final assumption is that focus group participants should be strangers. Acquaintances may seriously upset the dynamics of the group and inhibit responses (Smith 1972). Payne (1976) calls for screening out certain groups, "who, for some reason, might bias your re- sults." According to her, respondents from the same neighborhood, church, club, or ethnic group are not de- sirable.
Contradictory Evidence
Although no empirical evidence was found to support the assumptions about focus groups, evidence that seems to contradict these assumptions was found in related fields. In one study, brainstorming groups of undergrad- uate men generated fewer ideas and lower quality ideas than those men working alone (Taylor, Berry, and Black 1958). Dunnette, Campbell, and Jaastad (1963) repli- cated Taylor's findings with research scientists and ad- vertising personnel. In a problem-solving task (Campbell 1968), managers working alone generated higher quality solutions than four-man groups.
Studies in psychotherapy raise the question of whether therapists are necessary. Vicino et al. (1973) found that self-administered training groups were effective in ac- counting for behavior change among group members.
Poser (1966) found that groups of mental patients treated by lay therapists (i.e., undergraduate women with no background in psychology) scored higher on post-ther- apy tests than patients treated by certified psychiatrists.
The evidence seems to indicate that other means of structuring group interaction may be just as effective as using group moderators.
Little agreement is found among brainstorming re- searchers on optimal group size Slater (1958) found five members to be optimum for discussing a human relations problem. Osborn (1953) suggested five to 10 members as optimum for brainstorming. Others have found no difference in the number of ideas produced between five- and nine-member groups (Bouchard and Hare 1970) and between four- and seven-member groups (Bouchard, Barsaloux, and Drauden 1974).
None of the aforementioned studies varied acquain- tanceship. However, Brown (1970) has shown that face- saving behavior occurred after the embarrassing act of
sucking, biting, and licking a rubber pacifier in front of others. Brown and Garland (1971) found that male stu- dents spent as much time singing in front of acquain- tances as they did in front of strangers. When these re- searchers used "closest" friends as the audience, subjects sang for a significantly shorter time than when acquain- tances or strangers were the audience. Although the sex of the audience had no effect on male singers (Garland and Brown 1972), women sang four times longer in front of men than in front of women. Also, women facing other women felt more inadequate, more foolish, and less similar to their audience, and felt that their audience was more critical of them.
The Nature of the Research Task
The research hypotheses related solely to a task con- sidered to be exploratory. The purpose of the research was to generate as many different ideas as possible.
Therefore caution should be used in generalizing from the particular task used in the study to other tasks fre- quently performed by focus groups. The study did not call for uncovering motives, evaluating ad copy, or dis- cussing product experiences. Calder (1977), from a phi- losophy of science perspective, delineates three distinct approaches to focus group interviewing. We recognized these differences and purposely constrained the study to exploring ideas relevant to a particular discussion topic.
The Discussion Topic
The major criterion in choosing a discussion topic was that little or no information be available on the topic. A second criterion was that the topic be fairly complex.
Some topic areas might raise so few discussion issues that the average person could mention all of them in one interview. A third criterion was relevance to a marketing problem (e.g., eliciting thoughts or ideas). A final con- sideration was the availability of funds for the project.
The Department of Defense was interested in an ex- ploratory study on the public's thoughts about "expand- ing the role of women in the military." This topic ap- peared to meet the criteria and its selection provided funding for the project. Moreover, the sponsor agreed to judge the quality of ideas generated in the project.
Research Hypotheses
The first set of hypotheses embodies the assumption that focus groups provide more information than indi- vidual interviews.
H111: Moderated (unmoderated) groups of eight mem- (H1I21) bers will generate more different ideas than the
combined output of eight individuals.
Hi112: Moderated (unmoderated) groups of four mem- (H1 22) bers will generate more different ideas than the
combined output of four individuals.
The next set of hypotheses addresses the issue of whether the difference between focus groups and indi- vidual interviews is related to group size. According to
some focus group researchers, the superiority of groups over individuals should increase as group size increases.
Hi131: The difference in number of ideas generated be-
(HI 32) tween moderated (unmoderated) groups and in- dividuals is greater in groups of eight members than in groups of four members.
Whether the combined output of individuals is of higher quality than the output of groups is tested in the following hypothesis.
H,I40: Moderated groups will generate ideas of higher judged quality than indivduals.
The literature on focus groups suggests that group moderators have an instrumental role in the focus group process. The following research hypotheses test the ef- fect of moderators on idea generation.
H2.11 (H2.12)
H2.13:
Moderated focus groups of eight (four) members will generate more different ideas than unmod- erated groups of eight (four) members.
Moderated focus groups will generate ideas of higher judged quality than unmoderated groups.
As group size increases, the role of the moderators may become more critical. Guidance and management activities would be expected to be more important in large groups than small. This common-sense notion is tested in the next hypothesis.
H2.21: For the number of different ideas generated, the difference between moderated groups and unmod- erated groups is greater in groups of eight mem- bers than in groups of four members.
If there is a difference between unmoderated groups and moderated groups, it may be due to the effectiveness of the moderator. This difference is thought to be a result of the moderator's participation in the interview process and is referred to as the moderator effect. The difference between nominal groups of individuals working alone and individuals being interviewed should measure the interviewer's effectiveness and is referred to as the in- terviewer effect. It is generally acknowledged that focus group moderators are required to have more education, skill, and experience than interviewers. By comparing the interviewer effect with the moderator effect one can show whether these moderator requirements translate into greater effect in generating ideas.
H2.31: For the number of different ideas generated, the
(H2.32) moderator effect in groups of eight (four) mem- bers is greater than the interviewer effect.
Eight-member focus groups should generate signifi- cantly more ideas than groups of four members if the optimum group size is eight. The brainstorming litera- ture, however, appears to suggest no significant differ- ence between these two group sizes. The focus group position on this issue is summarized as:
H3.11: Moderated (unmoderated) groups of eight mem-
(H3 12) bers will generate more different ideas than mod- erated (unmoderated) groups of four members.
A reason given for using large focus groups is that larger groups produce a greater synergistic effect (Hess
1968). A counterargument to the synergism hypothesis is that the incremental number of ideas generated as group size increases will decrease-diminishing returns will occur. Testing these competing hypotheses requires the antithetical notion that individuals are groups of one member-groups by definition have more than one member.
H321: The incremental number of different ideas will in- crease as moderated group size increases from one to eight members.
Anonymity has been offered by focus group research- ers as a factor accounting for enthusiastic, honest, and spontaneous responses among group participants. The anonymity afforded by strangers might lead to less in- hibited behavior and greater spontaneity of response.
These notions are tested in the following hypothesis.
H4.0: Focus groups of eight strangers will generate more different ideas than focus groups of eight acquain- tances.
METHOD Research Setting
The study was conducted at the Behavioral Science Laboratory of a large university in the Midwest. Three experimental sessions were held each day over a seven- day period. All data were collected within a relatively short time period to avoid problems such as a news story relating to the discussion topic appearing in the media.
The Behavioral Science Lab is not like a kitchen or livingroom setting in which focus groups are frequently held, but does provide several advantages for this type of study. First, the lab is equipped to provide equivalent interview settings across experimental conditions. Sec- ond, the lab has recording equipment and a trained tech- nician to ensure that the equipment is set up and oper- ating properly. Finally, the lab had been used previously by commercial market research firms to conduct focus groups.
Recruiting and Sample Characteristics
Because social class homogeneity within focus groups is claimed to be important by authors writing on the use of focus groups, we determined that all respondents in the study should be as homogeneous as possible. There- fore, participants were recruited from the 4300-member women's auxiliary of a children's hospital and from a local garden club. As most of the women in the auxiliary and garden club were 30- to 50-year-old middleclass housewives residing in the suburbs, they seemed to meet Goldman's (1962) criterion of homogeneity.
Payment was a $10 donation per participant to the organization. All recruiting was done by a member of
the organization from approximately 200 neighborhood units of the auxiliary. The recruiter asked the women to particpate in a study on "job opportunities for women."
Every effort was made to maintain the subjects' ano- nymity to the researcher, and the subjects were asked to provide only their first names and the name of their organization.
The Design
The design was a partial factorial with group size (three levels), group type (two levels), acquaintanceship (two levels), and moderators (two levels) as the factors.
Because four or eight individuals were required for a single group observation, the cost of a full factorial de- sign (with 16 experimental conditions) would have been prohibitive. Also, much of the additional information provided by a factorial design would have been irrele- vant. The treatment conditions examined are depicted in Figure 1.
Independent Variables
The independent variables in the study were group type, moderator, group size, and acquaintanceship.
Group type. Group type was varied at two levels.
Respondents were either part of an interacting discussion group (real group) or they discussed the topic individ- ually and their ideas were aggregated as though they had been generated by a group (nominal group). Direct com- parison of eight individuals with eight-member groups would have resulted in a greater amount of redundancy for individuals. Therefore, the transcripts of ideas from individuals were drawn randomly (either four or eight) and assigned to the nominal group condition. Within each nominal and interacting group, redundant ideas were eliminated (i.e., each idea was only counted once) to make them directly comparable.
Moderator. Groups were either moderated or unmod- erated. Real groups were moderated by four experienced focus group moderators donated by research firms and
Figure 1 RESEARCH DESIGN
Group Size
One Four Eight
Group Type
Real Real Nominal Real Nominal
Acquaintanceship
Strangers Strangers Strangers Strangers Acquaint. Strangers Moderator
No Yes No Yes No Yes No Yes Yes No Yes
1 2 3 4 5 6 7 8 9 10 11
n=32 n=32 n=16 n=16 n=16 n=16 n=32 n=32 n=32 n=32 n=32
g9= g=4 4 4 g g=4 4 g=4 =4
n = number of individuals per cell g = number of groups per cell
advertising agencies. Nominal group interviews were conducted by professionally trained interviewers. All moderators and interviewers were women within the age range of the respondents. Each moderator conducted one group in each experimental condition and the experi- mental conditions were varied systematically across time periods so time of day would not be confounding.
Group size. Groups were composed of either one, four, or eight members. Additional groups of 12 would have been desirable to increase the range of this variable beyond the hypothesized ideal group size and to increase generalizability. However, the additional cost was pro- hibitive.
Acquaintanceship. Acquaintanceship was manipulated by assigning members from different neighborhood units of the 200-unit auxiliary to the stranger condition and members from the same local unit to the acquaintance condition. Choosing acquaintances from the same or- ganization was done for two reasons. First, if acquain- tances were no less productive than groups of strangers, all eight people in a focus group could be recruited from the same organization. This approach might both sim- plify respondent recruiting and reduce the cost. The sec- ond reason is that finding groups of eight that had stronger friendship ties would be difficult. We deter- mined that more control could be achieved by recruiting acquaintances within an organization and there were practical management implications in doing so.
Dependent Variables
Two primary dependent variables were used: (1) the number of different or unique ideas relevant to the dis- cussion topic and (2) the judged quality of ideas.
The number of different relevant ideas is simply the total number of ideas relevant to the discussion topic minus the redundant ideas. An idea was counted only once even though it was given by different members of a group. For nominal groups redundancy was eliminated after the individual interviews were combined into a group transcript.
The second primary dependent variable of interest is the quality of ideas generated. There is little agreement among researchers as to what constitutes quality of ideas. Most operational definitions of quality are specific to the particular task the subjects are asked to do. Some of the definitions that have been used are (1) originality, (2) feasibility, (3) effectiveness, (4) importance, and (5) uniqueness.
Much like the tasks in creative problem-solving and brainstorming studies, the specific task in this study was to determine the perceived consequences of an event.
The task differed from brainstorming tasks in that "free wheeling" or wild ideas were not solicited. The focus of the task, therefore, was not on creativity-this is not to say that the respondents were uncreative. The purpose of the task was to generate ideas useful to a policy maker in one of two ways: (1) to provide insights into focal issues for a communications strategy or (2) to suggest
areas of public concern to be included in a survey study.
Logically, the policy maker should judge the ideas in terms of how useful or how good they might be for his purposes. The policy maker agreed to do the judging task and was provided lists of ideas and several sugges- tions to help use his time most efficiently. Admittedly, this procedure poses a problem common in focus group research-the quality of information is based on a single judgment.
Experimental Procedure
Upon the participants' arrival at the Behavioral Sci- ence Lab, the researcher listed their first names and the names of their organizations. All subjects were greeted at the door and ushered into a waiting room where coffee was available. As no attempt was made to isolate indi- vidual subjects, strangers had an opportunity to converse and to become casually acquainted during a 10-minute period. This meeting could have weakened the acquain- tanceship manipulation by decreasing anonymity.
When all subjects for a given session had arrived, every other one on the list was selected for a group ses- sion; the rest were assigned to individual sessions. As a result, some pairs of subjects who had driven together were assigned to the same group in the stranger condi- tion. One group moderator reported a pair of friends in her group but she did not think it had an effect on the group's performance. Subjects selected for the group conditions were ushered first into the group room for their discussion. The remaining subjects were then taken, one at a time, to the individual rooms. When the group session was finished, all subjects were reunited and debriefed.
Experimental Conditions
The experimental conditions were (1) individuals working alone, (2) individual interviews, (3) groups of four strangers without moderators, (4) groups of eight strangers without moderators, (5) moderated focus groups with four strangers, (6) moderated focus groups with eight strangers, and (7) moderated focus groups with eight acquaintances.
Individuals working alone were led to small rooms and told, "Please have a seat, read the instructions, and do as they say. I will be back in about 30 minutes." A set of instructions was placed on a table in front of the subject along with two microphones which were ap- proximately two feet away. Ten minutes usually was required to read the instructions, leaving approximately 20 minutes for idea generation. At the end of 30 min- utes, the researcher returned with a post-experimental questionnaire; all subjects had finished the task by this time.
Individual interviews were conducted in the rooms used by the persons working alone. A slight alteration of the seating arrangement was necessary to accommo- date the interviewer.
The unmoderated group discussions of four and eight
members were held at a conference table in a room ap- proximately 12 by 20 feet. Subjects were positioned so that each was approximately two to three feet from a microphone. Each subject was provided the same in- structions as those given the individuals working alone.
One hour and 15 minutes was allowed for groups of four and two hours was allowed for groups of eight.
Moderated focus groups of four strangers, eight strangers, and eight acquaintances were conducted in the same room as were the unmoderated groups. The table arrangement was altered to accommodate the moderator.
Focus group moderators chose to instruct the subjects rather than have them read the instructions as in the other conditions. The moderators also used a portion of the available time (approximately 20 minutes) to lead into the discussion topic. However, the total time allotted for moderated focus groups was equivalent to the time al- lotted to the other groups. A couple of groups in each of the moderated-unmoderated conditions finished be- fore the allotted time was up. In these cases, and when the allotted time ran out, the group discussion was ter- minated and the post-experimental questionnaire was administered.
Data Preparation and Editing
Fifty hours of recorded conversation resulted in 600 pages of verbatim transcripts. The researcher randomly selected segments from all transcripts and checked them against the original tapes to ensure the transcripts' ac- curacy.
Two female MBA students edited the transcripts for major ideas (Horowitz and Newman 1964). First they bracketed all ideas which included major, subordinate, and ancillary thoughts. Communication and orientation signals (i.e., signals that an idea was forthcoming and questions about the task) were ignored. A second check through the transcripts was made to determine which ideas were major. Major ideas were defined as utter- ances expressing a thought in a meaningful, relevant, and unique way. The thought had to be meaningful to the editor, relevant to the discussion topic, and unique rather than a restatement or elaboration of a previously stated idea. The editors were provided a set of instruc- tions and six randomly selected pages of transcript upon which to practice. In the total number of ideas consid- ered major, the two editors were in agreement 60% of the time. The composite reliability coefficient (Holsti 1969, p. 137) was .75.' The disagreements were due mainly to differences in handling (1) agree/disagree re- sponses to the interviewers' questions, (2) statements
that were too general, and (3) examples or illustrations.
Several additional decision rules were added to reduce these differences: (1) simple agreement type responses did not count as ideas, (2) thoughts deemed too general were not counted, (3) statements with obvious but not explicit references were counted, and (4) explained ex- amples were counted but simple listings of examples were not.
RESULTS
Each of the hypotheses tested consisted of a compar- ison between two means or among several means and was tested by using the t-statistic (Kirk 1968). Figure 2 is a graphic summary of the data. Table 1 reports the mean number of ideas and the standard deviations for each experimental condition. Table 2 lists the contrast coefficients used to test each hypothesis, the observed difference between means, the critical t-values calcu- lated from these differences, and the probability of falsely rejecting the null hypothesis (Type I error). A contrast is simply the difference between two means.
Contrast coefficients multiply the means to obtain the differences and vary as a function of the specific hy- pothesis being tested. Additional information on the technical aspects of the test statistics is provided in the appendix.
Groups Compared with Individuals
For only those ideas relevant to the discussion topic, nominal groups of eight individual interviews generated
Figure 2
MEAN NUMBER OF RELEVANT IDEAS AS A FUNCTION OF GROUP SIZE
200 -
175 -
150 - c vA
4!
a0) o 0
E
c
la
r
2 Co I
'The composite reliability coefficient is given by CR = 2(.60)
CR =
1 + .60
where .60 is the proportion of statements upon which the editors agreed.
125 -
Individual Interviews
Individuals Alone
Focus Groups Unmoderated Groups 100 -
75 -
50 -
25 -
I I I
1 4 8
Group Size
"
Table 1
MEAN NUMBER OF TOTAL IDEAS
Group
size One Four Eight
Group
type Real Real Nominal Real Nominal
Acquaint- Acquain-
anceship Strangers Strangers Strangers Strangers tances Strangers
Moderator no yes no yes no yes no yes yes no yes
Cell number 1 2 3 4 5 6 7 8 9 10 11
Mean 24.2 27.7 63.5 78.0 100.7 112.2 109.7 114.7 88.0 165.5 199.7
Standard 9.1 8.5 20.2 24.4 15.6 30.4 48.8 14.6 32.2 27.4 9.0
deviation
Note: Four observations per cell.
significantly more ideas than eight-member focus groups. unmoderated condition (H1 32). The gap between indi- On average, nominal groups of eight members generated viduals and groups is wider at the level of eight members 85 more ideas than focus groups of eight members. than at four members but is not significant.
Therefore, H1 1, appears to be rejected. The same result Additional analyses were done on both four-person is noted from the analysis involving four and eight-person focus groups of strangers with the total individuals-individual interviews produced 34.2 more number of different ideas as the dependent measure.
ideas than four-member focus groups. These results are presented in Tables 3 and 4. The gap In the comparison of unmoderated discussion groups between focus groups and nominal groups of eight mem- with individuals working alone, the individuals appear bers (HI .,) decreases to 59.75 ideas when total ideas are to be more productive. Regardless of whether the com- used and the difference is significant. However, for four- parison is between eight-member groups (H1 21) or four- person groups (H1i12) the difference is not significant.
member groups (H1 22), individuals produce significantly The difference between nominal groups and focus groups
more ideas. as group size changes from four to eight (H1 31) does not
For moderated groups the difference between individ- achieve significance when the total number of different uals and groups of eight members is significantly greater ideas is used as the criterion.
than the difference between the individuals and groups A separate analysis of the judged quality of ideas was of four members (H, 31). This result is not found in the conducted. Table 5 shows the mean and standard devia-
Table 2
CONTRAST COEFFICIENTS, OBSERVED DIFFERENCES, AND CALCULATED t-VALUES FOR TESTS ON NUMBER OF RELEVANT IDEAS
Cell number Observed Observed
1 2 3 4 5 6 7 8 9 10 11 difference t-value Probability
Groups vs. individuals
H,i, 0 0 0 0 0 0 0 1 0 0 -1 -85.0 -5.03 <.001
Hl12 0 0 0 1 0 -1 0 0 0 0 0 -34.2 -2.03 .062
H, 21 0 0 0 0 0 0 1 0 0 -1 0 -55.7 -3.30 .007
H1.22 0 0 1 0 -1 0 0 0 0 0 0 -37.2 -2.20 .045
H, 31 0 0 0 -1 0 1 0 1 0 0 -1 -50.8 -2.12 .050
H1,32 0 0 -1 0 1 0 1 0 0 -1 0 -18.6 .78 .450
Moderator
H2 11 0 0 0 0 0 0 -1 1 0 0 0 5.0 .30 .770
H2.12 0 0 -1 1 0 0 0 0 0 0 0 14.5 .86 .410
H221 0 0 1 -1 0 0 -1 1 0 0 0 -9.5 -.40 .700
H231 0 0 0 0 0 0 -1 1 0 1 -1 -29.2 -1.22 .244
H232 0 0 -1 1 1 -1 0 0 0 0 0 3.0 .12 >.800
Group size
H311 0 0 0 -1 0 0 0 1 0 0 0 36.7 2.17 .047
H312 0 0 -1 0 0 0 1 0 0 0 0 46.2 2.73 .016
Acquaintanceship
H4.00 0 0 0 0 0 0 0 1 -1 0 0 26.7 1.59 .138
Note: The t-test for heterogeneous variances (Edwards 1960) was used with four observations per cell and 16 degrees of freedom.
Table 3
MEAN NUMBER OF DIFFERENT RELEVANT IDEAS Group
size One Four Eight
Group
type Real Real Nominal Real Nominal
Acquain- Acquain-
tanceship Strangers Strangers Strangers Strangers tances Strangers
Moderator no yes no yes no yes no yes yes no yes
Cell number 1 2 3 4 5 6 7 8 9 10 11
Mean 24.2 27.7 63.5 98.2 100.7 112.2 109.7 140.0 88.0 165.5 199.7
Standard
deviation 9.1 8.5 20.2 23.4 15.6 30.4 48.8 28.4 32.2 27.4 9.0
Table 4
CONTRAST COEFFICIENTS, OBSERVED DIFFERENCES, AND CALCULATED t-VALUES FOR TESTS ON TOTAL NUMBER OF IDEAS
Cell number Observed
1 2 3 4 5 6 7 8 9 10 11 differences t-value Probability
Groups vs. individuals
Hl111 0 0 0 0 0 0 0 1 0 0 -1 -59.7 -3.39 .005
Hi12 0 0 0 1 0 -1 0 0 0 0 0 -14.0 -.79 .454
H1I21 0 0 0 0 0 0 1 0 0 -1 0 -55.7 -3.30 .006
H1I22 0 0 1 0 -1 0 0 0 0 0 0 -37.2 -2.20 .045
Hi31 0 0 0 -1 0 1 0 1 0 0 - 1 -45.7 -1.83 .089
H132 0 0 -1 0 1 0 1 0 0 -1 0 -18.6 .78 .458
Moderator
H211 0 0 0 0 0 0 -1 1 0 0 0 30.2 1.72 .106
H2,12 0 0 -1 1 0 0 0 0 0 0 0 34.7 1.97 .070
H2.2 0 0 1 -1 0 0 -1 1 0 0 0 -4.5 -.18 >.800
H231 0 0 0 0 0 0 -1 1 0 1 -1 -4.0 -.16 >.800
H232 0 0 -1 1 1 -1 0 0 0 0 0 23.2 .93 .389
Group size
H31, 0 0 0 -1 0 0 0 1 0 0 0 41.7 2.37 .338
H312 0 0 -1 0 0 0 0 0 0 0 46.2 2.73 .016
Acquaintanceship
H4.0 0 0 0 0 0 0 0 1 - 1 0 0
aThe transcripts from focus groups of acquaintances were not reedited for the total number of ideas.
Table 5
MEAN AND STANDARD DEVIATIONS FOR THE QUALITY OF IDEAS
Summed quality "Good ideas"
Standard Standard
Condition Mean deviation Mean deviation
Unmoderated groups 193.5 92.7 29.0 16.4
Moderated groups 218.7 27.4 40.0 7.0
Individual interviews 367.2 25.5 62.5 11.1
tion of the summed quality of ideas and the number of
"good" ideas for groups of eight members. "Good"
ideas are those rated a 4 or 5 on a 5-point scale. Results of the analysis are reported in Table 6. Contrary to the prediction of HI 23, the summed quality of ideas from individual interviews is significantly higher than the summed quality of ideas from focus groups. Individual
interviews also account for significantly more "good"
ideas than focus groups.
Responding to a semantic differential questionnaire item with "conspicuous" and "anonymous" as an- chors, individuals interviewed alone reported feeling more anonymous (X = 5.69) than did individuals in fo- cus groups (X = 4.1). This finding seems to contradict
Table 6
COMPARISONS OF QUALITY OF IDEAS BETWEEN FOCUS GROUPS AND INDIVIDUAL INTERVIEWS
Cell
number
Dependent number Observed Proba-
variable 8 11 difference t-value bility Summed
quality 1 -1 -148.5 -7.9 .000
Good
ideas 1 -1 -22.5 -3.4 .014
the notion that individuals in focus groups use a "hide- in-the-crowd" strategy or become uninhibited because of anonymity. The same result is found in comparing four-person focus groups (X = 4.6) with individual in- terviews (X = 5.7). However, no difference in felt an- onymity is noted between four-person and eight-person focus groups. If individuals employ a "hide-in-the- crowd" strategy they might be expected to feel more anonymous in larger groups. Apparently this is not the case.
Because the interviewers and moderators were not the same people, a semantic differential format was used on the questionnaire to measure moderator-interviewer dif- ferences. The bipolar adjectives used were thought to measure characteristics desirable in choosing a moder- ator. No significant difference is found between how subjects felt about the moderator and how they felt about the interviewer (Table 7).
Moderated Groups Compared with Unmoderated Groups
Groups led by focus group moderators did not produce significantly more ideas than unmoderated discussion groups. The result is the same whether moderators led groups of four members (H2.12) or groups of eight mem- bers (H2 1). This comparison was also made for the total number of different ideas. Focus groups are superior by 34.7 ideas for four-person groups and 30.2 ideas for eight-person groups but these differences are not signif- icant at the .05 level.
With the judged quality of ideas as the dependent vari- able, focus groups are not significantly different from unmoderated groups (Table 8). Nor are focus groups
Table 7
DIFFERENCES IN AVERAGE RATINGS OF FEELINGS TOWARD MODERATORS AND INTERVIEWERS
2-tail Interviewer Moderator t-value prob.
Sensitive 5.87 5.59 -.57 .569
Sincere 6.56 6.00 -1.46 .150
Domineering 3.18 3.69 1.12 .267
Friendly 6.34 6.34 .00 1.000
Table 8
COMPARISONS OF QUALITY OF IDEAS BETWEEN FOCUS GROUPS AND UNMODERATED GROUPS
Cell
Dependent number Observed Proba-
variable 7 8 difference t-value bility Summed
quality -1 1 25.2 .54 .62
Good
ideas -1 1 11.0 1.23 .26
different in terms of number of good ideas.
Several items were included in the questionnaire to measure some dimensions of the atmosphere in focus groups which have been reported by moderators. Re- sponses on semantic differential items (Table 9) indicate focus group participants (1) found their task more ex- citing, (2) felt more enthusiastic in doing the task, and (3) found the task more enjoyable. They did not see their contribution to the task as being more spontaneous than that of unmoderated group members, however.
Group size is not a significant factor in determining focus group moderators' effectiveness. That is, moder- ators are no more or less effective in groups of eight members than in groups of four members (H221), re- gardless of whether relevant-unique ideas or total-unique ideas is the dependent variable. Therefore, H2.21 is not supported by the data.
Individual interviewers are no more effective than fo- cus group moderators in eliciting relevant-unique ideas (H231 and H232). Additional support for this result is found when the total number of unique ideas generated in focus group sessions is used in the comparison-no significant difference is detected. Failure to support the moderator effect hypothesis seems to suggest the re- ported difference between nominal groups and focus groups may be due to the inhibiting effects of group in- teraction and not moderator-interviewer differences.
Effects of Group Size
Contrary to the brainstorming reports and in support of focus group researchers, a significant group size ef- fect is found. Focus groups of eight members generated 36.7 more ideas than focus groups of four members.
Table 9
COMPARISON BETWEEN FOCUS GROUP AND UNMODERATED GROUP TASK RATINGS
Focus Unmoderated 2-tail
groups groups t-value prob.
Excitement 6.66 4.23 -3.52 .001
Enthusiasm 6.72 4.97 -2.47 .016
Spontaneity 7.56 6.77 -1.10 .278
Enjoyment 7.53 5.71 -2.48 .016
Additionally, unmoderated groups of eight members generated 46.2 more ideas than groups of four members.
These findings tend to support H3 1 and H312. However, in Table 1, eight-member groups do not produce twice as many ideas as four-member groups.
Additional analyses were done to see whether the data conform to the multiplication of impact principle from social impact theory (Latane and Nida 1980). Contrary to the Hess (1968) "synergism" hypothesis, social im- pact theory would have predicted diminishing returns as group size increased from one to eight members (I = N', where I = number of ideas generated, N = group size, and 0 < t < 1). To test this notion, the log transforma- tion of both the dependent variable (number of relevant- unique ideas) and the independent variable (group size) was subjected to regression analysis. If the slope of the linear function using log-log coordinates is greater than one (t > 1), the synergism hypothesis is supported; if less than one (t < 1), the diminishing returns hypothesis is supported. The resulting function is depicted in Figure 3. The exponent of the power function implied by these data is .7 with 86% of the variance explained (adjusted R2 = .86) by the group size factor. The exponent is sig- nificantly different from one (t = 3.50, p < .005). The exponential function derived from the regression anal- ysis is shown in Figure 4. The analysis appears to sup- port the diminishing returns hypothesis.
Acquaintanceship
The difference in number of relevant ideas between focus groups of strangers and focus groups of acquain- tances is 26.7 but is not significant. Because the tran- scripts from the acquaintanceship condition were not re- edited for the total number of ideas, no comparison was
Figure 3
NUMBER OF RELEVANT-UNIQUE IDEAS AS A FUNCTION OF GROUP SIZE
Figure 4
NUMBER OF RELEVANT-UNIQUE IDEAS AS A FUNCTION OF GROUP SIZE
120
100 .
v, aJ
-o 0
i'50-
E z
- - I = 27.2N7 ,o/
-
0o
4 8
Group Size
made between strangers and acquaintances on this vari- able. One confound should be reported. The number of ideas for acquaintances is slightly inflated. One focus group had been conducted using the set of instructions before the decision was made to not use instructions in focus group sessions. Therefore, this group had the whole time period (15-20 minutes more than the other groups) in which to discuss relevant ideas. This group generated 57 more ideas than the average of the other three groups in this condition. The data were reanalyzed with the outlying group removed. The t-test for unequal n showed focus groups of strangers generated signifi- cantly more ideas than focus groups of acquaintances (t = 2.25, p = .025). Because of the magnitude of the quality rating task and the amount of judging time re- quired, the quality of ideas generated by acquaintances was not evaluated.
/Log I = 3.3 + .7 Log N /1 -
? Gg Spa
Group Size (Log Spacing)
DISCUSSION
Several issues may have had an effect on the results.
One is whether the sample size was large enough to pro- vide a fair test of the hypotheses. Because the power of a statistical test depends on sample size among other things, one can reasonably argue that four observations may not have provided sufficient power to detect the moderated-unmoderated group differences. We had three primary reasons for using four groups per experimental cell. First and perhaps foremost was the cost of each observation. Rather than scale down the magnitude of the study to obtain an increase in the number of obser- vations, we decided to keep the sample at four groups per cell. This decision was influenced greatly by the small sample sizes used in the brainstorming and prob- lem-solving studies. The second reason was that the brainstorming studies achieved rather large treatment 5
7O
0O
(, 4
V) -e 4- 0 S.-
2. E z
/- 0 3
I~~~~~~~~~ I!I
1
