In purely logical terms, it is proper that the method to be used is decided last, as it is only after working out what is going to be addressed and
among whom that it makes any sense at all to decide upon the method.
The reason that the logical approach is often not followed will be addressed later in this chapter.
The main elements of a research design will now be taken in turn, but it must be said in advance that the biggest issue of all is correctly identifying the problem that is to be addressed, and the reasons for this difficulty will be discussed in detail.
Problem definition
The single most critical feature of a research design is to discover what the problem is – a better brief means better research (Pyke, 2000). Only in this way may the research be reduced to a series of objectives – or desired outcomes – around which the research may be designed.
In many ways it seems absurd to state that the setting of the objec- tives is so critical, but the fact that there is confusion over why the research is being done is the major reason for research failing. When the client believes one thing and the researcher another, then this is clearly a recipe for disaster and disappointment. So why can this happen?
The problem holder often does not know what the problem is On the whole, marketing people are not very good at expressing the problem in researchable ways (they tend to be ‘doing’ people in line with the mainstream of business employees), and it is necessary almost to interview them in depth to find out what the problem really is, and to get it couched in terms that are amenable to research.
For example, a brand manager has the idea that it would be ‘good’
to create a new variant of the brand and wants to know the scale of the opportunity for it. From this, the impact on production and the possible magnitude of the advertising spend can be assessed. This would sug- gest some form of quantitative concept test. But what exactly is the variant to be? How does it fit with other things that are happening on the market? Is the brand able to credibly stretch to encompass the vari- ant? There are many questions like this that are obviously the meat of qualitative research, and their answers could radically colour the deci- sion whether to make the variant at all. The discussion with the brand manager might miss out entirely worries that others in the company have, because he or she is too focused.
The market assumptions radically affect the design
Another typical problem could be that ‘We are selling less of this stuff and want to do some research to find out why so that we know how to sell more.’ There are fairly obvious possible causes for this, which are very easy to brainstorm. For example, is it price, distribution, com- petitor activity, the product proposition, the market and so on? To address all the potential reasons for the problem existing would require a very large-scale and probably multi-staged approach, which would be very expensive, take too much time, and is in fact generally not necessary.
The client will have some ideas about what the problem is, and these ideas can be used to reduce the scale of the research. Therefore, in the briefing meeting there is a need to identify as much as possible what the hypotheses of the organization are as to the poor performance of the product, and to use these in creating the design.
The researcher will attempt to find these out through talking to the client, and in doing so will sometimes have to use all his or her skill as an interviewer. Often this can become a process of negotiation, and has as much to do with the client and his or her experience (or its lack) as it does with the problem.
In this process lies a deep problem, in that the assumptions being taken into the research can radically impact on the outcome. If a given area is not deemed to be important, and is therefore not addressed in the research design, then the research will obviously not uncover it – even if it were to be the critical factor. There are some who believe that the need to produce ‘efficient’ designs is strongly distorting the use of the research in that the outcome effectively becomes ordained (Gordon, 1999).
So getting the balance right about the assumptions is important, and worthy of challenging. It is also sensible to include the assumptions that are being worked on in the research design documentation, so that over the course of time it is apparent why a particular subgroup or area of investigation (which with the benefit of hindsight was clearly of importance) was excluded from that particular piece of research.
Different clients see the world differently
The intrinsic ambiguity of problem definition is made worse because different people may see the problem in a different way. Typically this
could be within the organization’s hierarchy, when it is clear whose opinion counts most, but it can also be the case that a problem has a number of different stakeholders (see Chapter 1): for example, the advertising agency may have a different view from the manufacturing client for whom it works.
In the example given earlier in this section about the new variant, there may be those in the company who are concerned about the pos- sible cannibalization of the current brand by its variant, with no real increase in sales but at greater costs associated with managing two variants rather than one. Knowing that this is a concern of one of the stakeholders could put a very different spin on the resulting design.
This means that a briefing given by one person will often lead to a different view about the ‘problem’ and lead to a different research design than would have been the case had the briefing come from another person in the same organization.
A junior does the briefing
A common situation is that the client who is briefing the researcher is actually not the problem holder but is a junior who has been told to ‘go and get some research’. Under these circumstances the real problem holder does not have direct contact with the researcher, and there may be multiple links in the chain, with an abundant opportunity for the message to get confused and weakened. The junior manager probably thinks he or she understands the problem, might also have his or her own ideas (which may or may not be helpful) and may be quite forceful about this (marketing people are like that!), so it is very difficult for the researcher, when presented with such a person, to appreciate exactly how ignorant of the problem he or she really is.
An example of this was when a chief executive of a company, which was in the process of buying a major retail chain in the United States, speculated out loud about the relative consumer awareness of the com- peting chains. Within a day this was translated into a research request to conduct an awareness study in every state in the United States. This was said to have had the backing of the chief executive, and therefore by implication, money was no object. Fortunately, in this case the request did not go directly to a research agency. If it had, it would have been quite impossible for it to have discovered that the chief executive in question had no recollection of even thinking about awareness, let alone asking for some research on it. In this case the request had gone
to the internal research department, which was able to find out the truth. The role of the internal research department has been discussed in some detail in Chapter 4.
Remote briefing
Since getting to the problem can be very difficult at the best of times, there is clearly a major problem when the client is unprepared to spend time with the researcher to go through these preliminary stages.
Unfortunately this is not uncommon, and it is not that unusual for a brief to arrive in the form of a two-line fax: for example, ‘some groups are needed on a new packaging’! After this the client becomes impossible to contact, never being available and never returning calls. In-house researchers who are preparing a briefing for an outside qualitative researcher may be briefed on the way to lunch, at the end of a meeting on another topic or by similar limited commu- nications. It is under these circumstances that continuity of supply becomes a great advantage, either between the in-house researcher and his or her internal end-user clients, or though the direct relationship that a researcher builds up with the end-user, so that it is possible to devel- op some approximation to the telepathic skills needed to deal with clients like this.
The situation changes by magic
Telepathic skills are very useful to a researcher when dealing with a client.
Once a brief has been received and the best possible attempt had been made at getting it into an agreed form, in terms of setting the research objectives, it is possible that it will change. Such is the way of the world, that it is not uncommon for the researcher to be expected to know this by telepathy. This happens as much between internal researchers and their internal clients as it does for direct researcher–end-user relationships, and there is really not much that can be done about it other than to cope as best as possible when the changes are discovered – which could actually be in the debrief! So it really is important for there to be constant com- munication between the client and the agency.
The sample definition
One of the most important features of a research design is the defini-
tion of the sample of respondents to be used. It goes without saying that the information collected will depend upon the experience and orientation of the people being questioned.
The whole issue of the sample relates to the concept of the research franchisedeveloped in Chapter 1, which in turn relates to the varying perspectives of the research stakeholders. Consequently, the broad def- inition of the extent of the sample (or more correctly, samples) derives directly from the research objectives as viewed by different stakeholders.
Once this has been decided on, there is then a need to specify in more detail the nature of the various samples. This is done in completely different ways for quantitative research and qualitative research.
In quantitative research, the sample needs to be specified in such a way that the people questioned are representativeof something – possibly the population in general (as is the case in most omnibus work) or the participants in some ‘market’ or other (if we take a broad definition as to what a ‘market’ is). ‘Representativeness’ is important to quantitative researchers who are counting responses, and is a major way of achiev- ing reliability (that is, repeatability). Representativeness means not only that the right people should be in the sample, but that they should be there in the right proportions, or if not, the deviation from this propor- tion should be known so that corrections can be applied later at the analysis stage. So far this is easy to understand, but it is not so easy in the case of qualitative research.
In qualitative research there is no need for the sample to be construct- ed in this way, and in general this would be entirely counter-productive.
The need is to have representatives of different viewspresent so that these can be understood and taken into account. Many products that are the subject of qualitative research are mass-market products, and there is a tendency to spread the constituents of the sample through the demo- graphic, regional and possibly usage base of the market. However, it is very rare that the differences between these people are sufficiently great to merit separate reporting, and their presence simply helps to add to the overall richness of the information being collected. The qualitative researcher’s job therefore becomes one of putting together the story from the many fragments with which he or she is presented by the individu- al respondents. It should never be forgotten that much of what is researched is not something that people have spent much time, or any time for that matter, articulating themselves, so it is unlikely that anyone will have the whole story.
However, this situation is not always the case, and the structure of the sample can become an important part of the data collection itself.
A common divide is to compare users with lapsed users, in terms of what type of people they are, their relationship to the market and the needs that they are seeking to be addressed. Alternatively, one might want to look at committed users of different brands in the market in order to understand the brand positionsbetter, or select people who use the product in a different way, in order to examine the needsin the mar- ket. Occasionally one might want to counterpoint the views of people who express a different attitude to the products or the market, for example, those who are price sensitive as opposed to those who are not.
When a sample is constructed in this way, then the sample design is absolutely critical to the planned method of analysis, and possibly even the reporting of the results. This is an example of the ‘all at oneness’ of qualitative research, where all aspects of it seem to be coincident.
The use of the term ‘representative’ therefore becomes quite different when applied to qualitative research or to quantitative research, and when used in a survey it should be considered in terms of the specified sample design, not of the market in general.
Sample size
In quantitative research, the sample size is supposed to be determined by the statistical significance of any difference that it is hoped will be discovered. A given numerical difference can be attributed a ‘signifi- cance’, that is, the probability of it being ‘true’ (or more properly, the
‘null’ hypotheses being false), and this depends upon the variation in the base data and the number of measures of it (that is, respondents).
Consequently, the sample size should be selected against assump- tions about these two expectations. In reality sample sizes are rarely arrived at in this way, and are more to do with the credibility associated with the size of any subsamples that will be analysed and the cost of the research.
The way that the numbers of interviews are worked out for a quali- tative sample does have a parallel to that of a quantitative survey, but not much. The issue at stake is how similar the views of the respon- dents are to the main issues. If they speak with one voice, then in all practical respects one only needs to talk to one of them to have all the information. If the segmentation of views is categorical, for example all
brand users say one thing and non-brand users say another, then again in principle, only one respondent from each category would be needed – as good quality findingswould be obtained in such a hypothetical case as if a thousand people had been interviewed (see Chapter 5).
Of course, the majority of topics that are subjected to commercial qualitative research are not burning issues in the minds of the respon- dents, and help is needed to get them to articulate ‘good stuff’.
However, even under the most skilled interviewing, in most cases it seems that only bits of the picture emerge from the various interviewees and it is from these bits that the whole must be synthesized.
Consequently, even under these circumstances, it is necessary to have a range of people. Sometimes, if the researcher is lucky, there will be one person among them who enables the researcher to make sense of all these pieces and allow the jigsaw to be put together.
Sample size and sample segmentation go hand in hand in the quali- tative research designs for good reasons. Not only does a quality and sensible segmentation of respondents help with obtaining a clear analy- sis – because different views become concentrated into different groups – but also it can actually reduce the total number of respondents needed because of this greater clarity. In the quantitative case, segmentation always increases the numbers of respondents needed, as they need to be identified in a sub-break at credible analytical numbers.
Coverage
A fairly obvious part of the research design is the questions to be asked.
However, the notion of ‘question’ is very much one that comes from the quantitative research paradigm, so I am using the term ‘coverage’ as a more general term applicable to both quantitative and qualitative research.
Clients tend only to think about ‘questions’ and ‘issues’ and are inclined to confuse the businessquestions that are to be answered by the research with the actual questions that are to be asked to get the information required from the respondents.
Obviously, a number of business questions can be answered by directly asking the respondent. ‘What don’t you like about this prod- uct?’ is an example. But other business questions, especially those related to the question ‘Why?’ – essentially motivations and needs – cannot be understood by direct questions. Consequently, indirect approaches need to be adopted.
In the case of quantitative research, questions need to be put together so that an analysis can be done. Often this involves ques- tions that enable the respondent to be characterized in a way that is helpful to the analysis. The most obvious of these is demographics.
But there are generally more powerful ways of contrasting respon- dent views and thereby gaining insight. For example, usage and other behavioural questions, although having obvious direct appli- cability, are very powerful at doing this. An example relevant to the leisure market is that whothe respondents are withwill have an effect on what sort of things that they do – a group of girls on a night out is different from a group of boys, and both are different when they mix. A considerable part of the skill in achieving a good quantitative analysis is in identifying the questions to be used to segment the sample.
Mathematical techniques can be used to extract information from the respondents. Examples of these are trade-off or multivariate meth- ods that impute levels of importance – so-called ‘latent methods’ – and these require particular schemes of questioning, the use of which may not be obviously apparent to the client. Generally speaking, clients are not good at understanding why questions of this type (that are present to enable analysis rather than directly generate informa- tion that has obvious utility) are included, and it may require some debate before they allow their inclusion.
As already mentioned, the notion of ‘question’ does not have much meaning in qualitative research, in that questions, such that they occur, are generally aimed at getting the respondents to start talking, and thereby start revealing their relationship to the topic in hand.
Normally the coverage of qualitative research is communicated through the medium of the ‘interview guide’. This is really a loose plan of action, and lays out which particular areas will be explored, in which order and in what way. These areas are themselves selected as being appropriate in order to elicit information that will help towards achieving the objectives of the research – that is, answer the client’s questions. Qualitative researchers will often use some form of enabling or projective technique with their respondents to help them access and express their thoughts. This is not intended to deceivethe respondents, but to help them marshal their thoughts in order to allow them to give information on a topic they will normally have thought very little about.