REVIEW OF THE LITERATURE
CHAPTER 4 RESEARCH METHODOLOGY
4.4. Data Collection
4.4.4. Bias in a questionnaire
The questionnaire should provide the researcher with responses as impartial as possible in order to ensure objectivity. In most of the cases, measurement contains a degree of measurement error. Two main forms of measurement errors are noise and bias. Both should be avoided.
Noise refers to an error in the measurement process that is non-patterned thus “there is no set direction to the error involved in the measurement process”
(Ruane, 2005). This means that the measurement could either surpass or underestimate the true value.
On the other hand, bias refers to a patterned error in the measurement process where according to Ruane (2005), “the mistakes are consistently in one direction or the other and the error may be consistently overestimating or consistently underestimating a true value”.
Chapter 4 – Research Methodology
130 In any case, the researcher should set the framework of the research methods in order to avoid –where possible- noise and bias in the analysis of the responses. The quality and the validity of the responses and at the same time the methods of analysis will be mainly affected by the way that a question is structured and the type of answer required. The most usual types of responses are the following:
• Open ended- where the interviewee is asked to describe their opinion without any indications or the restriction of lack of space.
• Category– where the interviewee is provided with a list of possible responses, such as: “What is the most important for you? :
Option 1
Option 2
Option 3
Option 4
• List- where the interviewee may select more than one from a list of responses.
• Ranking- where the respondent is asked to rank responses in order e.g. Put in order of importance the following cases (1 indicating the most important, 2 the next most important etc).
• Scale-Rating, for example: 1- Strongly Agree, 2- Agree, 3- Neutral, 4- Disagree, 5- Strongly Disagree.
• Forced choice, where the respondent is forced to make a choice between alternatives, for example: Yes/No. (Gray, 2004 and Gendall et.al., 1991) However, open-ended questions should be neutral in order to minimize bias into the findings of the research. Non-neutral questions may lead the respondents to answer in a different way regardless of how they think. In addition, since these questions allow the participants to respond in whatever way they choose, data are likely to be wide-ranging, complex and lengthy and for this reason could lead to bias, as the quality of the response depends on the subjective way that the researcher interprets the answer (Given, 2008).
In general, the lower the response rate in a survey, the higher the possibility of sampling bias since there is the risk of lower accuracy in the responses. For this reason, the following elements should be considered in designing the research in order to increase response rate:
131
• Send a compelling cover letter with the questionnaire explaining the purpose and the importance of the research.
• Formulate short and concise questions.
• Adopt an appealing layout for the questionnaire.
• Adopt a coherent organization of the questions.
• Judicious use of open ended questions.
• Provide clear instructions and advices on how each question should be completed.
• Keep contact with the delayed respondents by e-mail or telephone.
• Conduct a pilot survey, sending the questionnaire to a smaller sample before sending it to the final group of respondents, in order to test it and reveal its weaknesses. (Ruane, 2005 and Gray, 2004)
In addition to the above, Dillman (2007) cites a number of important elements that should be considered in the design of a questionnaire in order to increase the response rate. The researcher should try to:
• Make the questionnaire interesting and respondent – friendly e.g. including questions that are clear and easy to comprehend.
• Avoid embarrassment.
• Make questionnaires appear short and easy.
• Minimize requests to obtain personal information.
• Make the task appear important.
• In case of mail surveys, the researcher could include return envelopes with real first-class stamps.
After gathering the completed questionnaires, the data should be analysed using a specific method or a combination of methods in order to answer the research questions. On the one hand, it is easier for closed questions to be coded and analysed via software (e.g. SPSS) since a prompt list of possible answers is used.
On the other hand, the coding process for open questions is more complicated and time consuming, since the questions need to be coded into a number of categories depending on how the respondents answer and what answers are being looked for (Brace, 2004). In general terms, the answers are going to be separated or categorized into a smaller number of groups determined by the researcher, in order to facilitate the analytical process. However, according to Reja et.al. (2003) open – ended questions have the disadvantage of producing more missing data compared to close-ended questions.
Chapter 4 – Research Methodology
132 4.4.5. The Pilot Study
A good practice in order to evaluate the appropriateness of a questionnaire is to conduct a pilot study (Brace, 2004).
In this project, each one of the following four different questionnaires:
Suppliers, Packaging Manufacturers, Industrial Customers, Wholesalers and Retailers, were piloted with four business managers, one for each type. For Final Consumers, 10 questionnaires -approximately 5% of the total sample-were piloted.
The questionnaires were tested in terms of wording, clarity, style, content, time needed to be completed and structure of questions. Based on the feedback obtained, the questionnaires were restructured and improved in order to facilitate and effectively improve the data collection process. Where needed the questions were edited and some open questions were replaced by closed questions, in order to reduce the time needed for the respondents to answer and to avoid ambiguous meanings. In some questions, further instructions were provided because the wording was found to be too complicated for the respondents to answer in a meaningful way.