Chapter 5 Methodology Methodology
5.4. Pilot Study
A pilot study was conducted in this study, acting on the suggestion by Stopher (2012) that before a survey is committed to the field, the questionnaire and the various designs and procedures of the survey should be piloted. This is particularly pertinent, according to Clark-Carter (2010), when the researcher uses an existing questionnaire in a new population. Further supporting his advice, Stopher (2012) states that it has been proven time after time that surveys that are fielded without piloting result in problems that could have been avoided had pilot studies been conducted. Pilot studies can be used for various reasons, including:
1) To identify possible problems with the proposed research using a small sample of respondents before the main study is conducted (Kanjee, 1999). In this study, the main reason for conducting a pilot study was to ascertain whether the questions contained in the questionnaire made sense to participants. The wording of a questionnaire is very crucial in any survey (Rosnow & Rosenthal, 1996) and a pilot study was therefore used in this study to detect and correct ambiguities, and to check whether there were any words that needed to be simplified or adapted that could have been missed during the pre-testing of the questionnaire.
2) The researcher used the pilot to check the clarity of instructions and to determine whether any procedural difficulties were encountered during the administration of the questionnaire. This was done to ensure that all aspects in administering the questionnaire worked as planned or to ascertain whether there would be a need to refine the instrument and procedures before the main study was conducted.
3) Stopher (2012) adds that pilot studies may also be used to assess the response rates and completion rates of a survey. This was particularly relevant for this study because the researcher was concerned about the length of the questionnaire and the amount of time it would take to administer the questionnaire. Apart from establishing the average time it took to complete the questionnaire, the focus here was also on how many of the people who were approached to participate would decline permission to do so, or would not complete all the questions in the questionnaires, and what would be their reasons.
4) Lastly, according to Kanjee (1999), pilot studies are also useful to research assistants’ training. Given the nature and scope of this research, a total of five research assistants (graduates with psychology as a major and those who had previously participated in major research projects as research assistants) were recruited and trained to collect data for this study. In anticipation of the respondents being isiXhosa speakers in the Kokstad area, two of the five research assistants were fluent in speaking isiXhosa and the other three had a good command of the language. Piloting this study was useful in this regard because it afforded the research assistants an opportunity to become familiar with the procedures involved in and administration of the questionnaire. Attention was also paid to errors that could possibly be made by research assistants in their administration of the questionnaire. Research assistants were given notebooks and encouraged to write down questions and problems as they encountered them during the administration of the questionnaires. Debriefing meetings were held with research assistants at the end of each day of the pilot study to discuss their experiences and concerns. The pilot study was conducted for a period of one week in the month prior to data collection in the main study.
5.4.1. Pilot location.
The pilot sample was recruited from Greytown in the province of KwaZulu- Natal. Greytown is situated in Umzinyathi District, which falls under Umvoti Municipality. It is approximately 75 kilometres (kms) north of the capital city of KwaZulu-Natal, Pietermaritzburg. Greytown was selected because the researcher considered the town to have a population and economic characteristics somewhat similar to Greater Kokstad and Kwa Sani Municipalities. A meeting was held with both the offices of the municipal manager and the South African Police Service (SAPS) informing them of the pilot study, its aim and objectives as well as the presence of the researcher and research assistants in the community. This served as obtaining entry and obtaining gatekeepers’ permission in Greytown.
Both the municipal manager and SAPS attached their official stamps on copies of ethical clearance letters as an indication of their consent and approval for the pilot study to be conducted at Greytown. The letter showing these stamps is attached as Appendix E.
5.4.2. Pilot sample.
While there are no specific rules about the size of a pilot survey sample, Stopher (2012), and Czaja and Blair (2005) recommend that a relatively large number of participants should be recruited using the exact procedures planned for the study. Following on the suggestion made by Stopher (2012) that the sample size for a pilot survey should be in the range of 3 to 7 per cent (or more) of the main sample, 93 participants were recruited to participate in a pilot of this study. Steps followed to recruit pilot sample are similar to the ones used to recruit the sample for the main study and are detailed in section 5.6.2.2, under data collection.
Details of their demographic distribution show that of these 93 participants, 61 (65.6%) were females and 32 (34.4%) were males. Their ages ranged from 18 to 72 years, with 35.9 years being the mean age. Distribution according to age categories showed that there were 25 (55.9%) pilot participants between the ages of 18 – 34, 22 (23.7%) between the ages of 35 – 49, 14 (15.1%) between 50 – 64, and 5 (5.4%) in the category of 65 and above. Regarding their marital status, 68 (73.1%) were single, 18 (19.4%) married, 4 (4.3%) widowed and 3 (3.2%) were living with a partner. Distribution according to educational level shows that 5 (5.4%) of the pilot sample had never been to school, 11 (11.8%) had a primary level of education, 62 (66.7%) and 15 (16.1%) had attained secondary and tertiary levels of education, respectively. Regarding religion, 80 (86.0%) of the pilot sample were Christians, 4 (4.3%) Jehovah’s Witness, 1 (1.1%) Muslim, 6 (6.5%) African Religion, and 2 (2.2%) reported to be non-religious. 90 (96.1%) of the participants in the pilot sample speak isiZulu as their home language, and only 1 (1.1%) recorded Afrikaans, Sepedi, or Other, each, as their home language. Table 5.2 shows a summary of these results.
Data collected from the pilot study is not included in the results of this study because the aim of the pilot study was only to test methodologies for data collection.
Table 5.2: Demographic information of the pilot sample
5.4.3. Procedural difficulties encountered.
In general, not many procedural difficulties were encountered when piloting this study. The only difficulties experienced were related to: 1) the time it took to complete the questionnaire; 2) the high number of spoilt questionnaires;
and 3) the inclusion of non-African participants. It was initially stated in the information sheet of the pilot study that it would take 20 to 30 minutes to complete
Demographics n %
Gender
Female 61 65.6
Male 32 43.4
Age
18 – 34 52 55.9
35 – 49 22 23.7
50 – 64 14 15.1
65+ 5 5.4.
Marital status
Single 68 73.1
Married 18 19.4
Widowed 4 4.3
Living with a partner 3 3.2
Education level
Never went to school 5 5.4
Primary 11 11.8
Secondary 62 66.7
Tertiary 15 16.1
Religion
Christian 80 86.0
Jehovah’s Witness 4 4.3
Muslim 1 1.1
African Religion 6 6.5
Non-Religious 2 2.2
Home language
Afrikaans 1 1.1
isiZulu 90 96.8
Sepedi 1 1.1
Other 1 1.1
the questionnaire. However, it became apparent that to complete the questionnaire took a greater amount of time than initially thought. Most participants showed some level of uneasiness with the time spent to complete the questionnaire because they had been informed that only 30 minutes of their time was required for completion. Such participants were reminded that they were not required to complete answering the questionnaire, and some withdrew, but others went ahead to complete the questionnaire. Research assistants were each instructed to observe how much time, on average, it took to complete the questionnaire. A general impression was that the average time required to complete the questionnaire was 45 minutes.
The researcher checked the questionnaires completed at the end of each day with a focus to determine how many of them would be usable. About 10 questionnaires were spoilt on the first day of the pilot study. Many of these questionnaires were incomplete and some were answered without following instructions, for example, some participants selected more than one answer on several occasions where only one answer was required. To deal with this problem, the research assistants were instructed to be present and to monitor the participants to ensure that they followed the instructions when completing the questionnaire.
There were about three questionnaires that were completed by non- African participants, a further two were completed by Indian participants and one by a coloured participant. Reminding research assistants that the targeted sample of this study were ‘black Africans’ only became a standard part of the briefing every morning before the start of data collection.
5.4.4. Adaptation.
Adaptations to the questionnaire or procedure of administering the questionnaire included the following issues. Firstly, the time it took to complete the questionnaire on the information sheet had to be changed from 20 – 30 minutes to 45 minutes. This was helpful because it prepared participants for the realistic amount of time it would take for them to complete the questionnaire. This improved the rate of completed questionnaires and decreased spoilt questionnaires. Secondly, it was only noticed after the pilot study when data was captured that the category of ‘divorced’ was not included as an option under the
demographic information eliciting marital status of participants. This category was therefore added in the later version of the questionnaire used in this study.
5.4.5. What was learnt from the pilot study?
There are a number of important lessons that emerged from the pilot study.
Apart from the adaptations made to the questionnaire as stated above, the pilot study proved important in terms of the significance of thorough training of the research assistants, teamwork, and short briefing meetings every morning prior to proceeding with data collection. The importance of establishing rapport with the research participants was also learnt from the pilot study. Research assistants noticed that participants were more likely to answer the questionnaire in full when a good rapport was established, when participants understood what the study is all about and when the participants were informed about how important it would be for their ideas to be included in the study, as well as when they were informed from the beginning that close to an hour of their time was required to complete the questionnaire.
5.5. Reliability and Validity of the Study