Leading questions must be avoided
Ensure that children are given the „don‟t know‟ option in order to avoid guessing
Interviewing children on their home territory is ideal.
3.6.2 Measures used
3.6.2.1 Self-report questionnaire
Two age-based anonymous self-report questionnaires were designed by the researcher, due to copyright and financial constraints (See Appendix B and C). A combination of ideas from Olweus‟ studies on bullying (1993), Kohlberg‟s (1981) moral dilemmas and Selman‟s (1980) social perspective-taking vignettes were used, based on the theory that social perspective-taking is likely to influence moral
reasoning and thereby bullying behaviour. The questions were designed after a thorough review of the literature on what kinds of inquiry would elicit appropriate answers to the research questions. The first few questions were based on pictorial vignettes and briefly described scenarios in order to evoke an emotional (and hopefully more honest) response from the participants. They were open-ended and intended to reveal the emotions projected by the participants onto the characters in the picture. Such questions were balanced with closed-ended ones, in order to minimize analytical confusion. A rating scale section was also included in order to allow quantification of certain bully/victim experiences.
In designing the questionnaire the following were considered important (Davies, 2007):
Each question was essential and constructed to be comprehensible and well thought out
Ambiguous words and leading questions were avoided
Words that were familiar and appropriate to the sample were used (two different questionnaires were accordingly designed to cater for the wide range in ages). Such words were also considered to have the same meaning for both the researcher and the participants.
The length of the questionnaires was adapted to the age groups, being slightly shorter for the younger children.
In closed-ended questions, all potential responses were considered.
The scaled questions used words (sometimes, often, etc.) in order to aid understanding for the participants and to allow quantification of some of the findings.
Questions around sensitive areas were designed to minimize possible negative reactions from the participants (for example, by using projective techniques).
3.6.2.2 Validity and reliability
There are certain disadvantages inherent in using a questionnaire format (Owens, Daly & Slee, 2005): Participants may limit their responses to those which are socially desirable; they may feel uncomfortable with self-reporting and problems with
interpretation may occur. More specifically in the case of pen and paper questionnaires: the participants may not be familiar with questionnaires (test-
wiseness); forced choice answers usually give no chance for the participant‟s reasons behind their response or may not cover the answer the respondent would like to give;
and finally, errors in marking the incorrect choice through carelessness are more possible.
The analysis was not based on an existing test instrument and therefore the validity and reliability of the questionnaire was tested by means of a pilot study before the large-scale data collection commenced. The questionnaire was administered to a few girls from each age-level, and their answers were examined in order to ensure that the questions were measuring the correct construct, that they were understood by the participants and that they were capable of answering the research questions. The pilot study also revealed whether the questionnaires were an appropriate length, as
especially in the case of the younger children a lengthy questionnaire may lead to a lack of concentration towards the end, therefore resulting in hasty, invalid answers. If the pilot study had highlighted any potential problems in the methods of data
collection or the instruments, these could be attended to before the full-scale study commenced. No problems, however, were encountered.
Content validity can be deemed to be reasonable if the research adequately samples and represents the phenomenon being investigated. Face validity relates to whether or not the research appears to measure the appropriate domain. Construct validity
requires the theoretical soundness of the results in order that they can be generalizable to other settings. The issues around the validity of this questionnaire will be discussed in detail in Chapter Four.
3.6.2.3 Inter-rater reliability for codes
Another researcher also coded the raw data to determine if the final codes developed by the researcher satisfied the criteria for individual reliability. Cohen‟s Kappa was utilized to determine inter-rater reliability. This process is reported in the next chapter.
3.6.3 Administration procedure
All the participants were briefed before the questionnaire was administered. The following issues were clarified with the participants at this stage of the data-gathering:
1) The definition of „bullying‟ was discussed, in order to ensure that all the participants had a common understanding of the concepts involved. The concepts involved in both physical and non-physical bullying were included in the definition to ensure that participants had clarity, and an awareness of the range of actions which could fall under the topic.
2) It was emphasized that the answers should be from their own perspective, and that there were no „right‟ or „wrong‟ answers.
3) The participants were assured of anonymity and confidentiality with regard to their answers
4) The participants were requested to try and answer all of the questions, so as to ensure that the questionnaire would not have too many missing values which would invalidate it. However, they were also reminded that participation was voluntary, and that they were not forced to answer any question they found too distressing.
5) The participants were assured that the researcher could be approached at any time during the process for clarity or to answer any questions regarding their understanding of the questionnaire items.
6) The participants were encouraged to approach the researcher should they have difficulties with regard to being bullied, with the understanding that they would be referred to either the school psychologist or another mental health professional for ongoing support.
After the briefing, pen-and-paper questionnaires were administered to older girls (Grades Five to Twelve). In the case of the younger participants (Grades One to Four) who might be less able to reliably complete a questionnaire independently, the
researcher conducted individual briefings followed by structured interviews.