EVALUATION
3. RESEARCH DESIGN AND METHODOLOGY
3.5 THE QUESTIONNAIRE
3.5.1 CONSTRUCTION AND DESIGN OF THE QUESTIONNAIRE
In constructing the questionnaire the researcher was mindful of the literature review which seems to be highly critical of the dualistic nature of performance evaluation. On one hand, teacher evaluation systems have an employee bias and emphasize professional development: on the other hand, performance evaluation systems in the public sector under the immense influence of NPM with its emphasis on ‘value for money’ based on a market capitalism, and a style of management known as new managerialism stress accountability in the public service. Thus the central aim of this thesis is to explore the tension between accountability and professional development.
Pansegrouw (1982) states that performance appraisal is a single event with two objectives, namely, performance evaluation and coaching and development.
There are inherent problems that can be encountered when the evaluator is both ‘judge’
and ‘counselor’. To throw some light on the inherent problems associated with the IQMS, the questionnaire had to be designed in a specific way so has to solicit a response on the tension that could arise between accountability and professional development in teacher evaluations and obtain from educators their perceptions of what they would regard as the true purpose of teacher evaluation.
The appearance of the questionnaire is important. It must look interesting and
uncomplicated. A poor questionnaire may result in low response rates, unreliable information or invalid data, or inadequate and inappropriate information. Quality questionnaires require well-worded questions, clear responses and attractive layouts (O’Sullivan and Rassel 1999: 230; Verma and Mallick 1999: 12).
The researcher has endeavoured to design an instrument characterized by clarity of wording and simplicity of design. Emboldening will also be used to draw the respondents notice to significant features. Further attempts have been made to make the instructions as unambiguous as possible so that respondents know exactly what is required of them.
The researcher found it necessary, in the interest of clarity and logic to break down the questionnaire into subsections with section headings. The practice of sectionalizing and sub-lettering is a technique used for grouping together questions to do with a specific issue (Cohen, Manion and Morrison 2001: 259).
Finally, the researcher arranged the questionnaire in such a way as to maximize co- operation by including questions of general interest. Cohen, Manion and Morrison state:
Attitude questions should be dispersed throughout the schedule to allow respondents to air their views rather than merely describe their behaviour. Such questions relieve boredom and
frustration as well as provide valuable information in the process (2001: 259).
The questionnaire employed in this study uses both closed-ended questions and open- ended questions. Closed questions prescribe the range of responses from which the respondent chooses. They are quick to complete and straightforward to code using
computer analysis. Open questions, on the other hand, enable respondents to write ‘a free response, to explain and qualify their responses and avoid the limitations of preset
categories of responses’ (Wilson and McLean 1994: 21). However, it is the open-ended questions that are of vital importance as they would be a means to the respondent to voice his/her opinion on the chief concerns of this study which debates the issue of whether the IQMS is an accountability tool used to police teachers or if it is as envisaged, an instrument for professional development.
The IQMS, as is any scheme which attempts to evaluate the work of professionals, is a sensitive topic, therefore the designer of the questionnaire had to take this into
consideration in terms of how the respondents may respond to the questions asked.
Therefore, the sequencing of the questions was important as early questions had to set the tone and the mindset of the respondents to the later questions.
Hence the sequence of this questionnaire is as follows:
• Commences with unthreatening factual information which will give the researcher nominal data,
• Moves to closed questions,
• To move to open-ended questions that seek responses on attitudes, opinions, perceptions and views together with reason for the responses given
(Cohen, Manion, Morrison 2001: 257)
With this in mind the questionnaire was divided into the following sections:
Section A: Biographical and General Information Section B: Performance Evaluation
Section C: Professional development Section D: Accountability
Section E: General Comments
The questionnaire is designed to elicit responses to the critical questions in this study.
The questionnaire uses questions requiring a yes/no response especially with regard to personal attributes, for example, ‘gender’, and this was processed using the chi-square statistic. Multiple choice questions were used where the range of responses could be used to capture the likely range of responses to given statements. Anchor statements were provided to allow a degree of discrimination in response. The chi-square statistic and cross-tabulations were used to interpret data. Section A-E of the questionnaire has questions which also require a categorical response, rank order response or a scaled response (Burgess 2001: 45).
Rating scales are widely used in research as they combine the opportunity for flexible response with the ability to determine frequencies, correlations and other forms of quantitative analysis. They afford the researcher the freedom to fuse measurement with
opinion, quantity and quality. For instance, a Likert scale provides a range of responses to a given question or statement for example:
Statement 1 = strongly disagree
2 =disagree
3 =neither agree or disagree 4 =agree
5 =strongly agree
The Likert scale is easy to construct and simple to use and measures the strength and direction of an individual’s response. However, its major disadvantage ‘is its inability to explain a single score’ (Zikmund 1994: 372).
The open-ended questions invite a personal, honest comment from the respondent in addition to ticking numbers and boxes. The researcher hopes it will contain the ‘gems’ of information that otherwise would not have been caught in the questionnaires which , essentially seeks to elicit quantifiable information. The limitation of the open-ended question is that it carries with it the problem of interpretation. However, it is also presents a window of opportunity to shed light on the issue of whether the IQMS has been used at school level to ensure achieving minimum standards in teaching or is it an evaluation procedure which can shed light for the teacher on areas which can be improved or developed. It would also centre on the debate in this study regarding whether the IQMS’
summative and accountability demands can be reconciled successfully with its formative and professional development functions.