Chapter 4: Conduct of the Study
4.6 Analysis of the data
Of the eight cases where the questionnaire was administered four were completed and returned immediately, two were posted to South Africa at a later date and two were not returned. Thus the total number of questionnaires completed was six.
from that intended by the respondent. This highlights one of the weakness of questionnaires and a comparative strength of interviews because in the interview situation where the researcher is faced with an inadequate response there is the opportunity to probe for deeper meaning to seek clarity.
In the process of gathering and subsequently transcribing the data, in particular that from the interviews, the main contours of the responses had begun to be revealed in the form of central themes. This corresponds to the stage in analysis that Cohen et al (2000: 282) refer to as 'generating natural units of meaning'. These themes were explored in more detail in a step-by- step process of analysis of the transcriptions. In the first stage the responses correlating to each theme were coded and nine predominant clusters of issues were identified. Within each of the nine clusters patterns of responses associated with principles, policy and practice were also identified The most common cluster, and that about which respondents had the most views and opinions to express, was related to the underlying principles, and the technical and structural aspects of modularisation. The data was associated with issues such as modules and programmes, choice, and flexibility.
The remaining eight clusters were broadly identified as being: assessment;
credit; awards; student counselling and guidance; management;
administration; semesterisation and change. In both the stages of interpretation and reporting on the findings it has been necessary to focus only on one main cluster because an analysis of the remaining eight is beyond the bounds of this study. However, the importance of interrelationships between these issues and their influences in the processes of decision-making within the development of a modular system cannot be underestimated. Therefore, a brief discussion of each cluster is offered in Chapter 7.
The unstructured interviews in particular had generated a greater magnitude of data than the scope of the study required. Therefore, the next stage was to extract from the wealth of data that which corresponded to the themes on the questionnaire and ultimately to the research questions. This process appears
to have commonality with what Hycner (1985, cited in Cohen et aI, 2000) identifies as the activity where the researcher, having noted the units of general meaning, further reduces them by 'delineating units of meaning relevant to the research questions'. The sequence of questions in the questionnaire became the framework for the deeper analysis and interpretation of the data which is presented in Chapter 5.
During the process of transcription and analysis of the data the potential pitfalls described in the literature took on new meaning. A crucial point is raised by Cohen et al (2000: 282) who say that:
In qualitative data the data analysis is almost inevitably interpretive, hence the data analysis is less a completely accurate representation (as in the numerical, positivist tradition) but more of a reflexive, reactive interaction between the researcher and the decontextualised data that are already interpretations of a social encounter.
One of the challenges posed was not to filter out meaning, or impose an interpretation inconsistent with the original sense of the responses. For this reason, where appropriate the report of the data analysis has drawn heavily on the use of direct quotations.
In this study, although the transcriptions were captured and manipulated in electronic format the data was essentially analysed 'by hand'. In qualitative research studies, although it might appear that a computer would do what qualitative researchers want to avoid, namely standardise the process (Tesch, 1990) there is a growing use of computers to assist in the actual analysis of the data. Thus, several relevant software packages have been developed, for example SPSS, SpinxSurvey and Ethnograph (Cohen et aI, 2000). Whilst the scale of this study did not militate for the use of computer analysis any follow- up study might benefit from the application of such software.
Document analysis
The participants in the study at each of the universities were generous with contributing documentary data. Several different types of document were provided including: Undergraduate and Post-graduate prospectuses; Awards Descriptions; Programme handbooks; module descriptors; policy documents;
Academic Procedures; and relevant discussion documents. These documents were a significant source of data and an analysis of their contents would have been a study in its own right. The information contained within the documents was used in this study to clarify and add further detail, where necessary, to some of the issues and points raised by respondents. The documents therefore contributed to the study in a number of ways, in particular by adding depth and richness to the data, and by enabling issues relating to policy, process and practices to be verified. In this way, as explained in Chapter 3, the documentary evidence contributed to the validity of the data. In Chapter 3 one possible definition for the strategy of triangulation was given as being 'the use of two or more methods of data collection' (Cohen et ai, 2000). It is argued that in the conduct of this study the analysis of the documents, as a different source of data from the questionnaires and interviews, allowed for a type of methodological triangulation.
4.7 Ethics, reliability and validity in this study
It could be perceived to be stating the obvious to comment that in undertaking any study the researcher must be mindful of ethical issues. What requires great thought on the part of the researcher is the application of ethical principles in practice in the context of the particular study. In this study permission to visit the department, school or unit had been obtained in advance from someone in a position of authority. There was also total openness about the methods to be used for data collection and the use of the findings. Thus, the issue of gaining 'informed consent' was dealt with fairly simply. Informed consent is defined as 'the procedures in which individuals choose whether to participate in an investigation after being informed of the facts that would be likely to influence their decisions' (Diener and Crandall, 1978 cited in Cohen et ai, 2000: 51). Participants in this study agreed to be interviewed or to complete the questionnaire and at no time was there any covert gathering of information.
Compared to informed consent the issues of privacy and confidentiality were potentially more complex. There are three perspectives from which to
consider privacy. These are: the sensitivity of the information; the setting being observed and the dissemination of information. Linked to the issue of privacy is the obligation to protect the anonymity of the participants and to keep the data confidential (Cohen et ai, 2000). In a qualitative study this can pose some dilemmas. The most appropriate way to present a rich and thick description in the report of the findings is to give the account in the respondents' own words in the form of a direct or indirect quote. The tension that is then created is between ensuring the non-violation of the privacy of the individual who made the statement and giving some sense of the context in which the response was made. In this case the context included aspects like the specific university, the department or school, the modular programme and the role and scope of responsibility of the respondent.
Ultimately a decision was taken that since nothing that is cited in the report reflects negatively on any of the universities, and since they are part of the public domain, their identity could be revealed. The anonymity of the respondents was maintained by not revealing their names and through keeping the full transcriptions confidential. Where direct or indirect quotes from their responses have been reported two approaches were adopted. If the information is descriptive their role is included with the quote. If the information could in any way be interpreted as being sensitive or critical of the institution no identifier is included. Two examples, taken from Chapter 5 and indicated with bold italicised text, are given below to illustrate this approach:
The academic focus switches from the Course Team to [the] subject areas [such as]
microbiology, animal physiology, cell and molecular biology ... so once the Course Team has decided that there will be a unit, for example biochemistry ... 10 credit unit, assessment, entry requirements etc ... the Subject Group is informed by the Course Team of the types of student operational framework and the Subject Team deSigns the programme within the structure determined by the Course Team (Deputy Head of Department: Manchester Metropolitan University) .
... As one academic from University of the West of England explained in an unstructured interview: There is a danger of some students sort of lOSing their affinity for a particular award ... there might be a group of twenty students enrolled on a particular degree and they go off to modules where there are another hundred and eighty students who they have never seen before (or since) and if you are not careful the cohort tends to get subsumed in this large mass ... danger that affinity ... loyalty ... is decreased' .
As Chapter 3 explained in qualitative research the vocabulary used for assessing the quality of the study shifts from the traditionally used 'validity and reliability'. For example, Lincoln and Guba (1981, cited in Tesch, 1990) replace them with the term 'trustworthiness'. Thus, the validity of qualitative research does not depend on replicable outcomes. According to Tesch (1990: 304) it depends on 'the employment of the data 'reduction' process that leads to a result that others can accept as representing the data'. A good reduction selects and emphasises the essential features and presents the essence. It is in this spirit that the data is presented in Chapter 5.