3.6 Data Collection Instruments
3.6.3 Qualitative Instruments – Document Analysis and Interviews
Implementation &
Integration)
- testing the maturity level of the respondents’ HEIs regarding the application, implementation and integration of ERM framework and concepts
Group C Questions (ERM Integration)
The participants’ perceptions and feedback on the already-implemented ERM policies and guidelines adopted in their institutions, and how effective they may be in relation to their academic institutions
The responses from the survey participants were then turned into statistical data and analysed using descriptive statistical analyses and non-parametric data test procedures, by running them through the specialised statistics application instrument of SPSS, as will be further explained in Section 3.8.1.
the researcher to substantiate and evidence the data collected from the participants through questionnaires. The review of documents also provided the researcher with the opportunity to utilise existing information to support the answers to the research questions, as well as to triangulate the survey data. It specifically supported the answers provided by respondents to the major research question (RQ1), where it informed the researcher of the development, applicability and integration of risk management policies in the selected HEIs.
Data elicited from document analysis is then combined with data from the interviews “to minimise bias and establish credibility” (Bowen 2009, p. 38). The document analysis in this study was conducted on the available and accessible risk management documents and related academic effectiveness policies obtained from two sources: the risk management and ERM policies and bylaws publicly available on the websites of UAE higher education authorities and agencies such as the UAE CAA 2019 Standards, and the risk management and ERM policies and manuals applied by some of the targeted HEIs. The thematic categories elicited from the analysis of those document would include the three major conceptual areas that comprise the subject of this study: ERM adoption, ERM implementation and ERM integration. They also provided informed insight into what areas the academic stakeholders in the UAE need to improve on in order to improve and sustain the effectiveness of ERM integration into their existing policies.
Despite all the limitations of the document analysis research process, such as the difficulty of access as well as the difficulties arising from the confidentiality of the documents, this approach derived value in the study from the fact that it enabled the researcher to obtain written evidence, which saved time and expense in transcribing extended interviews with the participants when asking them for full details of their existing and applicable ERM policies and manuals.
3.6.3.2 Semi-Structured Interviews
In their definition of interviews as a qualitative research instrument, Cohen, Manion and Morrison (2018, p. 506) posited that “the interview is a social, interpersonal encounter, not merely a data-collection exercise”. The authors also went beyond that by quoting Hochschild (2009) on the notion that interviews
“can do what surveys cannot, which is to explore issues in depth, to see how and why people frame their ideas in the way that they do, [and] how they make connections between ideas, values, events, opinions, behaviours, etc.” (Cohen, Manion & Morrison 2018, p. 506). In this sense, the researcher used the interviews in order to add more insights to the findings of the questionnaire.
The questions in the interview schedule (Appendix 3) were drafted based on the research questions, as well as the findings from both the questionnaire and document analysis (Fraenkel & Wallen 2015, p.
119). The interview questions were shared with the interviewees before the agreed interview time for awareness and research ethics’ considerations. The interview instrument was designed to answer not only RQ3, but also to partially answer the major RQ1, Group C questions. In fact, the Group C questions in the questionnaire were designed by the researcher and directed to the respondents to obtain their perceptions and awareness of the already-implemented ERM policies and manuals adopted in their institutions. In doing so, two informal pilot interviews with two expert faculty members in the fields of QA and ERM were conducted using an electronic recording device. The interviews were later revisited and reviewed to identify areas of enhancement and change for the interview questions. It was found by the researcher that the respondents’ time was a sensitive factor, and therefore the length of the interviews would need to be modified depending on the interviewees’ time and availability. The researcher therefore decided to shorten and decrease the number of questions. Each one of the two pilot interviews took almost 50 minutes to complete, where it was the researcher’s intention to spend one hour with each of the interviewees. Other than the lesson learnt regarding time management, the researcher gained other insights from the pilot interviews in terms of the more important areas to focus on when posing the questions, such as the quality of the questions, the wording, the use and understanding of terms and concepts and how they fit into conceiving better and more reliable findings with regards to RQ3 in particular.
The interview schedule helped the researcher clarify questions to the interviewees and expand on answers. The researcher phrased ten interview questions so that the answers fell into certain categories and themes that would fulfil the researcher’s objectives in answering two of the research questions of the study (partially RQ1, and fully RQ3). The face-to-face interviews were conducted in a way to purposefully select the site of the participants who were conveniently selected for the interviews, where the number and difference of sites would be an issue (Creswell 2014, p. 189). The interviewees included five key respondents (three risk management administrators and two faculty members), identified by the researcher on the basis of the convenience sampling and selected based on their availability, as well as their profound knowledge in the field. The number of interviewees could have ranged between 10 and 20 in order to achieve what Mason (2010) referred to as the “saturation level”. However, through the analysis of the five interviews conducted by the researcher, it became evident that the majority of codes and themes elicited from these five interviews were repeated with indication to similar results.
Additionally, due to the fact that this study started with the quantitative data collection and analysis with a higher priority, the number of interviewees was reduced to five given that the qualitative responses were not the major data source to conceive the findings in this study but were rather a supporting tool
only. The interviews involved semi-structured and “generally open-ended questions that are few in number and intend to elicit views and opinions from the participants” (Creswell 2014, p. 190). The faculty members and risk management administrators were then requested to answer ten (n= 10) open-ended questions (Appendix 3), to identity the existing ERM and/or QA policies and process applied in their respective institutions, and to define their perceptions regarding the effectiveness of their current and existing ERM and/or QA policies and processes. The interviews were conducted using an electronic device with recording capability, as well as online video call and online meeting applications (i.e., Microsoft Teams and Zoom) after getting consent from the interviewees. The researcher also asked the questions orally, with the answers recorded and then coded in writing at a later stage. All the interviews were conducted in the English language, and then transcribed verbatim by the researcher into archived texts.
The main themes that informed the interviews were very similar to the themes that informed the process of document analysis, since both phases of the study were directed to answer the same research questions.
The thematic categories derived from the interviews covered the three major conceptual subject areas of this study: ERM adoption, implementation and integration, as reflected and practised in the available ERM policies and manuals in the respective HEI. Added to these themes, the major RQ1 also informed the researcher of the requirement to gather perceptions from the study respondents on their existing ERM and/or QA policies and processes, and how they are using them as indicators of the effectiveness of their academic processes.