sequencing of the questions and adapted the questions according to the knowledge that was elicited from the interviewee. There were instances that necessitated the omission, addition and adaptation of the pre-planned questions so that the questionnaire provided a structure that enhanced the prospect of creating an enabling environment where the researcher is able to engage the interviewee in a conversational context. As Creswell suggests, the main idea is to learn about the problem by implementing strategies that enable seamless elicitation of information to enhance the richness of the engagement with the study’s respondents.
From an overview perspective, the pilot study panel were of the opinion that the questions targeted the main issues regarding agile software development methodology. However, a few aspects needed to be noted as omissions, additions or general comments that will arguably ensure that the questions asked during the interview sessions had a good balance and targeted pivotal issues in the domain of agile software development. These aspects are classified according to the designation of the members of the pilot study panel, and are listed below:
From the industry based professionals:
The use of a pair programming strategy was not done as it is formally implemented in many academic settings (such as the case at UKZN). The comment made in this regard is that pair programming is done more on an ad hoc basis rather as an institutionalised strategy;
The tracking of project progress is conducted via a plain old whiteboard (POW) strategy rather than making use of any formalised project management tools such as the Gant and Pert Chart, as is the case in an academic context. The POW approach entails the drawing of columns on a whiteboard where project teams document features of an application that ‘need to be done’, are ‘in- progress’ and ‘have been completed’. This approach has a strong resonance with agile methodology, in particular, the Kanban Storyboard;
The issue of organisational culture (OC) is not something that many IT professional will identify with from a theoretical perspective.
However, they will be knowledgeable about the influence of OC from a pragmatic perspective. A suggestion made in this regard was to provide an explanation of the different ‘strands’ of OC so that the IT professionals will be able to identify with the theoretical version a lot more easily and provide a more meaningful response in this regard;
A currently emerging aspect of agile development is the dilemma regarding scalability. Many organisations were resorting to a DevOps-based version of agile methodology. A question/reference to DevOps should be included;
The original set of interview questions were too long and many industry professionals may not have the time to engage with these issues in a focused manner for such a lengthy period. In addition, there were instances of overlap between aspects that were included as discussion questions. These aspects should be conflated into a single/follow-up question that fitted in seamlessly into the discussion rather than being included as a separate discussion points. These aspects included issues dealing with design/architecture as well as iterative and incremental development.
From the academic representatives:
It would be ‘nice to know’ the actual paradigm of development that was followed in industry. This comment was a reference to the use of object-oriented (OO) development, a classical/structured approach or a hybrid approach. This comment was deemed to be pertinent to the study because a pure OO approach would entail the use of Unified Modelling Language (UML) which has been claimed to be documentation-intensive and contrary to the objectives of agile methodology (Petre, 2013; Rumpe & Schröder, 2014; Turk et al., 2014). The objective of this knowledge is that it would provide an insight into the modelling/architectural requirements for the implementation an agile approach to software development;
Issues pertaining to the design of the user interface should be given some sort of coverage in the interview questionnaire. This suggestion has been endorsed by the researcher as a valid one and
it is an issue that has been the subject of extensive deliberations as alluded to in Brhel et al. (2015).
While the input obtained from the pilot study was quite useful in ensuring a measure of validity with regards to the content of the questionnaire, the overall design of the questionnaire was guided by the dictates of Rubin and Rubin (2012, p. 6) who suggest that the structure of an interview should revolve around three types of linked questions. These are the main questions, probes and follow-up questions. The main questions attempt to ensure that there is adequate linkage with the research questions, probes are used to encourage the interviewee to continue talking and follow-up questions are used to explore main themes discussed enabling the researcher to elicit more depth from the interview. The questionnaire (see Appendix A) used as a guide for the interview sessions in the current study is structured along the lines of main questions, follow-up questions and probe type questions
The questionnaire content is guided by the suggestion in Rubin and Rubin (2012) that the main questions should emanate primarily from the researcher’s knowledge and experience in the study’s domain. A secondary source could be the academic literature. However, they do warn that the formulation of main interview questions from the academic literature is not ideal, because “…it will blind the researcher to what is actually out there” (Rubin & Rubin, 2012, p. 134). The academic literature should be used sparingly and questions based on the literature should be worded carefully and simply so as not to convey the idea that the interview entails an examination of the interviewee’s theoretical knowledge. Also, in order to obtain an insight into the lived experience(s) of the interviewee, the researcher needs to pose questions that are open-ended and to make use of a technique called responsive interviewing (Rubin & Rubin, 2012, p. 4) where the researcher is able to dynamically “…change questions in response to what he or she is learning.” This responsive style of questioning has been adhered to in the current study by virtue of the probes and follow-up question that have been included in order to arguably ensure that there is a measure of responsiveness with regards to the questions asked. This also enables the interview process to
resemble a conversational partnership that the researcher establishes with the interviewee (whom Rubin and Rubin (2005, p. 6) refer to as a conversational partner). The probes and follow-up questions that were dynamically generated and converged to a finite set is highlighted in the questionnaire (see Appendix A).