Focus group
5.9 Validity, Reliability and Trustworthiness
Yin (2016) argues that the objectivity of a piece of qualitative research is evaluated in terms of its reliability and validity. Validity and reliability are key aspects of all research. Meticulous attention paid to these two aspects can make the difference between good and poor research, and can ensure that fellow researchers accept the findings as credible and trustworthy (Merrens & Ginsberg, 2008). Creswell (2017) argues that the traditional criteria for ensuring the credibility of the research data’s objectivity is through reliability and validity.
5.9.1 Validity
Validity is the extent to which the data accurately measures what they were intended to measure (Creswell, 2017). Five key elements, as presented by Leedy & Ormrod (2013), who highlight that the validity of the research instrument intends to address the following areas:
1. Validity addresses the issue of whether the researcher is actually measuring what he/
she has set out to do.
2. Face validity requires the research instrument to be relevant to participants in the study.
3. Content validity is similar to face validity except that the researcher must seek the opinion of experts in the field on the adequacy of his/her research instrument.
4. Predictive validity refers to the capacity of a respondent’s ratings and responses to items on the instrument to predict behavior outside the immediate framework of the research instrument.
5. Concurrent validity indicates whether the level of responses to items on the research instrument is parallel to other facets of the respondent’s overall behavior.
The following discussion confirms that organisations that participated in the research are involved with the PPP process authentically. In doing so, the study validated the list of organisations involved in this research, the positions of the individuals that were interviewed within the organisation, the role of participants within the PPP projects sampled, the project
147
stage of development within the PPP process, and the period of existence (life period since inception) of the project. Before the researcher could solicit the views of the participants, it was critical to confirm that the participants were involved in the PPP process from both the organisation’ perspective and as individuals within an organisation. The research information required needed the involvement of both the individual interviewed and the organisation that the individual worked for to be involved in the PPP process. This would give credibility to the information gathered in the sense that the participants would have been verified as a credible source of information and were providing information based on their experience.
Involvement of individuals and organisations interviewed about PPPs
Robinson, et al. (2010), argue that, it is important to acknowledge and understand the key enablers affecting the delivery of an effective public private partnership procurement process. All the participants interviewed confirmed that their organizations were involved in the public private partnership process and were also using public private partnerships as a procurement process. 30% of these participants were involved as enablers within the process, for example, Treasury, financial institutions and tourism agencies. According to Robinson (2010), one of the key elements for the PPP, is the supporting units or enabling units to this process which, in this case are the four institutions, namely; the Provincial and National Treasury, Department of Economic Development, Tourism and Environmental Affairs and the financing institutions.
Further on the credibility side, the researcher had to establish the role that the participants played within the PPP projects in their organizations. The objective was to gauge the level of involvement and responsibility within the project. This would assist in ensuring that the views presented are from a balanced group exposed to the project from different levels and roles.
Following is the list of roles played by different participants interviewed within the PPP process. These are listed in no specific order:
Managing the process on the operational side as project team member
Project Manager and managing the strategic direction of PPP’s internally
Project team member leading the procurement process
148
Auditing and advising on the project
Coordinating PPP Initiatives
PPP Project Leader
PPP Project Facilitator
PPP Project Oversight
Facilitation and Coordinating the PPP Projects Implementation
Planning, Facilitation and Administration of PPP Projects
PPP Projects Facilitation
Based on the above, it is clear that the roles span across leadership, facilitation, coordination, administration, advising and auditing. There were strong overlaps on project facilitation and project management roles. The strong involvement of these participants in the PPP projects enhances the credibility of the information provided.
5.9.2 Reliability
The reliability of a research instrument refers to the consistency or repeatability of the measurement of some phenomena (Creswell, 2017). The reliability (the extent to which the data collection method will yield consistent findings if replicated by others) of the data collected depends largely on the research instruments. Smith and Ragan (2005) present three forms of measuring reliability, i.e.
1. Parallel forms of reliability: A measure of equivalence that involves administering two different forms of measurements to the same group of participants and obtaining a positive correlation between the two forms.
2. Test-retest reliability: This essentially involves administering the same research instrument at two different points in time to the same research subjects and obtaining a correlation between the two sets of responses.
3. Inter-rater reliability: A measure of homogeneity. With inter-rater reliability one measures the amount of agreement between two people who rate a behavior, object or phenomenon.
149
All the participants interviewed confirmed that they were involved in the public private partnership procurement process within their organisations. This information assisted in ensuring that the research focused on the correct people within the targeted organisations and therefore, ensured the trustworthiness of participants and assured the reliability of the information gathered.
Both the participants and the organisations involved in the study were involved in the tourism public private partnership procurement process. Two of the participants’ organisations were involved in the process as enablers. Therefore, the information gathered was from a balanced sample. The next theme covers the understanding that the participants had about PPPs in general and PPPs as a procurement process within the tourism sector.
Since qualitative studies are usually not based upon standardized instruments and often utilize smaller, non-random samples, the question is whether these reliability evaluation criteria have any value in qualitative studies. Patton (2014) cautions researchers that assessing the accuracy of qualitative findings is not easy. However, there are several possible strategies and criteria that can be used to enhance the trustworthiness of qualitative research findings.
5.9.3 Trustworthiness
Trustworthiness is the corresponding term to reliability and validity used in qualitative research as a measure of the quality of research. It is the extent to which the data and data analysis are believable and trustworthy. Creswell (2017:27) suggests that “trustworthiness of qualitative research can be established by using four strategies: credibility, transferability, dependability and conformability.”
Effectiveness requires planning beforehand to ensure that the data can objectively be analysed afterwards (Creswell, 2017). The researcher produced an interview schedule, to control the line of questioning. Through the focus group, the researcher checked the validity and reliability of the information gathered. This is where the researcher confirmed and verified the accuracy of information. The issues related to the proposed solutions were further probed. The researcher further allowed debates among the participants which
150
provided further trustworthiness as different perspectives were debated in a combined group with participants in the focus groups triangulating the findings.