Chapter 4: Research methodology
4.16 Challenges and Limitations of the research study
Research often comes with challenges, weaknesses and shortcomings some of which are beyond the researcher’s control. The researcher must find ways of minimising or overcoming challenges and
171
limitations such as credibility, transferability, dependability, and confirmability which are found within the qualitative research paradigm. It is important to identify and document these challenges and limitations in order to strengthen the study as well as open the research to academic review.
Credibility
In qualitative research, credibility may be defined as the degree to which the study's findings are credible, believable, and trustworthy. According to Lincoln and Guba (1985) credibility refers to the extent to which the study reflects the overall thoughts of the participants in the study, or the truth value. Polit and Beck (2012:430) interpret credibility as the degree of trust in how effectively the data addresses the desired topic. It entails determining if the findings of the research are trustworthy or credible from the standpoint of the study participant.
Credibility of document data was established through the thorough examination of different sources of literature related to the study. This is supported by Bowen (2009:30) who states that “when there is convergence of information from different sources, readers of the research report usually have greater confidence in the trustworthiness (credibility) of the findings”. By exploring available documents, the researcher is thus able to improve credibility. Furthermore, the process of triangulation helped to ensure that data collected through documentation was corroborated across data sets as proposed by Bowen (2009:28).
To further establish credibility, the researcher referred to interview recordings and notes to ensure a clear link between what was received during the research and the researcher’s interpretations. It should also be noted that while inferences were made, the researcher included as much of the individuals' actual words as possible. Cutcliffe and McKenna (1999:378) propose this method of guaranteeing credibility, stating that the researcher may use the participant's own words to guarantee strong associations to some of the emerging findings.
To strengthen credibility, the findings of this study were presented to the participants for affirmation that the researcher had properly interpreted and documented their views. This technique is supported by Kumar (2010:185) who asserts that the level of concordance is assured by presenting the findings to the research participants for “confirmation, congruence, validation and approval”.
Interviewing individuals from various perspectives has the potential to improve the credibility of conclusions (Rubin & Rubin; 2005:66). To ensure this, multiple interviewees were selected from both the Zimbabwean government and the MNCs as way of increasing the wealth and depth of information. This provided an informed perspective to the research and added to the trustworthiness of the study as the participants were selected based on their comprehensive knowledge of policies and organisational strategies.
The use of the case study approach is also seen as a reliable method of ensuring credibility as it allows the researcher to contrast and synthesise data to arrive at a sound conclusion. Elo et al.
172
(2014:7) support this by stating that in order to assure the trustworthiness and credibility of the outcomes, it is necessary to analyse how effectively categories cover the data and discover whether there are similarities within and differences across cases. This was done through cross-case analysis to compare and contrast the two cases and present findings.
Transferability
The degree to which study findings may be transferred to similar circumstances or contexts is measured by transferability. Elo et al. (2014:2) state that it is based on the assumption that results may be generalised or applied to different settings or populations. Moon et al. (2016:3) highlight the importance of transferability expressing that it can be critical to the application of research findings because policymakers and investors can count on data, conclusions, and recommendations from research projects.
On the surface, ensuring transferability with research of this nature may be deemed difficult considering that the findings in one particular industry may differ from others. Organisations are known to practices CSR strategies that are specific to the sector in which they operate, thereby making transferability to other sectors difficult. Dabic et al. (2016:251) confirm this and state that the
“industry or sector in which a firm operates determines CSR practice, as industries have their specific rules, norms, practices and trends”. Moreover, even within a sector such as mining, the initiatives will differ depending on the mineral mined and the method of extraction employed. As such, it cannot be assumed that knowledge derived from one study will necessarily be applicable to another. Despite this, by utilising a case study approach, similarities may be found thereby ensuring transferability especially within the mining sector. By providing sufficient data and outlining the research processes, the researcher anticipated that transferability would be made easier.
Although case study research delivers conclusions that are not statistically generalisable, it is helpful in developing theory especially when compared to other available studies (Gibbert et al. 2008:4). As such, the researcher selected appropriate cases and research participants which took into account the scope of the indigenisation policy. By utilising case study research and the purposive sampling method, the researcher purposefully selected participants who represented knowledge and insightful contributors to the study. This means that the data collected would easily resonate with other individuals in similar professions or spheres of knowledge.
According to Elo et al. (2014:6), academics may provide recommendations on transferability, but it is ultimately up to other researchers’ discretion to decide whether or not the given findings are transferrable to other studies. To aid this, it is however the responsibility of the researcher to ensure high quality results and reporting of the research process. To this end, the researcher endeavoured to provide an in-depth account of the research process as well as its findings in a manner that would provide as much meaning to any reader. The researcher gave a detailed account of descriptive data, including the context in which the study was conducted (the evolution of the Act and its
173
consequences for the study process), the sample, sampling strategy, interview procedure and questions as well as the limitations of the study.
Dependability
According to Thomas and Magilvy (2011:153), the dependability of a study “occurs when another researcher can follow the decision trail used by the researcher”. It explores the extent to which another researcher would be able to reach the same empirical results or conclusions after following the same processes used by the original researcher. Elo et al. (2014:2) go on to say that dependability refers to data stability across time and under varied settings. This points to the need to ensure that research must be auditable and repeatable as suggested by Yin (2014:49).
The aim of dependability is to reduce errors and bias in research and in order to accomplish this, the researcher documented the procedures undertaken during the research and produced a case-study database. This was done to ensure that all documents and information received and referred to throughout the study were effectively captured and stored. Documentation is important because it demonstrates that a systematic method was used when performing the case study, and if the same case study were performed by another researcher following the same technique, then it is possible that the other researcher would reach a similar conclusion.
According to Best and Kahn (2006:329) dependability may also be established by comparing responses given by participants with other submissions and sources. As such, the responses from the interviews were corroborated with the findings from other sources such as other interviewees’
responses, annual reports and government statements. With these measures in place, it is expected that a high degree of dependability will be maintained.
Validity
Validity measures the extent to which a study measures up to its objectives. According to Leininger (1985:60), in qualitative research, the term validity refers to obtaining information and understanding the nature (that is, the meaning, qualities, and features) of the phenomena under investigation. It is concerned with the veracity and integrity of scientific findings (Le Comple & Goetz, 1982:32). The interpretation of the research should be deemed to be valid and answer two important questions according to Kumar (2010:178):
1. “Is the research investigation providing answers to the research questions for which it was undertaken?
2. If so, is it providing these answers using appropriate methods and procedures?”
To answer these questions, the researcher focused on two types of validity limitations, namely, surface validity and construct validity. Surface validity, also known as face validity, evaluates the worth of the research technique being used. Construct validity on the other hand, is assurance that
174
the “theoretical concepts are adequately reflected by the operational definitions and measures of empirical phenomenon” (Modell, 2005:237).
To tackle face validity, the researcher tested the interview schedule beforehand and submitted the research findings to participants for feedback. Participants were also given a copy of the transcript to ensure that it accurately reflected their viewpoints on the topic under investigation.
In addition, the research process and findings were subjected to a peer review which revealed for example, anomalies and slight bias in the interview questions. Peer review assures that the research is valid and that multiple perspectives on the process have been considered, ultimately reinforcing the study findings. The reviewer is described by Lincoln and Guba (1985:308) as the "devil's advocate" who challenges the research through inquiries into the methodology, assumptions, interpretations, and findings of the study. This research was peer reviewed by an ex-colleague who holds a PhD and is a senior executive at a mining company in Johannesburg.
The researcher applied the triangulation approach, gathering data from several sources, thereby contributing towards the enrichment of the interview process and subsequent analysis which in turn supported the validity of the research. Triangulation is defined by Cohen et al. (2000:112) as the use of various data collection methods to explore a particular phenomenon. It is regarded as a highly effective verification technique to converge all the information to draw themes and eventual outcomes. This is supported by Yin (2014:47) who encourages convergency as a means of enhancing construct validity of case studies.
Cutcliffe and McKenna (1999:375) assert there is only one reality, and its validity is a measure of its correctness. Therefore, the researcher placed considerable effort in ensuring that this research met the validity test and sufficiently represented the objectives set out in the study.
Access to information
Accessing information is another challenge that is faced by researchers who deal with government departments and large organisations. Some participants are typically hesitant to offer complete information on specific issues because they are concerned about the sensitivity of some material.
For instance, when engaging with some members of the Zimbabwean government, this researcher was often reminded that some information was sensitive and could not be shared without approval from the President’s office. Zimbabwe is considered as a secretive and closed country and as such the researcher needed to be cautious in the approach to government departments as well as to be aware not to probe deeper than the participants were willing to divulge. The presentation of a research permission letter from the university also helped to settle participants and improve confidence in the line of questioning.
To ensure that the most accurate and relevant information was gathered from government and organisations, the researcher determined that officials at the highest levels of the institutions needed
175
to be engaged. Unfortunately, engaging these high-level officials or elite interviewees is often difficult and this meant that the researcher had to deal with gatekeepers, scheduling and had to comply with strict protocols. The researcher also had to be prepared for repeated questions on the purpose, goals and uses of the study before and during the interviews even though substantial information on this had been submitted beforehand. To overcome these challenges, the researcher sought and received assistance and support from individuals who had private and professional relationships with some of the elite participants. In this way, the researcher was received favourably especially by the mining organisations and this resulted in successful interviews being conducted.
Despite these obstacles, the selected participants provided useful information for the study. This was complemented and corroborated by data gathered through desk research. The researcher encountered no substantial hurdles to getting information from desk research, which increased the study's credibility.
Further limitations
The scope of the study was constrained by financial and time resources available to conduct multiple case study research. Research of this nature can be cumbersome and time-consuming as cases often provide more data than is needed for the research. This was overcome by sieving through the collected data to ensure that only the most valuable information was used.
The Covid-19 pandemic which gripped the world in 2020 was also a great impediment to this research. Various countries initiated stringent protocols in an attempt to curb the transmission of the highly infectious respiratory disease. Lockdown measures were implemented in countries throughout the world, originally involving the complete closure of businesses and schools, as well as strong directives for individuals to remain indoors. On the 23rd of March 2020, the President of South Africa, Cyril Ramaphosa, announced a 21-day national lockdown (effective from 26 March 2020), which included a directive that all South Africans were to stay at home as well as the suspension of international flights and the closure of borders. Only essential services personnel such as nurses, doctors and emergency workers were permitted to continue with their duties but under very strict conditions. The South African Defence Force was also deployed across the country to support the police in ensure public compliance. Similar measures were implemented in Zimbabwe, starting with a 21-day lockdown from 30 March 2020.
Following the first countrywide lockdowns, these restrictions were progressively eased, but the contagious nature of the disease and the absence of a cure meant that strict measures would be maintained throughout the year and beyond. All businesses, with special attention on the mining sector, were adversely affected by these measures. For example, Reuters (2020) reported that as a leading producer of minerals, South Africa's labour-intensive mining sector was regarded as a super spreader because workers were frequently confined to compact spaces and cluttered accommodation and that the shutdown would harm production and revenue.
176
For this research, the strict Covid-19 preventative measures and restrictions meant that the scheduled interviews could not be conducted on the expected dates. It had been expected that the researcher would attend meetings at the South African Head offices and the operations in Zimbabwe.
However, the strict measures, which included border closures meant that these visits would have to be rescheduled. Rescheduling of meetings was hampered by the uncertainty of how the Covid-19 measures would be handled by governments and ultimately, the two mining houses. The researcher, nevertheless, spent most of the year conducting desktop research to compliment and prepare for the interviews in the event that the Covid-19 restrictions were lifted. However, the pandemic stretched longer than anticipated and this meant that the research interview process was hosted via videoconferencing and not face to face as initially anticipated. A study extension was also granted by the university after the realisation that most students had been adversely affected by the pandemic and the measures that had been put in place by governments.