2. Methodology
2.12 Validity and Reliability of Research
According to Denzin and Lincoln (2000), researchers within a constructivist paradigm, such as that used in a case study research methodology, attempt to reconstruct participants‘
understanding of the world, such that traditional criteria of internal and external validity are replaced by concepts like trustworthiness, credibility, authenticity, transferability etc. For this study, dependability and auditability was used in the place of reliability (Guba & Lincoln, 1985), where the question to be answered was whether the research design, purpose, data collection and analysis strategies indicate coherence. Internal validity is replaced by credibility or authenticity, where the aim is to ensure that the findings and outcomes can be depended upon.
For the study to be meaningful and for it to contribute to the body of knowledge, the outcomes and conclusions have to be transferable to other contexts, so that external validity is replaced by transferability or fittingness (Patton, 2002; Yin, 1993).
2.12.1 Trustworthiness
According to Folger et al. (1984), the development of a good coding scheme is the key to trustworthiness in research employing content analysis. Lincoln and Guba (1985) defined the four criteria central to trustworthiness in content analysis research as: credibility, transferability, dependability and conformability. This section explains how trustworthiness was improved in the study.
2.12.2 Credibility
As evidence of trustworthiness, credibility or internal consistency is a key requirement of summative content analysis. Credibility was defined by Bradley (1993:436) as ―the adequate representation of the constructions of the social world‖. The need to demonstrate that textual
evidence is consistent with interpretation (Weber, 1992; Hsieh & Shannon, 2005) was made easier by the fact that the words under study are technical terms which mean the same thing across businesses and industries. Professional colleagues in the climate change field were used to check reproducibility of the coding categories for the climate change response initiatives.
Reliability was calculated by using Cohen's Kappa (Cohen, 1960), which calculates the proportion of units on which raters agree, taking into account chance agreements. A Cohen Kappa of 0.83 was considered good enough for the coding categories to be considered credible.
Only one researcher was involved in the study both from a data collection and coding perspective, so the question of inter-coder reliability is automatically eliminated in this study.
Validation of the coding scheme was provided by an industrial psychologist who was the content expert (Hsieh & Shannon, 2005; Krippendorf, 2004) in understanding organisational decision making biases and bounded rationality concepts.
Other actions taken to improve credibility included triangulation of data, where secondary data from the publicly available Corporate Sustainability Reports, JSE SRI and CDP submissions were also used. Negative case analysis (Patton, 2002) was actively pursued in the study where all outliers were carefully analysed. Interpretations were checked consistently against raw data;
while peer debriefs (Lincoln & Guba, 1985) using climate change experts and an industrial psychologist were consulted extensively throughout the data collection and data analysis phases of the study. The data collection adequately solicited representation by involving as many functional areas as necessary. Some of the conclusions were drawn directly from the raw data.
2.12.3 Transferability and External Validity
Transferability is the degree to which a researcher‘s working hypothesis can be applied to another context (Krippendorf, 2004). For this study, data sets and descriptors are provided in the Findings section and Annexure 3, whose transferability to other settings and contexts other researchers can determine. Dense description (Lincoln & Guba, 1985), also called ―thick description‖, was used throughout this research study. To thickly describe social action is to
―begin interpreting it by recording the circumstances, meanings, intentions, strategies, motivations, and so on that characterize a particular episode. It is the interpretive characteristic
of description rather than detail per se that makes it thick‖ (Schwandt 2001: 255). Theoretical and analytical generalisation was employed through ―pattern matching‖ (Yin, 1984), where patterns found in the data were linked to patterns found in climate change response and bounded rationality theories, thus aiding in conceptual generalisations.
2.12.4 Conformability
Conformability is the extent to which the characteristics of the data posited by a researcher can be confirmed by others who read or review the research results (Bradley, 1993).
Both transferability and conformability can only be judged by reviewers of this work. Audits of the research process (including methods, decisions and analytical strategies), findings and checking the internal coherence of the research product by checking the raw data (some of the interview transcripts included in Annexure 3), the findings and the recommendations have all been verified by the academic panel throughout the course of the research.
2.12.5 Validity
Triangulation is the in-built design tool most accepted as a way of increasing the validity of a research process (Lincoln & Guba, 1985; Patton, 2002). Multiple sources of information for data (data triangulation), theory triangulation (multiple theories/perspectives used to interpret the findings of the study) and interpretation were used in this research. ―Thick description‖
(Schwandt, 2001) of design phenomenon and of the findings were closely associated with corporate climate change responses and bounded rationality theoretical underpinnings.
Literature Review
This section is divided into three chapters. Chapter 3 describes the climate change challenge, and it‘s relevance to business. The risks and opportunities presented by climate change are discussed followed by some of the key drivers that are motivating companies to respond to climate change. Chapters 4 and 5 describe the key theories that are used to construct the climate change response framework. The influence of bounded rationality to industrial organisations and its impact in strategic decision making are emphasised. Multi-criteria decision aid theories and their relevance in solving multi-criteria decision problems are presented in chapter 5.