One of the fundamental choices facing a researcher when formally coding data is a choice between depth and specificity, which generally translates to a trade off between reliability and validity (Babbie, 1995). In terms of this, there are two types of content that may be identified: manifest (explicit) and latent (implicit) content (Babbie, 1995; Neuman, 1997).
Manifest content refers to the visible, surface content of text. This type of coding is regarded as highly reliable because it only considers whether a word appears in the text or not, and is thus very specific. However, the validity of manifest coding may be
compromised because it does not consider the connotations of words or phrases, nor the context in which the text has emerged (Krippendorf, 1980; Potter & Wetherell, 1987).
On the other hand, latent content refers to the underlying and implicit meaning of the text.
This type of coding is a process of abstraction. Latent coding is viewed as less reliable since it is based on the subjective categorisation of the researcher. However, the lower specificity strengthens the validity. Babbie (1995) suggests that one way of overcoming this dilemma is to use both methods where possible.
Since the purpose of this study was to explore and describe concepts, both the manifest and latent levels of the data were important sources of information. For this reason both levels of coding were used. In terms of this, it was important to assess the reliability and validity of the data.
3.13.1 Assessing reliability
For research results to be considered valid the results must be reliable. In order to test reliability, some form of duplication is necessary. In other words, the process or method of data analysis should yield the same results of the same phenomenon regardless of when the procedures are applied (Krippendorf, 1980).
Three types of reliability are important in content analysis (Krippendorf, 1980; Weber, 1985). These are: stability, reproducibility and accuracy.
Stability refers the degree to which the process of coding the content of the data remains stable and unchanging over time. This type of reliability becomes evident during test- retest conditions, such as when a coder recodes the data after an interval. Emerging inconsistencies and discrepancies constitute unreliability. Stability however is the weakest form of reliability since only one person is involved in coding the data. In order to obtain high levels of stability in this study, I coded and recoded the text three times over a period of three weeks.
Reproducibility refers to the extent to which the process of coding produces the same results when more than one person codes the text at different times. This type of reliability is known as intercoder reliability or external reliability, and is central to content analysis.
Disagreements between the coding of different individuals reflects ambiguous coding instructions, random coding errors or intercoder differences in terms of the way in which they perceive the contents of the data.
Once I had completed the process of coding the data for this study, the texts were given to an independent researcher to code. During this process, ambiguities in the instructions as well as random coding errors were eliminated. The level of reliability was established for each focus group using the following equation (Boyatzis, 1998):
Percentage agreement = number of times both coders agree number of times coding was possible
Focus group 1: 96% = 48 50
Focus group 2: 90% = 44 49
Focus group 3: 9 5 % = 4J_
43
Reproducibility also considers the way in which the data was collected. In the case of this study, a semi-structured open-ended questionnaire was used to guide the focus group. This semi-structured questionnaire has an important role to play in terms of ensuring the
approximate replication of this study by other researchers.
Accuracy is the strongest form of reliability and refers to the extent to which the results of the content of the classification conform to a standard or norm (Krippendorf, 1980; Weber,
1985). This type of reliability is rarely available for texts, and thus seldom used for this form of data. Accuracy was not established since this study was largely explorative in nature.
3.13.2 Assessing validity
Validity refers to the quality of research results that leads them to being accepted as truth or empirical fact. Content analysis is considered valid if inferences are maintained in the face of independently obtained evidence (Krippendorf, 1980). In content analysis, there are two distinctions regarding validation. The first refers to the degree of correspondence between two sets of concepts and variables, and the second involves the issue of
generalisability of the results (Weber, 1985). Ensuring the validity of a study requires that one validate research evidence in terms of the nature of the data, results obtained or the process connecting the data and the results (Krippendorf, 1980).
Two types of validity relating to the nature of the data are: sampling validity and face validity. Sampling validity refers to the extent to which the sample from which the data obtained is statistically representative of the population. In content analysis, the data obtained is generally from a sample selected in terms of particular criteria and is therefore not representative, as was the case in this particular study.
The small, non-random sample of this study thus limits generalisability of the results onto a greater population. However, the results of this study are useful in terms of enabling the identification of basic features of concepts unique to this group of participants, which may
then be a stimulus for further research in this area of study. Since the aim of this study was exploratory the generalisability of the findings was not an issue and sampling validity was not achieved.
Face validity refers to the degree to which a category appears to measure the construct it intends to measure, and is based on the coder's definition of a concept or category (Krippendorf 1980; Weber, 1985). This type of validity is the weakest form of validity, but content analysis relies heavily on face validity. Generally face validity is achieved when more than one coder agree about the definitions of the concepts. In this study, two independent coders coded the data and verified the definitions in an effort to achieve face validity.
The units of analysis selected for the data analysis also have an impact on the validity of the coding. In the case of this study, the thematic units of analysis required a deep level of investigation resulting in high levels of validity. This analysis involved analysing explicit as well as implicit meanings found in the content of the text. Reliability however, may have been compromised because although general concepts are usually easy to recognise, it is often more difficult for multiple coders to reliably identify them. Furthermore, it is important to be aware of potential researcher bias during the concept formation stage of data creation. This potential bias may have negative implications on the validity of the data. In terms of this, my goal was to remain reflexive throughout the process.
Semantic validity is achieved when two or more coders classify coding units in the same categories, and both agree that these units have similar meanings or connotations
(Krippendorf, 1980; Weber, 1985). Semantic validity was achieved in this study and confirmed by both coders.
According to Stewart & Shamdasani (1998), focus groups result in data that has ecological validity. In terms of this, rich data is obtained that is expressed in the participants' own words. During the course of the focus group, they are able to qualify and their responses, and in this way clarify any misunderstandings or ambiguities.