CHAPTER 3 METHODOLOGY AND RESEARCH DESIGN
3.4 Validity and Reliability
The researcher was very careful in conducting this study in order to obtain valid and reliable data as well as results at the stages of both collecting and analyzing the data. The validity and reliability testing procedure of this study are explained below.
3.4.1 Validity
The research validity is important about the scope of study results and findings that denote what is going on (Jill Collis, 2003). It is the degree that the researcher uses to measure accurately reflecting or assessing the specific conception in the study (Jupp, 2006). It ensures both the sufficiency of the coverage of the concept taken to measure and also the accuracy. The term validity describes the measurement that correctly represents the perception it is planned to measure (Babbie, 2010). It directs the acceptability, accurateness, and the truthfulness of the study results. For the present study, content validity and construct validity were obligatory. The processes of content and construct validity are described below.
3.4.1.1 Content Validity by Experts
The item-objective congruence (IOC) index introduced by Rovinelli (1977) was used to measure the face validity and content validity of the items of the questionnaires (Rovinelli & Hambleton, 1977). The item (content) experts rated each item concerning how perfectly the items matched or not matched the study questions or objectives or the variables of the mentioned conceptual framework of this study.
The rankings were -1, 0 and +1; and -1 was counted as totally invalid, 0 was designated as no comment, and finally, +1 was denoted as a totally valid question or item. Further, the items of the questionnaire were also validated by a panel of innovation content specialized experts, consisting of four members. Two of them were from Bangladesh and the other two members were from Thailand. Since the field area of the study was the district and upazila level of administration of Bangladesh, the experts from Bangladesh have better experiences about the leadership
and organizational support there, the organizational culture of the field administration in the Bangladesh context, and therefore half of the experts were from Bangladesh.
After getting the approval of the content experts concerning the questionnaire with 85 (without 07 demographic items) items under 19 variables, the questionnaire was used for the survey for this study. The details of the content experts are given below.
Table 3.3 The Details of the Content Experts Serial
No.
Name of the Expert The Present Position of the Expert
1. Kasemsarn Chotchakornpant Ph.D. Associate Professor GSPA, NIDA, Bangkok, Thailand.
2. Mobasser Monem, Ph.D. Professor and Chairman, Dept. of Public Administration,
University of Dhaka, Dhaka, Bangladesh.
3. Sanwar Jahan Bhuiyan, Ph.D. Deputy Secretary (DS) to the Govt. of Bangladesh & Director (PPR), BPATC, Dhaka,
Bangladesh.
4. Nuttakrit Powintara, Ph.D Assistant Professor, GSPA, NIDA, Bangkok, Thailand.
Turner and Carlson endorsed by Rovinelli and Hambleton (Rovinelli &
Hambleton, 1977) explained the IOC as follows: “in a situation in which four content experts are being used to assess a set of items, a minimal criterion might be the index value that would be attained if a minimum of three of the four experts classified an item as hypothesized by the developer (Turner & Carlson, 2003). Thus, although the cutoff value is a floating criterion, a generally accepted value might be a minimum of .75.” The items were removed from the questionnaire that were scored less than .75.
And finally, as per the demand of the experts, the questionnaire was revised and
rewritten. After this process, the researcher conducted a pilot study before the collection of the full volume data.
3.4.1.2 Construct Validity
According to Babbie (2013) “construct validity is based on the logical relationships among variables. Construct validity is the degree to which a measure relates to other variables as expected within a system of theoretical relationships”
(Babbie, 2013). Thus, the main theme of construct validity ensures that the theoretical relationship with the items of the operational definitions really measures what it is proposed to measure. All of the related models, theories, and assumptions of public service innovation, transformational leadership, perceived organizational support, and organizational culture were reviewed in order to ensure construct validity. The theoretical framework that has been proposed for this study strongly supported the transformational leadership of Bass and Northouse (Bass, 2006), the POS of Eisenberger (Eisenberger et al., 1986), the perceived organizational culture of Hofstede (Geert Hofstede, 1991), (Hofstede, 1997) public service innovation of Rogers, Amabile, and West, and creativity of Amabile (1996), Evans (2007), Kaufman, Plucker, and Baer (2008). Further, in order to obtain the construct validity for this study, the previous studies and literature were reviewed exclusively. It is essential to mention here the questionnaire items, which were developed after a rigorous study with Bass and Northouse on transformational leadership (Bass, 2006), with Hofstede for organizational culture (Hofstede, 1991), with Eisenberger regarding perceived organizational support (Eisenberger et al., 1986), and Amabile and Tidd regarding innovation (Amabile, 1988) (Tidd, 2007), and finally for creativity with Amabile, Evans, Kaufman, Plucker and Baer. In all cases, the items were customized and developed with the public service innovation outcome context of Bangladesh.
Additionally, factor analysis was done, which was exploratory, in order to examine the correlations among the variables. The validity of the constructs is normally confirmed by the examination of the factor loadings.
3.4.2 Reliability
Reliability determines the accurateness and correctness of the events, measurements, and also the ability to replicate the study. That means if the exact and
same procedures are followed repeatedly, the findings will be the same, as indicated by Babbie: “Reliability refers to the matter of whether a technique applied repeatedly to the same object yields the same result each time” (Babbie, 2013). Reliability concerns whether the measure is unchanging and constant or not. Consistency is the perfect meaning of reliability, which is a degree of measurement. Cronbach’s alpha is the most frequently used measurement procedure of core constancy and reliability.
For this study, data were collected from the members of the innovation teams of the district and upazila administration of Bangladesh. All the needed actions were taken to collect the data and ensured the confidentiality to secure and reduce biases. In a survey questionnaire when a researcher uses multiple Likert scale questions, then it is necessary to measure whether the scale is reliable or not. .70 is the minimum reliability coefficient that is generally accepted (Hair, Black, Babin, Anderson, &
Tatham, 2010). The results of the test are shown in the table 4.5.