CHAPTER FIVE – RESEARCH METHODOLOGY
5.4 Data collection techniques
5.4.3 Pre-testing
Pre-testing is the administration of the data collection tools to a small group of potential respondents for the purpose of identifying weaknesses with the tools which can then be corrected before the actual data collection (Bowden et al. 2002; Schaller 2005). Pre-tests are administered with the understanding that problems which may occur during the pre-tests are likely to occur also during the actual data collection. Weaknesses in data collection tools could be manifested in the mode of administration, terminology, structure or sensitivity of the questions asked (Collins 2003). Some scholars also argue that the researchers must conduct pre-tests themselves without delegating them to assistants because they understand the aims of the research better (Bowden et al. 2002). Pre- testing data collection tools and techniques helps the researchers to establish whether the respondents understand the concepts being probed in a consistent way and as intended by the researcher (Collins 2003).
Several pre-testing techniques exist. Some of these include focus group discussions, cognitive interviews and field pre-testing (Fowler 1998; Forsyth, Rothgeb and Willis 2004). The researcher applied the cognitive (intensive) interview technique and discussed with research experts, representing users, and Librarianship experts, representing librarians to test the suitability of the data
89 Estimates by librarians in the case libraries during preliminary interviews conducted on 13-17 October 2008 and confirmed during the actual data collection.
140
collection techniques and tools. The choice of cognitive interview was based on the understanding that it is more in-depth than the other techniques and evaluates not just responses to questions but also the thought processes influencing the answers. Thus, it was likely to generate better assessment of the questions, data collection tools and the approaches in administering them than the other techniques. The researcher interviewed twenty-seven respondents comprised of sixteen librarians and eleven researchers. The participants were chosen purposively based on expertise, convenience and availability. All the data collection techniques (see 5.4.1) and tools were pre-tested. The pre- tested data collection tools included questionnaires for both librarians and users; focus group discussion guides for both librarians and users; observation checklists; mystery shopping scenarios;
and social analysis questions. Based on the findings, the researcher made several changes to the techniques and tools summarized hereunder.
5.4.3.1 Changes made to the questionnaires
The researcher made the following changes to the questionnaires:
Questionnaire for librarians
1. The researcher found that the question on library models for librarians was misunderstood most. The researcher listed the common models and made provision for explaining them during the interviews;
2. The researcher also reworked the wording of question 3a; some respondents felt that the question, as originally phrased, sounded like it was testing the librarians;
3. The researcher also included choices for the levels of importance for questions 3c and 4c;
4. The researcher removed question 3d which sought the view of the librarians on services they are offering but are not popular. There were views that it would be humiliating for librarians to confirm that they are wasting scarce resources on services which are not being used. The answer to this question has been deduced from responses to question 3c; and
5. The researcher also modified question 5 to integrate challenges and the solutions. So he added a column for solutions to enable the respondents to suggest possible solutions to the corresponding challenges they identified.
141
Questionnaire for users
1. The researcher added a new question (1d) asking the respondents to indicate their general area of research. This was useful in comparing the responses of the researchers from different areas of research;
2. The researcher also included the levels of importance in the table to make responses easier for question 3d;
3. The researcher reworked the Lickert scale of question 4f by adding “Very Poor”; and
4. The researcher also reworded question 6a to make it clearer. The previous version was long- winded and presented in passive tense.
The final versions of the questionnaires for the librarians (Appendix 5.5) and users (Appendix 5.6) are enclosed in the Appendices.
The researcher also used the test-retest approach in which he administered the same set of questionnaires twice to the same sets of respondents (one set comprised of three users and another of three librarians chosen by purposive sampling) after a fortnight and correlated the scores from both testing periods. The scores were generally consistent.
5.4.3.2 Changes made to the Focus Group Discussion approach for librarians
The researcher also changed the approach for the Focus Group Discussions (FGDs) for librarians.
This change followed the realization that the discussions would be more fruitful if the researcher brought together librarians from all the institutions. The change was necessitated by the fact that there are very few librarians (an average of three) in every case library. So, he held two sets of FGDs;
one for the head librarians of all the case libraries and another for the other librarians (librarians and assistant librarians) from all the five cases. The researcher had difficulties in organizing the FGD for head librarians due to their tight schedules and some degree of reluctance in creating time for the discussions. He therefore organized a half-day workshop at ILRI, Nairobi Campus, facilitated by Mr.
Peter Ballantyne90 which attracted them. Mr. Ballantyne made a presentation on the potential
90 Mr. Ballantyne is the Head of Knowledge Management and Information Services at ILRI. He is based in Addis Ababa, Ethiopia and is renowned for application of Web 2.0 tools in Knowledge Management. He presented his experiences in harnessing the potential of Web 2.0 tools in delivering information services with a focus on their potential for research libraries in Kenya.
142
applications of Web 2.0 tools in research libraries in Kenya. Thereafter, the researcher facilitated a focus group discussion based on the FGD guide for librarians. The programme of the workshop is enclosed as Appendix 5.9.
5.4.3.3 Changes of approach to Social Network Analysis
The researcher had initially set out to conduct direct interviews with the researchers and librarians to get information on social networks in and around the libraries. However, the pre-test indicated that many respondents were not comfortable giving the information. The researcher then consulted some experienced researchers in Library and Information Systems such as Professor Dennis Ocholla of the University of Zululand and Dr Bosire Onyancha of the University of South Africa (UNISA) who have experience with applying Social Network Analysis to Information Sciences for advice. It emerged from the discussions that there are very few cases in which researchers have conducted direct interviews with respondents to unveil their social networks. Therefore, the researcher conducted a fresh literature review on Social Network Analysis and established that the reluctance of the respondents to give information on their social ties arises from some intrinsic disadvantages of Social Network Analysis as a technique. Some of these include:
1. The questions to elicit social ties are often of a sensitive nature – some questions can easily cross lines into personal matters and are tantamount to invasion of privacy especially when respondents name people in their social networks who have not consented to participate in the research;
2. Sociograms (social network diagrams) do not hide the individual responses using statistical aggregates. The data in sociograms is raw, to a large extent, and is just presented as is. There is no room for anonymity. Further, the respondents have to disclose their own names, the names of the colleagues with whom they relate and the type of relationship(s) they are in.
Thus, anonymity cannot be ensured. This degrades confidentiality of responses;
3. The respondents view the researcher as an “outsider” to the institutions and may not cooperate effectively;
4. Social Network Analysis facilitates the “participation” of the people identified by others even if they chose not to participate. Opting out of a study that involves Social Network Analysis may not totally preclude one from being portrayed in the sociograms;
143
5. Results of Social Network Analysis can be used to penalize respondents (for instance, when the sociograms indicate unfavourable linkages) so many potential respondents would not want to be involved in such a research project; and
6. The results of Social Network Analysis may result in hurt feelings of those members of the institutions who are portrayed in negative light. Similarly, participants may not want to list anyone‟s name unless it is in a positive context.
It also emerged from the consultations and literature review that direct Social Network Analysis is more suitably applied by members of a network or an institution. It is not easy for a researcher from outside the institution to conduct it effectively. Under these circumstances, it became apparent that the respondents would not be willing to participate in the study, let alone being able to be honest in their responses. Under these circumstances, the research data would have been skewed.
The researcher, therefore, decided to analyze the social networks using indicators of linkages such as research collaboration and co-authorship. The researcher conducted an analysis of co-authorship amongst researchers as an indicator of collaboration and social networks. The researcher also did the same for the librarians. The analysis was based on the entries of publications from the institutions listed in the online Web of Science (Science Citation Index; Social Science Citation Index; Arts and Humanities Citation Index; Conference Proceedings Citation Index – Science; and Conference Proceedings Citation Index – Social Science and Humanities). The researcher focussed on articles published from 1945 to the present. The revised set of questions for Social Network Analysis is enclosed as Appendix 5.7.