Evaluation of Library and Information Services

A complete list of all titles in the series can be found at the back of this volume. Information Center at the Library Association, Toby Bainton at the Standing Conference of National and University Libraries (SCONUL), Kate Cole at Westminster Libraries &. Declining government spending in the 80s was exacerbated by economic recession, which encouraged consumers to spend more carefully and look critically at the goods and services they purchased.

Introduction, reasons for evaluation and related factors

While this makes it possible to redeploy staff to the 'front of house', the impact on the services users receive needs to be assessed. The Internet has created 'learning centres', indifferent library types whose use requires assessment, and a new type of user, the 'lifelong learner', whose study and IT skills may have been overestimated by the UK government. The service must be suitable for the needs of the user, which, in turn, raises the question: What is a user.

Identifying Performance Issues for Evaluation

It links library evaluation with the Citizen's Charter and gives practical examples of surveys and performance indicators. Determine the need for library plans, benchmarking of key performance areas and a list of key performance indicators. It contains 25 definitions of standard terms used with performance indicators and the criteria a performance indicator must meet.

Quantitative methods

They are insufficiently detailed and are likely to mask dissatisfaction with specific areas of the service. Despite this, respondents criticized what they perceived as jargon in the terminology of the questions. This procedure is known as gathering inferential statistics, which attempts to make statements about the parent population based on the evidence from the sample.

The starting point is chosen randomly and then every nth number, fourth, tenth or whatever depending on the desired sample size. This can be quite a large number in itself and may not be possible; in this case, at least part of the sample should be examined. Interviewers should have ID badges and it may be wise to notify the police of their intended work.

In academic libraries, the time chosen may be related to the nature of the intended work, but the beginning of the calendar year is a good time. They can hand out forms to users who enter the library and explain the purpose of the study when asked. Summaries of the report, in the form of a press release, can be sent to the local newspaper.

In special libraries, the report can be summarized in a house magazine or newsletter if one exists and a copy of the report can be sent to the governing body.

Qualitative methods

In higher education, they can be used to 'float' new ideas, such as embarking on a major fundraising venture. Planning can be a real problem and if people cannot be recruited it can threaten the bottom line. If the module contains a stop word list of obscenities and offensive expressions, these can be filtered out.

Nevertheless, they are a good way of collecting data that is difficult to collect in any other way, provided that the data collected can be put into some kind of context. It can be seen as an extension of the meeting method, but by talking to just one person, the experiences and reactions of the respondents can be studied in detail. An ethical question arises, as observation without consent can be interpreted as an invasion of privacy.

Observation can be very time consuming and finally the subjectivity of the observer must be taken into account. This helps to distance the observer from them, an important way to address the questions of reliability and validity. It may be necessary to mount a further study of the group to find out why.

The consequences in the form of angry letters to local newspapers or articles in student magazines can be serious, and it may be necessary to launch a PR exercise to clarify the situation.

Pitfalls and Progress

Completion of a piece of survey work does not necessarily mark the end of the exercise. The results should include recommendations for action, but a number of factors can influence the results of the study and even lead to their modification. It can be difficult to gather information on which to base decision-making, as the Brent study showed.

If the cooperation and support of departments outside the library is necessary, e.g. manual staff support for extended opening hours, it may be difficult to proceed, but at least solid data is available to make a case. These may be the results of faulty methodology or too much attention to pressure from vocal interest groups. It may be necessary to revisit the same problem areas on a regular basis to identify new and changing needs. zv69.

Over a period of years or years it should be possible to develop a systematic regime of .. evaluation consisting of questionnaire work, formal qualitative methods such as focus groups and other meetings, supporting methods such as suggestion boxes and comparisons with other departments within the institution . However, it is difficult to compare services in different institutions that are very qualitative, such as the Surveys service. Measures or benchmarks must be chosen to compare libraries and it can be difficult to find multiple libraries for which the same set of measures is suitable.

While these pilots have proven useful, it has proven difficult to devise generally acceptable measures and due to ethical and confidentiality issues, the results have not been widely reported in the literature.

Case Studies and Future Developments

In response to the question—However, have you used the library service since you joined?—only 25% answered 'regularly' while 65% answered 'a few times'. University College Swansea Library used the Van House originals independently of the SCONUL initiative and Glasgow Caledonian University Library used the Van House general user satisfaction survey in an increasingly modified form. As a result of various research initiatives, a modified version of the Van House user satisfaction survey questionnaire was designed at Glasgow Caledonian University.

The list of questions reflected in section 1 of the Van House questionnaire had not been found to be particularly relevant and the questionnaire proved difficult to analyze satisfactorily. The library at the University of Munster in northwestern Germany is one of the most active in Germany in the field of evaluation and performance measurement. Perhaps, because of the relative rarity of surveying, it was a substantial and methodologically complex exercise.

Surveying special libraries is quite different from public and academic libraries because of the small number of users involved. The Scottish Agricultural Sciences Agency library carried out an unpublished study of its library use in zv89 January 1996. One of the outcomes of the evaluation program was a library service charter which includes a commitment to monitoring and customer feedback.

There are general higher education charters for England and Scotland, and the National Union of Students has developed a student charter which states, among other things, that students must have 'the right to effective learning supports'. Among the specific library charters, one of the most attractively produced is that of the University of London Library. A mass observation of the Contemporary Public Library by Leeds Metropolitan University: June 1999 to 31 March 2000 (funded by the Libraries and Information Commission).

Figure 3 is an example of a modified proforma, derived from Van House by SCONUL and apparently originating from the mythical Poppleton Metropolitan University so

Performance Measurement for the Electronic Library

The electronic library redefines the concept of the user and introduces the idea of the. They are extremely practical and cover essential areas, but should be considered in light of the largely qualitative issues raised above. The proposed performance indicators will be supported by user surveys to collect qualitative information that will complement the numerical nature of the set of performance indicators.

The distinctive mission of the library's electronic information floor (EIF) was not clear to users who saw it simply as a collection of computers in the library. Much of its use was unsophisticated and focused on email, the Internet, and word processing. Much of the learning was from other students and not from the library or other staff.

Whether this will change over time is one of the biggest problems in the evaluation of the electronic library. However, observation requires considerable IT expertise, as the observer must be able to recognize which service is being used and form some impressions of the user's IT skills. While the use of a variety of techniques facilitates comparisons (triangulation) and consequently validates the data, systematic evaluation of the electronic library can prove to be extremely labor-intensive due to the skills required and the importance of human-based qualitative studies.

The past few years have seen the spread of the performance indicator movement and the prescription of reliable tested performance indicators along with methods for their use.