serVICe enVIronMent FACtors AFFeCtIng tHe FUtUre oF
AssessMent In ACADeMIC LIBrArIes
Larry Nash White
Reviewing the literature on the assessment of library service provides the reader with a sense of an uncertain journey. Assessment in libraries has a documented history, “beginning in 1876, with Cutter using cost benefit analysis in a study of cataloging effectiveness. The traditional techniques of performance measure- ment developed during the early history of performance measurement in librar- ies, which include interviews, input/output analysis, costs analysis, and activity analysis, are still the most popular forms of performance measurement today”
(White 2002, 28–29).
Lubans (1975, 2) stated that Shaw and Rider were two of the early pioneers in performance assessment innovation as they introduced “scientific management”
and a “total system approach” to the operation and evaluation of library service.
“Rider was quoted (in the 1930s) as saying that if librarians did not use scientific management or cost benefit analysis to justify performance, non-librarians would come in and do it for us” (White 2002, 29).
The literature reveals that between the 1930s and the 1980s, the assess- ment of library services generally remained static in the types of processes and techniques employed, with little if any innovation. Because of this lack of in- novation and increasing concerns and demands from stakeholders beginning in the late 1980s, libraries began to incorporate a limited number of assessment processes and techniques from outside the library profession (i.e., business and education). These techniques included a variety of methods, measurement styles, and processes to assess performance and service, including benchmarking, out- comes assessment, best practices, LibQUAL™, the Balanced Score Card, and other similar quantitative measurements. A continued review of the literature does not reveal any examples of major new processes and techniques originated and implemented from within the library profession during this time. Thus, it can
be concluded that the limited, primary innovations that have been implemented were from outside the library profession. Rider’s prediction of the future of library assessment has become truer than he or we may prefer.
White (2002) states that the absence of assessment innovation within li- braries is based in large part on “the lack of consensus among members of the profession as to what performance measures should be used and the reasons for their use” (30). As a combined result of a lack of direction from within our profession and a lack of consensus as to future directions, the assessment of library services is experiencing little progress and has little support for the need for innovation in the future. These resulting conditions have left many librar- ies on their own to develop new methods or have left them dependent on their stakeholders and their service environments to provide them with assessment techniques.
oVerVIeW oF AssessMent In ACADeMIC LIBrArIes
Today’s academic libraries operate in dynamic educational environments where providing services is constantly being reshaped by the settings in which they operate. Changes in customers, competitors, technologies, and stakeholders are creating powerful competitive forces. These forces are causing many educa- tional institutions to innovate or transition their services, identities, capabilities, and missions in order to respond, better meet stakeholder needs, and compete.
The impact on smaller and larger academic libraries is the same. The differences are in the scale and scope of the service environment–library interactions and the resources available in order to respond.
One of the strongest competitive service environment forces in education is assessment: assessment of quality, of value/impact, of efficiency and utiliza- tion, of mission support, and of meeting needs. An effective academic library is interwoven with its total environment, and the library shares many of the same service forces. (This may be even truer in the smaller academic library because of the increased blending of roles such a library often assumes.) Thus, assess- ment for academic libraries, just as for the whole of the educational setting, is becoming an increasingly difficult strategic challenge when it comes to providing evidence of accountability, efficiency, and the reporting of impact to stakeholders while maintaining alignment with their total surroundings. When this challenge is combined with a lack of innovation and consensus as to the future direction of assessment from within the library profession, academic libraries are left in the predicament of needing to be responsive and innovative yet are uncertain where this future lies.
oVerVIeW oF serVICe enVIronMent ForCes AFFeCtIng tHe FUtUre oF AssessMent
In attempting to provide insight into the future of assessment in smaller aca- demic libraries, one must first identify the variety of service environment forces
0
and then determine their effect. Through reviewing the literature, six factors were identified as primary indicators: collaboration, competition, distance edu- cation, mission/vision, technology, and assessment. These factors work indepen- dently and in combination with each other to directly affect the educational (and indirectly the academic library) context (see Figure 13.1). The following sections illustrate the theoretical interchanges between the six forces and provide the background for predicting possible future developments. Table 13.1 lists the specific impacts on assessment in academic libraries.
Technology
At the center of Figure 13.1 is technology. Technology serves as the hub of the service environment, as it is the only naturally neutral force. Technology facilitates and connects the other forces to allow for wider audience access and delivery. As a universal access point into the system, technology is frequently the common platform for innovation and implementation.
Figure 13.1
Service Environment Factors Affecting the Future of Assessment in Aca- demic Libraries (design by Larry Nash White, PhD. Copyright 2006. All rights reserved)
Summary of Service Environment Force Impacts on Assessment Service Environment
Forces Force Impacts on Academic Library Assessment Assessment
(general trends and developments)
“The evolution will continue . . .”
➢ Array of assessment tools/capabilities vs. a “silver bul- let.”
➢ Continuous development of assessment tools and re- porting capabilities.
➢ Culture of assessment will be required, not considered.
➢ Increased need by administrators for assessment data for data-driven decision making.
➢ Intangibles assessment (i.e., intellectual capital) will increase.
➢ Need to be less dependent on assessment innovations/
methods/processes from outside fields.
➢ More patron-mediated/patron-involved assessment processes.
➢ Need for assessment results to be reported to a wider audience.
➢ Need for/ability to frequently conduct assessment.
➢ Proactive assessment vs. mandated assessment.
➢ Resources for conducting assessment and training staff in assessment will increase and become a fixed part of the budget.
➢ Training in assessment will increase in need and value.
➢ Transitioning from evaluating existing service/prod- ucts to designing/valuing new services/products.
➢ Value assessment reporting will increase as output as- sessment reporting decreases.
Collaboration
“More faces to reach more places”
➢ Creates new service/strategic partners with differing/
new data types/assessment systems.
➢ Need for proactive assessment to identify new mar- kets/new needs/new services.
➢ New assessments needed for diverse customers with dynamic needs.
➢ New participants in assessment data collection, analy- sis, and reporting processes.
➢ New stakeholders requiring new assessment reporting of addressing needs and effectiveness.
➢ Wider audience to effectively communicate assess- ment results.
Competition
“Grow and go vs.
stop and drop”
➢ Increased need of awareness of service environment and competitors.
➢ Increased offensive/proactive strategic response assess- ment information needed.
(continued)
Service Environment
Forces
Force Impacts on Academic Library Assessment Competition (cont.) ➢ Less defensive/reactive strategic response assessment
information needed.
➢ Wider array of assessment tools/processes needed to accomplish an effective result.
Distance Education
“the new frontier”
➢ Evolving service delivery will require evolving assess- ment processes.
➢ New/nontraditional customers with differing needs to effectively identify and address.
➢ Ranges of assessment and impact reporting.
➢ Transitioning from internal only assessment to inclu- sive assessment.
➢ Wider area of service delivery/impact to collect data in and report assessment results.
Mission/vision
“new opportunities”
➢ Flexible assessment processes.
➢ Frequent assessment retooling and innovations.
➢ Increase need and use of environmental SWOT; value and effectiveness assessment information.
➢ Require assessment to serve more as a prognosticator of future needs/identifier of future missions.
Technology
“innovate and integrate”
➢ Assessment results available faster to wider audience.
➢ Increased need of assessment of technical capabil- ity/services.
➢ Increased use of technology in conducting assessment processes.
➢ Increased ability to assess intangible value.
➢ Integrated assessment processes in all organizational functions.
➢ More automated assessment processes.
➢ Need for/ability to frequently conduct assessment.
➢ Need to use assessment results faster to create strate- gic and other values.
Assessment
Impact from the general field of assessment on academic library evaluation will create multiple “paradigm shifts without a clutch” as it sparks innovations, questions, and opportunities. In Figure 13.1, the reader will note that assessment’s impact on the academic service environment is interconnected between compe- Table 13.1 (continued)
tition and mission/vision via technology. Using the wider access and integra- tion of technology, assessment processes will become more utilized for outward, proactive-based assessments that will identify competitor activities (i.e., envi- ronmental scanning) while providing more frequent, faster access to a wider audi- ence of the library’s assessment processes and results. Technology will allow more participants from outside academic library facilities to participate in data collec- tion, analysis, and dissemination while delivering proactive strategic information in real-time status for assessing, utilizing, and aligning the mission/vision of the library with its educational setting and its mission/vision.
As stated earlier, our profession adapted assessment procedures from outside the field of libraries. At present, the more frequently adapted ones focus on short- term quantitative data regarding customer impact or on the long-term affects of effort (i.e., outcomes), both of which are reactive in nature and not always aligned.
The nonlibrary evaluation processes and techniques often require the imple- mentation of substantial adaptations within the library setting and may still not deliver the specific information academic libraries require to address stakeholder questions of accountability and effectiveness. Emphasis on academic libraries’
ability to develop procedures for collecting and reporting strategic data to address accountability will be a priority from within the library profession.
Mission/Vision
The mission/vision impacts displayed in Figure 13.1 interconnect the general assessment forces and the “distance education” forces within the service environ- ment. Having increased assessment information and access will provide proactive data and direction to a wider audience of stakeholders for developing, adjust- ing, and aligning the “mission/vision” of the educational (and library) service environment to address needs. As more educational providers use technology to deliver distance education to users in new markets as a component of their mis- sion/vision via library services, academic libraries will be increasingly called on by stakeholders to proactively assess how these distance education library services promote, align, or generate the library’s abilities to support institutional goals.
The dynamic nature and transformation of the academic organization’s mis- sion/vision will continue to demand assessment innovations. Flexible and quickly revised processes will be required by leaders and administrators to obtain critical information needed to develop effective strategic new missions/visions that sup- port the overall institution.
Distance Education
Distance education forces interconnect mission/vision forces and collabora- tion forces in Figure 13.1, using technology to align and provide access of the mission/vision of the educational institution to potential and existing collabo- rators. Distance education is creating new markets, customers, and stakehold-
ers and will cause library assessment processes and techniques to become more outwardly focused and self-mediated and will increase the numbers of customers, information requirement, and the geographic area to be assessed. This increase in participants, needs, and areas of assessment will also require more open/acces- sible evaluation processes with wider ranges of reporting abilities that address a larger audience for assessment results, a greater diversity of evaluative data to be collected in order to provide feedback on collaborators’ impact and effectiveness in addressing the mission/vision of the education service environment, and iden- tifying desired or unproductive collaborators to fill or correct strategic needs.
Collaboration
Collaboration is displayed in Figure 13.1 interconnecting the distance ed- ucation and competition service environment force impacts. As the distance education technology force becomes more necessary and intertwined with the institution and library, collaboration will increase the scope, access, location, and capability of the resources, individuals, and groups required or available to adequately address service challenges.
The increased amounts of cooperation within and between service environ- ments will create additional assessment challenges for academic libraries. Assess- ment processes and techniques will need to be highly flexible and customizable in order to adapt to a larger, dynamic set of collaborators (many being unfamiliar with library services and their evaluation) who will bring their own assessment processes and data needs into the service environment. Collaboration will also require academic library assessment to proactively expand outside the facility and into new settings that have their own forces and competitors. These will require accountability through proactive offensive (i.e., assessment that promotes or pre- vents) and defensive assessment (i.e., assessment that reaffirms and validates).
Competition
Competition is the final interconnecting service environment force depicted in Figure 13.1, connecting collaboration and assessment. By increasing the col- laboration within the educational and library service environments, the amount of competitive forces is increased as well. Each new collaborator contributes new resources and opportunities while also bringing competitors into the service envi- ronment. Technology provides the competition force with a fast access point; this access is used to identify the service environment status and collaborators and to identify weaknesses and opportunities.
Therefore, the competition force will require assessment in academic libraries that efficiently creates proactive information about the environmental competi- tors and their potential impact on the service environment. Innovations must focus on developing a wide array of offensive assessment processes and techniques that will overlap in addressing stakeholder accountability and become less depen- dent on reactive defensive assessment processes and techniques.
ConCLUsIons
The future of assessment in smaller academic libraries is foreboding for some because there is a lack of direction and a need for innovative processes and techniques, yet the future of assessment in smaller academic libraries is also golden for others, as there is an assessment research and application horizon wait- ing to be explored and utilized. In order to provide the innovations necessary to effectively respond and report pertinent evaluation to stakeholders, academic libraries will have to become more independent designers, implementers, and users of assessment than ever before.
The key to the future of effective appraisal in academic libraries will be the ability to overcome the lack of consensus in assessment directions, the lack of innovations in this area from within the library profession itself, and the need for developing processes to allow smaller academic libraries to closely identify with their institutions’ need to dynamically transition into the future.
reFerenCes
Lubans, J. J., and Edward A. Chapman, eds. 1975. Reader in Library Systems Analysis: Reader Series in Library and Information Science. Englewood, Colo.: Microcard Edition Books.
White, Larry Nash. 2002. “Does Counting Count: An Evaluative Study of the Use and Impact of Performance Measurement in Florida Public Libraries.” PhD diss., University of Florida.