10
Automating Assessment Instruments
Jay J. Patel and Teresa Y. Neely
with Jessame Ferguson
A
utomating a survey instrument involves a wide range of issues, fromcampus. Campus departments submit a job description and students apply for specific jobs. The graduate school distributes completed applications to participating departments, and if selected, students are paid through the graduate school. When we wanted to hire someone to automate the UMBC’s assessment instrument, we found a graduate student funded through this program.
Students, both graduate and undergraduate, should not be overlooked as a source for the technological expertise needed to pursue your information literacy program and agenda. In fact, an undergraduate student (double) ma- joring in theater and computer science was responsible for the animation and programming of the Data Game at Colorado State University,1and an under- graduate student majoring in computer science was responsible for au- tomating the instrument and building the accompanying database for Neely’s 2000 dissertation.2
In selecting staff for instrument automation, it is important to look for a skill set that combines good web programming skills with sufficient knowl- edge of relational databases. The staff member should also possess excellent interpersonal and communications skills in order to interact with library fac- ulty, staff, and others from various campus departments. The UMBC Task Force hired a graduate student in computer science to automate the instru- ment and participate as a task force member. In this position, he interacted on a regular basis not only with the task force and reference department but also with campus employees from the Office of Information Technology (in- cluding the chief information officer), the director of the library, library fac- ulty, instruction librarians from other University System of Maryland cam- puses, and the director of the Office of Faculty Development at UMBC.
The following job description was designed specifically for the position:
The Student Library Associate will assist reference librarians by providing support and technical expertise to various reference projects, including developing the technical aspect of Web-based user interfaces, tutorials, and stand alone modules for information literacy—content will be developed by librarians; designing and developing various reference-related databases (statistics, survey data, etc.); and reviewing, updating, and maintaining Web pages for reference, instruction, and other areas as needed. Associates can expect to be hired based on their skill set relating to an individual reference project.
Automating Assessment Instruments 173
The Associate should have a basic understanding of how infor- mation is organized, relational database development, technical competence with computers and databases, word processing, HTML, and production software. The associate should also be mo- tivated, able to work in a team-based collaborative environment or independently, possess excellent oral and written communication skills, display demonstrated professionalism, and good judgment skills, and have excellent interpersonal skills.
When writing job descriptions, it is important to ask for what you need.
Once you know what types of skills you require for instrument automation, it is critical to put those in the job description. If you are unsure of the exact skills needed, talk to the head of library computing services or the equivalent in your library. Personnel from this department may also be willing to assist you when interviewing for this position.
This particular job description elicited responses from potential candi- dates with technical backgrounds as well as those with graphic design back- grounds. While you might be tempted to hire someone with the graphic de- sign background to ensure that the website or instrument will be aesthetically pleasing, you must make your decision based on functionality and the tech- nological needs of your particular program. The job description posted by the UMBC Task Force covers a lot of ground, including the development of web-based interfaces for a variety of projects (tutorials, modules), database development (specifically relational), and web page development. Of specific concern for the UMBC Survey, the task force decided that they needed someone who had a basic understanding of how information is organized—a basic tenet of working in libraries. The task force also wanted someone who had great interpersonal and communication skills and the ability to work alone, as well as in a team environment.
Once the job description is finalized and the applicant selected, it is im- portant to treat the new hire as an integral part of the group. At the Univer- sity of Maryland, Baltimore County, the graduate student assistant (GSA) op- erated as a full member of the task force. He attended regularly scheduled meetings and provided feedback and input during instrument development.
This is important. Instrument automation should not be isolated from the de- velopment process, and the person responsible for automating the instru- ment should know the overall goals and objectives for instrumentation. By at- tending meetings, the GSA had the opportunity to understand the kind of 174 Automating Assessment Instruments
data being collected and why, which helped him develop a database that would support the goals and objectives of the assessment. Similarly, the task force, as the developers of the instrument, needed to know about the possi- bilities and limitations of the web-based medium they had chosen for instru- ment development and implementation. For example, if designed improp- erly, test takers’ use of the back button of a web browser may cause a set of responses to be submitted multiple times. Keeping the GSA in the loop en- sures that the transition from a paper-based instrument to a web-based one is smooth.