Completing All modesAll modes

The four stages of test administration

Stage 4: Completing All modesAll modes

*23. Use norm tables to find relevant percentile and/or standard scores and complete candidates’ records.

Stage 4: Completing All modes

24. Keep all test materials and test data in a secure place and ensure that access is not given to unauthorized personnel.

25. Ensure that all mandatory requirements relating to candidates’ and clients’ rights and obligations under the Data Protection Act have been clearly explained to all parties (i.e. clients and candidates).

26. Ensure that data are stored according to the requirements of the Data Protection Act.

* Some or all of the items marked with an asterisk may not be necessary for computer-based assessment systems.

TABLE 2.1.1: continued

?

Stage 1: Preparation

Issues to consider

OPEN QUESTION

Imagine that you are invited to take part in a testing session as a candidate.

The session may be a part of your professional development or it may be a part of a job selection process. Whichever you have imagined, use a few seconds now to jot down some of the things you would like to know, in advance, about the session.

1. ...

2. ...

3. ...

4. ...

5. ...

•

Perhaps you thought about practical things. Will I be able to do the test online at home, or will I have to go somewhere to be tested? Where will the session be held? How long will it take? Will I need to do any pre-paration before the test? If I have to go somewhere, will there be a need to bring anything to the session?

•

Or maybe you thought about the tests themselves. What will they be about? Will they need any special knowledge? Will they be relevant to what they are being used for? Who will see the answer sheets afterwards?

•

Or, as you now know a bit more about testing, maybe you thought about some of the technical properties of the tests. Will the tests be reliable, valid and have appropriate norms?

Whatever questions have occurred to you, there is no reason to believe that anyone you test in future will not also be asking these, and more, questions before they take their test.

It is the responsibility of the test user to choose which tests are to be used and to see that prospective candidates are given as much information beforehand as it is possible and ethical to give. As test administrator, you are likely to be involved in helping the test user with the second of these responsibilities.

The preparations that the test user has to make begin at the point at which the decision is made to administer the test.

There are a number of issues that test users have to consider when deciding whether or not to use psychometric tests in a particular context:

1. Decide what information is needed for, or from, the candidates and whether or not psychometric testing will provide some of that information.

TEST ADMINISTRATION 69

2. If it will, look to see which tests have the potential to provide the information you need.

3. Look at the technical properties and the practicalities

associated with each of the possible tests and decide which is most appropriate for your purposes. The test manual is the most likely source of this information.

As test administrator, on behalf of the test user, you will need to:

1. Check what materials you will need for the session and see whether there are sufficient supplies in stock if this is a test you already use; if not, decide what you need to order. Do not make photocopies of test mate-rials: it is illegal. If you are using computer-based tests that require release codes or certain numbers of administration units to be pre-purchased, you will need to ensure that adequate provision has been made or submit the necessary orders in good time.

2. Check what equipment (from stopwatches and pens to computers and workbenches, depending on the test and its mode of administra-tion) and conditions are required and whether these can be made available. This information will be available in the test manual.

3. On the basis of the information you have collected, decide how many candidates you are able to test in one session, how many staff you may need to support you, and how long each test session will last.

Be aware that if you are testing more than 15 people in one group session you will probably need some assistance in distributing and col-lecting materials and in monitoring the candidates during the test.

4. Plan the session, book the rooms you will require, order any materials you need, and book the staff time that will be needed.

5. Write to the candidates to invite them to the session and to inform them about it.

Write to brief them about any requirement for online Controlled Mode testing, explaining the log-in procedures and support availability.

There are, in addition, a number of issues need to be considered before using tests with any individual candidate.

Dealing with candidates who have specific problems

Psychometric tests are instruments which are designed to measure the strength of or the amount of a person’s specified attributes or traits. It is important to make sure that the measurement of these attributes is not being adversely affected by difficulties in coping with the ‘mechanics’ of the test, for example in understanding the instructions or in recording the responses. Before you use tests on anybody you should check that the per-son has the necessary numeracy and literacy skills to understand what is required and to undertake the test, and that they do not have any disabilities that might make it difficult or impossible for them to complete the test.

As a test administrator you should try to ensure that all candidates are pro-vided with sufficient information about the tests they will be required to take and the modes of administration. If they are, then they can be asked to notify you of any conditions which they believe might make taking the tests

unfair or unreasonable for them. You should then raise these issues with the qualified test user to see whether any dispensations or special provisions need to be made.

•

You should never make modifications to a test without the prior authority of the qualified test user.

•

You should never deviate from the standardized procedures without authority from the qualified test user.

•

Not every disability will affect the standardized testing conditions.

Where one does, every circumstance will be different.

•

Any authorized modifications to the test or the conditions of admin-istration and the reasons for them should be recorded in the Test Session Log.

Candidates with numeracy, literacy and language difficulties Many tests assume a reasonably high level of literacy on the part of the test taker. This can affect both tests of language-related skills and the measurement of skills which do not require literacy: for example, tests of manual dexterity with written instructions. Those for whom English is a second language may have particular problems with the language used in tests and test instructions. The qualified test user may be able to make provision for foreign language versions of the tests to be used and for a foreign language-speaking person to assist with the administration.

Clearly, such tests should not be used with people who have literacy or language problems, unless the job requires verbal reasoning skills and this is what is being tested.

The same is true of numeracy difficulties: if a test is designed to measure some attribute other than numeracy, but requires the test taker to be numerate in order to complete the test, then it would have an unfair impact on those with numeracy problems.

Candidates with physical or sensory disabilities

As with language, literacy and numeracy problems, it is important when testing candidates with physical or sensory disabilities to make sure that the ability being tested is not affected by difficulties in physically coping with the instructions, or by problems in recording their responses. For example, a candidate with a hearing impairment may be disadvantaged with instructions that are given orally, whilst a candidate with visual impairment may be disadvantaged by written instructions. Written responses might cause similar difficulties for those with visual impairments.

Clients with hand–eye co-ordination problems, hand tremors, etc.

may have difficulty in writing their responses or making accurate use of a multiple-choice format answer sheet. Such problems may result in them taking longer than other candidates physically to complete their responses, with the consequence that they may be especially disadvantaged in speeded tests.

Sometimes the disability can be coped with within the standardized administration procedures. For example, if the instructions are to

TEST ADMINISTRATION 71

be read aloud by the administrator and simultaneously read by the candidate, provided there is not also a performance or recording problem, the candidate with a hearing impairment or a visual impairment may not be disadvantaged.

But it is sometimes impossible to get a true measure of the candidate’s ability within the standardized administration procedure of the test.

If a candidate does present a problem because of disability, there are a number of things that the test user needs to consider: whether special provision can be made in terms of equipment, whether some adjustment in terms of time is appropriate and permissible, whether there are alternative forms of the test available (for example in Braille for blind people).

•

If the disability is likely to affect the standardized conditions, the test user should take advice from the test publishers about the extent to which changes can be made and how the

interpretation of the results might be affected.

For computer-based testing or tests delivered over the internet you need to check whether it is appropriate to use the various ‘accessibility’ functions built in to some operating systems. It is often a relatively simple matter to increase font size, for example, but doing this might have adverse effects on the test. As in all other cases, do not make adjustments unless authorized to do so by the responsible test user.

Planning the session

When planning a supervised test session there are lots of things to do and details to remember so you need to be systematic and well organized.

Exercise 2.1.1 will give you practice in planning a supervised test session.

The checklist in Table 2.1.2 describes what you will need to do prior to the test session. It is planned as if you had three weeks in which to make the arrangements. Ideally that would be the case, but if you have less time these points still need to be covered in the time that you do have available. ( This is a general-purpose checklist. In practice you may need to tailor it to fit your specific situation.)

If you have read through the whole of Module 2 once, attempt Exercises 2.1.1 and 2.1.2 now.

TABLE 2.1.2: Checklist for planning a test session About 3 weeks before:

– If you have not administered the test before, study the Manual, familiarize yourself with it and check the administration procedures.

– Check the Test Manual(s) to see what materials are required.

– Check to see if there are sufficient supplies of psychometric materials (for example, answer sheets, test booklets, release codes for computer administration of tests, etc.) in stock, and order any necessary extras.

– Check that the other materials (pencils, erasers, computers, stopwatches, etc.) are in stock and in working order.

– Book the rooms and equipment needed.

– Arrange for additional clerical and administrative support if required.

About 2 weeks before:

– Write to candidates inviting them to the test session. Include any practice materials (or information about how to access online practice materials) that the test publisher recommends (see Exercise 2.1.2).

– Retain a list of candidates to refer to when preparing the test log and name tags.

– Plan the timetable for the day.

About 3 days before:

– Assemble all materials and check everything is present (see the Test Pack, Section 3, for a general checklist of materials).

– Check that the room and equipment are booked.

– Organize the reception arrangements for the candidates.

– Brief any assistant(s) about their role on the day, making sure that they understand the importance of maintaining standard test conditions within the test session.

The day before the session:

– Prepare the log and the score sheet (if required).

– Refamiliarize yourself and your assistant(s) with the materials.

On the day:

– Prepare the room, the tables and materials and check arrangements for the candidates’ arrival.

– Put ‘Test in Progress’ signs on the door.

– Unplug any telephones and divert calls.

– Welcome candidates and follow the timetable.

Do not attempt these exercises until you have read through the whole of Module 2 once.

EXERCISE 2.1.1: Planning the test session

Scenario: A test session is being planned: it will be held in three weeks’ time to test 12 candidates. There is a room which is large enough to accommodate 20 tables and which is quiet and can be reserved for the session. Three com-puters can be accommodated in the room and can be free for the test session.

A full day is available in which to arrange the session. All the test materials can be ordered at short notice if they are not in stock. It has been decided that the tests are fair and suitable for the purpose for which they are to be used.

What has not been decided is which of the tests to use.

Using the administration instructions from two different manuals given below, plan two separate sessions, each for 12 candidates using the two tests described as the basis for the sessions.

TEST 1. This is a paper-and-pencil test of ability. It requires that candidates sit at individual tables placed at least three feet apart and all facing towards the administrator. The test has inbuilt practice questions which require that the administrator walks around the room and looks at the candidates’

answers to see that they are using the right boxes. The answers are read out to check that the examples are understood and questions are allowed up to this point. The Manual recommends that no more than eight candidates are tested in any one session unless there is a second administrator present TABLE 2.1.2: continued

TEST ADMINISTRATION 73 to assist. The test takes approximately 45 minutes: the administration of materials and practice session takes approximately 15 minutes and the timed test element is exactly 25 minutes long.

TEST 2. This ability test is administered by computer. Practice tests are available on the test publisher’s website (http://www.publishername.com/

practicetests). These may be openly accessed by candidates to help prepare them for the test by familiarizing them with the kinds of items that the test contains. It is recommended that the candidate takes this practice test before coming to the test session. The test itself has two example items to familiar-ize the candidate with the question format and the use of the computer keys. The test is timed and runs for 20 minutes exactly. It is suitable for group administration.

Plan sessions for each of these tests. Assume that materials will have to be ordered from the publisher. At this stage, do not write to the candidates, but include it in your planning.

EXERCISE 2.1.2: Inviting the candidate to the test session

As a result of your planning, the time scale and staff availability it is decided that only the computer-based test will be used. Write to one of the candidates inviting him or her to the first session and giving the following information:

•

Time, date and venue of the session.

•

Length of time the session will take.

•

As much information as you can to prepare the candidate for the tests:

send out practice leaflets or inform the test taker about available practice tests or direct test takers to appropriate internet testing practice sites.

•

Advance notice that they will not be permitted to receive or make calls or text messages during the test session.

•

A reminder to bring glasses if they need to wear them.

•

Information about the confidentiality of the tests – who will see the answers, what will happen to the data, and who will know the results (including the candidate); how feedback will be given.

•

A request for any special requirements that may be needed.

•

A request for the candidate’s signed permission to provide information from the test to the people it is intended for.

Now that you have thought about the advance preparation that is necessary for a testing session, you can plan for the administration of Test A in the Test Pack. Whilst this will be on an individual basis or for a small group, there is still a need to plan ahead.

A test administrator needs to be calm, friendly but business-like. Good advance preparation and planning ensures that the administrator is in con-trol. A second aspect is to consider the personal preparation that you need to be a good test administrator.

Preparation of yourself as administrator

One of the best preparations for administering a test is to have done the test yourself. This is essential if you are about to administer a test that is new to you. Once you have administered the test and you are familiar with it, it is still good practice to re-read the instructions and think about the requirements before the test administration.

If you have read through the whole of Module 2, attempt Exercise 2.1.3 now.

EXERCISE 2.1.3: Familiarizing yourself with the Test Pack materials

Turn to the Test Pack and take out one of the Test A test booklets (Test Pack, Section 2) and the Administration Instructions (Test Pack, Section 3). Test A is an ability test.

Turn to the Administration Instructions. Provide yourself with the relevant materials from the General Checklist of Materials (Test Pack, Section 3). Then read through the Administration Instructions. When you are certain about the examples, check your pencils, start the stopwatch and begin the test. Do not worry about the timing. The stopwatch will give you an indication of how long the test takes for you to complete and should reassure you about the timing.

Complete the test and leave the scoring for the moment.

Turn to the Test Pack and take out one of the P5 Personality Inventory test booklets (Test Pack, Section 2). The administration instructions are part of the test booklet. Provide yourself with the relevant materials from the General checklist of Materials (Test Pack, Section 3). Then read through the Administration Instructions. When you are certain about the examples, check your pencils and begin the test. This is an untimed test, but you might want to set the stopwatch running when you start to see how long you take to complete the questionnaire.

Complete the test and leave the scoring for the moment.

Now you can begin to plan your test session. To do this you can use the blank planning schedule below.

Choose three people to act as test takers and plan the testing session for them.

You may wish to test all three together or use three separate administrations;

much will depend (as in any testing session) on your situation. Do remember that these tests have been written for training purposes only and that you cannot give your candidates meaningful feedback. Make this clear to the candidates both in your advance preparation and later in your preamble to the testing session.

Do not invite your candidate(s) to a testing session until you have read through this Module at least once, and you have thoroughly familiarized yourself with the materials and procedure.

Planning schedule

Use the following headings to plan your schedule (see Table 2.1.1 and Exercise 2.1.1).