The BPS Occupational Test

(1)

Psychological Testing:

The BPS Occupational Test

Administration Open Learning Programme

David Bartram and Patricia A. Lindley

(2)

BLACKWELL PUBLISHING

350 Main Street, Malden, MA 02148–5020, USA 9600 Garsington Road, Oxford OX4 2DQ, UK

550 Swanston Street, Carlton, Victoria 3053, Australia

The right of David Bartram and Patricia A. Lindley to be identified as the Authors of this Work has been asserted in accordance with the UK Copyright, Designs, and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK, Copyright, Designs, and Patents Act 1988, without the prior permission of the publisher.

First published 2006 by The British Psychological Society and Blackwell Publishing Ltd

1 2006

Library of Congress Cataloging-in-Publication Data Bartram, David.

Psychological testing : the BPS Occupational Test Administration Open Learning Programme / David Bartram and Patricia A. Lindley.

p. cm.

ISBN-13: 978–1–4051–3107–0 (hardcover : alk. paper)

1. Psychological tests —Study guides. I. Lindley, Patricia A.

II. British Psychological Society. III. Title.

BF176.B38 2006 150.28′7—dc22

2005033630 A catalogue record for this title is available from the British Library.

Set in 10.5/12pt Cheltenham by Graphicraft Limited, Hong Kong Printed and bound in the UK by Athenaeum Press

The publisher’s policy is to use permanent paper from mills that operate a sustainable forestry policy, and which has been anufactured from pulp processed using acid-free and elementary chlorine-free practices. Furthermore, the publisher ensures that the text paper and cover board used have met acceptable environmental accreditation standards.

For further information on

Blackwell Publishing, visit our website:

www.blackwellpublishing.com

(3)

INTRODUCTION AND STUDY GUIDE

...

1

Introduction to test administration ... 1

What do test administrators do? ... 2

Limits on the range of tests that may be administered ... 3

Keeping up to date ... 3

Summary ... 3

The BPS Certificate in Test Administration ... 4

The BPS test administration standards ... 4

What does the Certificate provide? ... 5

How is the Certificate obtained? ... 5

Becoming a test administrator ... 5

Acquire the necessary competence ... 6

Have your competence assessed ... 6

Checklist of materials in the BPS Open Learning Modules pack ... 6

What you will need to meet the test administration assessment requirements ... 7

How to use the Open Learning Modules ... 7

General hints on self-study methods ... 7

How to organize your work ... 8

Build up your portfolio of work for assessment ... 8

MODULE 1: AN INTRODUCTION TO PSYCHOLOGICAL TESTING FOR TEST ADMINISTRATORS

...

9

OVERVIEW ... 9

KEY AIMS ... 9

INTRODUCTION ... 9

About these modules ... 10

1.1 WHAT ARE PSYCHOLOGICAL TESTS? ... 11

What is a ‘psychological test’? ... 11

Test manuals ... 13

What do tests measure? ... 14

Summary ... 15

1.2 MAIN TYPES OF PSYCHOLOGICAL TEST ... 16

Measures of typical performance ... 16

Measures of maximum performance ... 18

Differences between ability and attainment tests ... 28

(4)

vi CONTENTS

Work samples, trainability tests and job simulations ... 29

Dexterity tests ... 30

Apparatus tests: issues for test administration ... 30

Computer Adaptive Testing ... 31

1.3 CONTROLLING THE QUALITY OF PSYCHOLOGICAL TESTING ... 33

Scope: norms and the process of referencing scores ... 34

Acceptability ... 36

Practicality ... 36

Fairness ... 37

SUMMARY OF MODULES 1.1 TO 1.3 ... 38

1.4 SCALES AND MEASUREMENT ... 40

Raw scores ... 40

Scales ... 41

Absolute and relative scores: raw scores and normative scores ... 43

Norm-referenced scores -- comparing people with other people ... 44

Self-referenced or ipsative tests -- comparing people with themselves ... 45

1.5 UNDERSTANDING AND USING TEST NORMS ... 47

Frequency distributions ... 47

Percentiles ... 47

Descriptive measures based on percentiles ... 48

Other standard scores ... 50

T-scores and sten scores ... 51

SUMMARY OF MODULES 1.4 and 1.5 ... 53

MODULE 1: ANSWERS TO EXERCISES AND SELF-ASSESSMENT QUESTIONS ... 54

Answers to Exercises ... 54

Exercise 1.5.1: Converting scores to percentiles and grades ... 54

Exercise 1.5.2: Using norm tables ... 54

Answers to Self-Assessment Questions ... 55

SAQ 1.2.1 ... 55

SAQ 1.2.2 ... 56

SAQ 1.4.1 ... 56

SAQ 1.5.1: Converting raw scores to percentiles ... 56

MODULE 2. TEST ADMINISTRATION AND SCORING

...

57

OVERVIEW ... 57

KEY AIMS ... 58

(5)

2.1 TEST ADMINISTRATION ... 59

High-stakes and low-stakes testing ... 60

What are the functions of the test administrator role? ... 60

Modes of test administration ... 61

Open Mode ... 61

Controlled Mode ... 62

Supervised Mode ... 63

Managed Mode ... 63

The International Test Commission (ITC) Guidelines ... 64

The four stages of test administration ... 66

Stage 1: Preparation ... 68

Issues to consider ... 68

Dealing with candidates who have specific problems ... 69

Planning the session ... 71

Preparation of yourself as administrator ... 74

Planning Schedule ... 74

Preparation for computer-based testing sessions ... 75

Stage 2: Administration ... 76

The Test Session Log ... 76

Introducing the test session ... 77

Administration of the test ... 78

Computer-based test (CBT ) administration and administration of tests over the internet ... 79

Detailing the level of control over the test conditions ... 79

Stage 3: Scoring ... 80

Checking answer sheets ... 80

Hand-scoring ... 80

Other scoring procedures ... 81

Converting raw scores to standard scores and percentiles ... 83

Summary of points to note when scoring tests ... 84

Stage 4: Completing the administration procedures ... 84

Paper-and-pencil materials ... 84

Computer materials ... 84

Record-keeping, monitoring and follow-up ... 84

2.2 ISSUES OF CONFIDENTIALITY AND SECURITY ... 85

Maintain the confidentiality of test-taker results ... 85

Security of test materials ... 85

Data Protection Act 1998 ... 86

Confidentiality of test data ... 87

(6)

viii CONTENTS

2.3 PUTTING IT INTO PRACTICE ... 89

Overview and practical task ... 90

2.4 FEEDBACK AND REPORTING ... 91

Preparation of scores for feedback ... 92

The test conditions ... 92

Comparisons with the performance of relevant others ... 92

Making arrangements for the feedback session ... 93

Helping in the preparation of reports ... 94

Generating computer-based reports ... 94

Creating summary reports ... 94

ENDPIECE ... 96

MODULE 2: ANSWERS TO EXERCISES AND SELF-ASSESSMENT QUESTIONS ... 97

Answers to Exercises ... 97

Exercise 2.1.1: Planning the session ... 97

Exercise 2.1.2: Inviting the candidate to the test session ... 99

Answers to Self-Assessment Questions ... 100

SAQ 2.1.1 ... 100

SAQ 2.3.1 ... 101

GLOSSARY ... 103

(7)

LIST OF FIGURES, TABLES AND EXERCISES

Figures

1.2.1 Example items from verbal, spatial, abstract

and numerical tests ... 20

1.2.2 The Crawford Small Parts Dexerity Test ... 26

1.2.3 Examples of computer-based tests ... 27

1.5.1 Correspondence between the normal distribution and a number of standard score-based scales ... 51

Tables 1.1.1 Some of the major differences between a psychological test and a set of questions ... 14

1.2.1 Types of item used in personality and interest inventories ... 18

1.2.2 Examples of the sort of items used in tests of maximum performance ... 19

1.2.3 Examples of apparatus tests ... 25

1.4.1 Example items for a mood inventory ... 41

1.4.2 Example items for a general ability test ... 42

1.4.3 Typical self-report inventory items ... 43

1.5.1 Example of a raw score to percentile conversion table ... 48

1.5.2 Commonly used standard score scales ... 50

2.1.1 Checklist of actions for the four stages ... 66

2.1.2 Checklist for planning a test session ... 71

Exercises 1.4.1 Self-administration of Test A ... 40

1.5.1 Converting raw scores to percentiles and grades ... 49

1.5.2 Using norm tables ... 52

2.1.1 Planning the test session ... 72

2.1.2 Inviting the candidate to the test session ... 73

2.1.3 Familiarizing yourself with the Test Pack materials ... 74

2.1.4 Introducing the session to the candidates ... 78

2.1.5 Checking the answer sheets ... 80

2.1.6 Scoring ... 81

2.1.7 Converting raw scores into percentiles and standard scores ... 83

2.3.1 Administering Tests A and B ... 90

2.4.1 Preparing information for the test user: case study 1 ... 93

2.4.2 Preparing information for the test user: case study 2 ... 95

(8)

1

Introduction and Study Guide

© No photocopying allowed

This introduction provides you with some guidelines on how to study, how to have your competence as a test user assessed and how to go about obtaining tests once you have attained competence.

Introduction to test administration

Good test administration is of vital importance in ensuring the quality of psychological testing and maximizing the value of the information obtained from tests.

In the past the role of the test administrator was relatively straightforward.

He or she would make the preparations for the test session, provide the instructions for testing to one or more people and supervise them while they completed the tests. He or she would then carry out various checks on the responses, do the scoring and produce the materials needed by the qualified test user to produce a report or provide feedback on the candidates’

performance.

While this still represents much of what test administrators do, with increasing use of computer-based testing and online delivery of tests, the role of the test administrator is changing and becoming more diverse. Not only do you need to know how to administer paper-and-pencil tests, but these days you need to be familiar with computer systems and with the internet. Many tests no longer need a test administrator to carry out scoring, but instead they need someone who can manipulate the scores from one computer database into another, or follow the procedure for generating a report from a set of test scores.

Before you start work on the Modules, please read through this introductory material. If you have not yet made any arrangements for being assessed for the British Psychological Society’s (BPS’s) Certificate of Competence in Occupational Testing (Test Administration) and wish to do so, then it is best to make these arrangements before you get too far into studying (see below for details).

(9)

It is assumed throughout these Modules that these basic operational skills either exist or can be learnt elsewhere. It is also assumed that any training for the specific skills and competences needed to operate particular software packages will be provided with that package. Online-based testing systems are generally easier to use than desktop systems, and assume that the user simply has general internet and web-browser operating skills.

What do test administrators do?

It is very important to know what, as a test administrator, you can do and what you cannot do. Understanding the limits of your competence is vital.

Qualification as a test administrator can be sought either as an end in itself or as a step on the road to qualification as a test user. In these units, when we refer to the ‘qualified test user’, we mean a person who has obtained a BPS Level A or B qualification, or has an equivalent level of competence, and who is therefore qualified to use a certain range of types of tests. For Level A this range is limited; for full Level B it can be extensive.

We can think of testing as involving a number of stages. In the table below you can see where the roles of the test administrator and test user differ.

Stage Test user Test administrator

Choosing whether to test or not Yes No

Choosing which test or tests to use Yes No

Managing the administration process Yes Yes, under direction of the test user Scoring the results and producing the Yes Yes, under direction materials necessary for interpretation of the test user

Interpreting the results Yes No

Providing written or oral feedback Yes No of the results

Evaluating the utility of the test in No No the longer term

As you can see, the test administrator’s role fits within a larger process and is one that is directed by the test user, who has ultimate responsibility for the testing. In effect, a qualified test user can delegate certain aspects of the testing process to test administrators. But the test user retains overall responsibility.

But it must be stressed that the role of test administration is important. It is important in two key respects:

1. It is the test administrator who is at the interface between the actual test and the test taker. How people perform on the test will have a lot more to do with how the test administrator operates than with the test user.

2. For the same reason, it is the test administrator who is the public face of the testing process. It is the test administrator who meets candidates for selection into an organization, and who has the task of represent- ing the organisation in a positive way.

(10)

INTRODUCTION AND STUDY GUIDE 3

Limits on the range of tests that may be administered

The BPS test administration qualification is not intended to cover all possible types of tests.

There are tests which require a high degree of knowledge of the test itself or of psychology to administer. Such tests are often quite interactive in the way in which they are administered. Examples would include batteries of ability measures like the British Ability Scales or the Weschler Adult Intelligence Scales. You will need to be guided by the qualified test user in terms of what tests you can administer for them, and what they will need to administer themselves. However, even for tests that require them to be involved in the actual administration, there are many supporting processes that you would still be able to manage – arranging the sessions, welcoming candidates, assisting with scoring and so on.

The BPS Occupational Testing standards are intended for test users who are working with ‘normal adults’.

•

By ‘normal’ we simply mean people being assessed in the normal run of things – for a new job, for guidance in career choice, for personal development and so on. Test administration in the areas of mental health or forensic testing requires specific skills and considerations that are not covered here – though the general test administration skills you will learn underpin all of these.

•

By ‘adult’ we are excluding the assessment of children (which comes under the general headings of either educational testing or clinical testing, depending on the purpose). We are also excluding testing of the elderly where that is being carried out for clinical or health reasons rather than simply in relation to work or lifestyle.

Keeping up to date

These Modules have been written with the future in view. They are based on the 2005 updates to the BPS Level A and Level B standards of competence in test use. The main respect in which they differ from the original standards is in the diversification of the role of test administration brought about by the increasing use of computer-based testing and testing on the internet.

Summary

Test administration is an essential part of competent test use. But it does not cover stages such as test choice, understanding the technical qualities of tests, or the interpretation and feedback of results. Consequently, as a test administrator you will not be responsible for these aspects of test use and will be expected to use tests only under the supervision of a qualified test user. If you reach Level A, you will be considered competent by the BPS to use certain types of tests on your own. From Level A onwards, you can start to increase the breadth and depth of your skills and understanding of testing and extend the range of tests you can use fully.

These Modules will enable you to set out on the path to test user qualification and take you along to the first important point: test administration.

(11)

The BPS Certificate in Occupational Test Administration

These Modules have been designed and are carefully structured to provide you with all the material and information you need to develop your basic competence as an administrator of psychological tests in occupational settings.

For practical purposes, the BPS has divided its specification of test user competence in occupational testing into two ‘levels’: Levels A and B. While the test administration qualification forms a meaningful qualification in its own right, it can also be used as a stepping stone to acquiring Level A and Level B test user qualifications. Level A defines the basic foundation skills and competence needed for the use of a limited range of types of test (those which are easier to interpret), including test administration. Level B extends this to cover the competences required for using most of the other psychological tests employed in occupational assessment (including measures of personality).

Level A is the starting point for progress on to Level B.

As with all professional development, there is no well-defined end point and there are many alternative routes one can take to achieving competence. Levels A and B mark points along a general developmental path. Some people may follow this path on beyond Level B, while others may choose to progress no further than test administration.

The BPS test administration standards

The Test Administration Certificate is based on a set of standards which relate to an individual’s ability:

•

to administer certain types of psychological test fairly and effectively within occupational settings (such as personnel selection, vocational guidance, management development);

•

to adhere to the codes of practice and professional conduct defined by the BPS and other relevant bodies (for example, the Chartered Institute of Personnel and Development).

The standards are defined by a detailed Checklist of Competences in Occupa- tional Testing (available from the BPS in the Test Administration General Informa- tion Pack which you can download at: http://www.psychtesting.org.uk), which specifies a range of knowledge and skills relating to the administration of a permitted range of types of psychological test in occupational settings.

It covers the following areas:

•

Relevant underpinning knowledge – especially concerning the nature of psychological testing.

•

Task skills – relating to the performance of test administration related activities.

•

Task management skills – required to achieve overall functional com- petence: organizing assessment procedures, control and security of materials, etc.

•

Contingency management skills – needed to deal with problems and difficulties, breakdowns in routine, candidates’ questions during test administration, etc.

•

Instrumental skills – relating to specific test administration modes and procedures, such as the use of computer-based test administration, remote administration over the internet, etc.

(12)

INTRODUCTION AND STUDY GUIDE 5

What does the Certificate provide?

Possession of the Test Administration Certificate provides evidence of your basic competence in certain areas of occupational testing. With a Test Administra- tion Certificate you are qualified to administer a wide range of attainment, ability, aptitude, personality, motivation and other tests under the supervision of a suitably qualified test user. Publishers generally classify their test materials in terms of the competence level required for their use. Most now use the BPS classification into Levels A and B, though this is not universal. You will not be eligible to purchase Level A or Level B tests in your own right.

It is important to note that the Test Administration Certificate does not constitute a qualification in psychology and does not confer any ‘psychologist’ status on the holder.

How is the Certificate obtained?

Any person who can provide sufficient evidence that they meet the standards required for all the items on the checklist of competences will be eligible to apply for a BPS Certificate of Competence in Occupational Testing (Test Administration).

To obtain the Certificate, your competence has to be assessed by someone who is recognized by the BPS as qualified to assess people for the Test Administra- tion Certificate. Assessment of competence is subject to a verification process carried out by the BPS which is designed to ensure that assessment is fair and that different assessors are not making very different demands. The BPS holds a register (the Register of Assessors) of these people, and you can obtain from the BPS a list of those who operate in your area. In some cases these are individual consultants, in others people working in consultancy companies.

The BPS provides guidelines on assessment for the Certificate and all registered assessors are subject to monitoring and quality checks by the BPS through its verification scheme. All assessors are Chartered Psychologists who themselves hold at least the Level A Certificate and who have expertise in occupational testing and assessment. You will need to contact one or more of the people on the Register of Assessors in your area to ask about their costs, what they can offer you and their availability.

Becoming a test administrator

The traditional route to becoming a test administrator has been to attend a one- or two-day course in test administration. This is still the fastest route.

However, training which is more spread out in time provides you with more time to absorb new ideas and concepts and to practise the skills and techniques.

While the BPS Test Administration Open Learning Modules can be used as teach- ing materials for conventional training courses, they have also been designed to offer an alternative option: flexible learning through self-study. This provides you with the freedom to pace your learning, it does not require as much time away from work, and it is far cheaper than a conventional training course.

There are two steps to becoming a competent test administrator through the BPS self-study route:

1. Acquire the necessary competence.

2. Have your competence assessed.

(13)

Acquire the necessary competence

You should be able to do most of this working on your own using these Open Learning Modules and associated materials. However, you may wish to have some extra guidance and help. Many of those in the Register of Assessors, who carry out competence assessments, also provide training and tutoring services.

At any point in your study, you can ask one of these people to be your tutor.

Some offer a telephone ‘helpline’ and face-to-face sessions (tutorials, small group workshops, back-up training sessions and so on). Naturally there will be a charge for any services, and you should ask for a clear statement of the charges before you proceed.

Have your competence assessed

As already stated, this has to be carried out by someone who is recognized by the BPS as a verified assessor.

Typically, the assessment will involve attending an assessment workshop for the assessment of your test administration skills and the submission of various items of evidence of your competence (your ‘portfolio’). The assessment process used by the authors of these Modules, for example, has included:

1. Written exercises to be completed in your own time and submitted for assessment.

2. Completion of all the Self-Assessment Questions and Exercises in these Open Learning Modules for inclusion in your portfolio.

3. A test administration assessment workshop in which you are observed carrying out one or more test administrations.

These are all items that can make up your portfolio (see below).

Checklist of materials in the BPS Open Learning Modules pack The complete Open Learning pack should contain:

•

Introduction to the Modules

•

Module 1: An Introduction to Psychological Testing for Test Administrators

•

Module 2: Test Administration and Scoring

•

Glossary of Terms

•

The Test Pack

•

The Assessment Portfolio

In addition, a copy of the General Information Pack for the Test Administra- tion Certificate is included. This provides detailed information about test administration and includes the Checklist of Competences in Occupational Testing ( Test Administration).

The Test Pack and the Assessment Portfolio contain sufficient copies of material for all your course work. All the material is protected by copyright and should not be reproduced by any means without the prior permission of the publisher. The authors and the publisher do permit the making of additional copies of the Test A and P5 Booklets, Test Session Logs and Candidate Evaluation Forms, as long as they are for your personal use only.

(14)

INTRODUCTION AND STUDY GUIDE 7 What you will need to meet the test administration assessment

requirements

To meet part of the assessment evidence requirements for the Test Adminis- tration Certificate you will need to have completed your Assessment Portfolio.

This contains all the SAQs and Exercises in the Modules.

If you want to obtain the Test Administration Certificate and are planning to use the Open Learning Modules for self-study, then you should consider registering with a verified assessor before you start your studies.

How to use the Open Learning Modules

General hints on self-study methods

This course covers all that you would normally cover in a two-day test administration course – and quite a lot more! Hence, the minimum amount of study time needed to master the essential parts of the material presented here is likely to be at least 12 or 18 hours. For most people, it would be more realistic to plan for about 15 to 20 hours. You should expect to master all of the materials within this time and be ready for your final assessment.

You will need to set aside time to work so you can concentrate well. If you are not used to self-study, then there are a number of points to note. The key to successful self-study is self-discipline, planning your time and establish- ing a clear ‘contract’ with those around you who will be affected by your studying.

•

If possible, work in a room which is away from telephones, televisions and other people.

•

Come to an agreement with those living with you that you are to be left alone during study periods. Agree when these are to be.

•

Plan your time. Work out a timetable now and treat it as if it were an evening class or some other formal commitment. Don’t expect to work by just grabbing odd opportunities when they arise – they won’t and you will become frustrated at your lack of progress.

•

It is better to have a regular time slot which you use whether you feel like it or not! Lots of very short periods or one or two very long ones are not so good. Psychologists have shown that learning is most efficient when study and practice periods are distributed across time in reasonable-sized chunks.

•

Ideally, each study period should be between one and three hours.

Within each period, try to change the type of activity you are engaged in as much as possible: reading, making notes, working out examples, etc. Everyone finds it difficult to keep their attention on their work for more than about half an hour at a stretch if there is no variety in what they are doing.

If you can, plan to spend two or three evenings during the week (say five hours in total) with half a day at the weekend (three hours). You should then be able to cover the full course in one to two weeks.

(15)

How to organize your work

Psychological testing involves some quite complex and technical concepts.

While you will not need to go into these in any detail for this qualification, some aspects are included in boxes to give you a good background in testing and to help you differentiate between test administration and test use.

1. Read through each Module quickly to get an overview – do not do the exercises or answer the questions.

2. Go back and work through carefully making notes as you go.

3. Carry out the practical work.

4. Complete the exercises and answer the questions.

It is best to work through the two Modules in sequence.

Build up your portfolio of work for assessment

All the self-assessment questions (SAQs) and exercises given in the Modules are duplicated in the Assessment Portfolio. Work through them in the order in which they occur in the Modules, but use the Assessment Portfolio as a workbook for writing in. All the work you do should be kept in your Assessment Portfolio as part of the evidence of competence you will need to provide for assessment purposes.

Your portfolio should contain all the completed SAQs, exercises, reports and feedback from people you have used in your test administration. Keep all this information together. Your assessor will want to see it as evidence of your competence.

Enjoy your learning and developing your practical skills.

(16)

9

An Introduction to

Psychological Testing for Test Administrators

OVERVIEW

Module 1 introduces two general categories of assessment instrument:

ability and attainment tests on the one hand, and measures of interests and personality on the other.

KEY AIMS

Having completed this Module you should be able to:

Describe the characteristics of a psychological test

Distinguish between tests of attainment, ability and aptitude, personality questionnaires and interest inventories

Give examples of each type of test used in occupational assessment

Introduction

As a test administrator, you should always be working under the direction of a qualified test user. A qualified test user is someone who has demonstrated competence in a range of skills associated with the use of tests and is registered with one or more suppliers of tests as someone who can purchase tests. Test users may have various different levels of qualification, but the minimum level recognized by the British Psychological Society (BPS) for test use in occupational settings is Level A. More details of Level A and other BPS qualifications can be found on the BPS’s Psychological Testing Centre website: www.psychtesting.org.uk

All the decisions that require a detailed knowledge of tests and testing (such as which test to use and how to make use of the results) will be the responsibility of the qualified test user. Your role as a test administrator is to assist and support the work of the test user. In order to understand the role of the test user and how you can support that, it is helpful if you know a bit about psychological tests and testing.

1 Module

(17)

About these modules

Most of the information in these modules is written in a ‘normal’ format in the same way as this paragraph is written. This format is used for both

•

information that will help you to put your role into context and understand the key role of good, professional, test administration in the testing process and

•

the areas of knowledge and skills for which you will be expected to demonstrate competence.

Other information is shown in grey boxes. These boxes contain information that is not needed for the BPS qualification. If you do read it, either now or later, you should be better able to understand some of the terms that the test user may use when talking about testing. However, the materials presented in these units do not provide the level or amount of knowledge and understanding needed for a Level A qualification.

Examples of questions from tests are placed in tables in the modules, and these should give you a clearer idea of what different types of test item look like.

(18)

11

What Are Psychological Tests?

Testing is probably the area of psychology which has had the greatest impact on members of the general public. Most people will, at some time in their life, have completed some form of psychological test. Increasingly, they are likely to encounter such tests as part of the process of getting a job, in career guidance and in the assessment of their career development and training needs. The proper use of well-developed tests in these situations can provide considerable benefits – both for the organizations using them and for the individuals being assessed. However, poor use of good tests or the use of badly designed tests can create a whole range of problems. These range from bias and unfair discrimination in selection to the giving of bad advice to people seeking help in their search for employment.

Poor administration of tests and the accompanying tasks of scoring and converting to standard scores can introduce another layer of bias into the process and thus reduce the value of good tests and compound the problems of badly designed tests. This means that a test administrator needs skills and knowledge to prevent this from happening. These Modules and the accompanying test pack are designed to provide you with the knowledge and skills you need to administer and score tests in a fair and competent manner.

What is a ‘psychological test’?

To help put your role into context and understand the critical importance of professional test administration it is important to understand what a

‘psychological test’ is.

In ordinary conversation, a test is something that you take; something that you pass or fail, the results of which are used to make judgements of worth about you. This usage is unfortunate for those who work in psychological testing, as in psychological testing the word ‘test’ refers to a much broader range of assessment procedures. It is probably a good idea to try to avoid using the word

‘test’ if possible, because of the connotations it has; assessment is a preferable term. However, the terminology psychological test has become very well estab- lished and so conflict between the two uses of the words is almost unavoidable.

This conflict is particularly great when we look at personality or interest assess- ment. For instance, it is clearly misleading to talk of personality testing since

1.1

(19)

the common meaning of testing leads us to assume that you can pass or fail a personality test and that therefore some personalities are better or worse than others. Of course, this is not so. A personality ‘test’ is called a test only because it has been constructed according to the principles of psychological test theory.

In these Modules we will use the term test in its technical sense to refer to an instrument that has been developed using psychometric principles. This is how the word is used in most of the testing literature and in any technical documentation you will come across. As a test administrator, you need to get clear in your own mind this distinction between the common and the psychological use of the word ‘test’. But you must also remain aware that most test takers will have only the common meaning in their minds when you talk of giving them tests or of testing their interests and personality. For that reason it is important to be very careful how you describe assessment procedures to those who don’t have a technical understanding of psychometrics.

OPEN QUESTION: Pause and reflect for a moment on the use of the word ‘test’.

1. What does the word ‘test’ mean to you personally? What images or memories does it conjure up?

2. Can you think of any other terms to use instead, which you could use to describe a psychological test to someone who was about to have a test administered to them

Write down your answers.

...

These questions are aimed at making you think about the notion of testing people, and how people might react to the idea of being ‘tested’. You will have a chance to look back at these ideas when you come to Module 2, which deals with practical test administration.

A psychological test consists of a collection of questions or tasks. These are known as test items. In a simple test, the test taker’s answer to each item is scored and the item scores added up to provide a single measure called a raw score or raw scale score. In more complex tests, there may be several scales. For example, the Sixteen Personality Factor Questionnaire (16PF) has 16 scales; the Occupational Personality Questionnaire (OPQ32) has 32 scales. In each of the above examples, each scale is used to measure some specific aspect of personality. Each scale has its own scale score or raw score. In most tests each item or question counts towards the score on only one scale.

So, what makes a test so different from a list of questions that anyone might devise? You cannot judge whether a list of questions is a psychological test simply by looking at it. While most tests of ability contain a list of

?

(20)

WHAT ARE PSYCHOLOGICAL TESTS? 13 questions of some sort or other, potentially the tasks presented to the test taker range widely and can be anything from which a measurement can be taken. For example, some tests assess the speed and accuracy with which the test taker can move pegs from one hole to another; others look at how well people can rotate images of shapes in their mind; yet others may be based on solving anagrams. However, all will have standardized administration instructions whether these are to be read out by the administrator or presented on a computer screen, and a test administrator needs to be aware of the wide range of instruments and tests that are available.

Test manuals

The things which distinguish whether a list of questions is or is not a psychological test are the technical information and the procedures laid down for its use. This information will be contained in the test’s manual. The technical documentation accompanying a psychological test, (usually called a test manual or test user manual), will tell you how to administer and score the test and will tell the test user what conclusions can reasonably be drawn from the results. Sometimes, test manuals are split into user manuals, which focus on test administration, scoring and interpretation, and a technical manual, which provides all the technical background on the test’s development, its reliability and validity. ‘Norms’, which are used for converting scores obtained on the test into measures on a standard scale, may be contained within the manual or are often provided as separate supplements. As a test administrator you may need to use norms when you come to scoring tests (see Module 2), however the choice of which norms to use is the responsibility of a qualified test user. The issue of norms is discussed further below and in more practical detail in Module 2. Nowadays it is quite common for user manuals, technical manuals and norms to be available from the test publisher’s website. This is so that the documents can be more easily kept up to date.

Tests differ from more informal assessment procedures in having standardized administration procedures (i.e. each time the test is administered, exactly the same standard procedure is followed). This is done to ensure that all test takers are provided with the same opportunity for doing well. It is crucially important for the test administrator to understand that standardized instructions provided in the test materials must be adhered to and given in the same way for every test taker and to ensure that the instructions are not skipped through or ignored altogether when they appear on a screen in a supervised computer administration.

The test’s documentation provides normative information: this enables us to see how people’s performance on the test compares with that of others. Such information might include data on the average scores for different groups (or ‘populations’) such as males under 30; blue-collar workers; female undergraduates, and so on.

As well as providing the average scores for such groups, a test manual will also describe how scores are distributed: that is, in general how likely people are to obtain particular scores on the test. This information enables test users to decide what sort of score is an above average one, what average is, and what is below average.

(21)

As a test administrator, you should discuss with the test user which set of norms will be used in any testing session. This will ensure that when the test takers’ scores are converted to standard scores (see Module 2 for a discus- sion of converting scores) they are being compared with the relevant group.

The technical section of a test manual will also include information about how accurate or reliable the scores are. It is vital for test users to know what the margin of error is on a score, if they are to make appropriate use of it.

Test users also need to know about the validity of a test. This tells the test user how well the test is measuring what it says it is measuring and what sort of conclusions can draw from the scores.

The concepts of standardization, reliability and validity lie at the heart of psychometrics and it is essential for test users to know about these.

What do tests measure?

The processes that psychological tests are designed to measure are not concrete things like height or weight. You cannot measure anxiety, spatial abil- ity or motor co-ordination directly in the same sense as you measure your shoe size. Anxiety is not something you can get hold of and measure. Rather, it is a word which provides a useful way of talking about a range of related types of feelings and behaviours. As such, anxiety is a useful idea that enables us to make sense of the way in which a number of different behaviours, signs and symptoms tend to be associated with each other.

TABLE 1.1.1: Some of the major differences between a psychological test and a set of questions

A psychological test

1. The scientific rationale for the test is presented.

2. Its method of construction is described – how the items were created and the criteria adopted for their selection.

3. For each scale, questions are selected which measure just one characteristic of a person.

4. Test administration procedures are documented and standardized.

5. Information is provided on the scores obtained by a large sample of people from a well-defined population.

6. Measures are provided of the accuracy of the test scores; on the degree of error present in any score you obtain.

7. Evidence is provided about the validity of the test: what it does measure and what it does not.

8. Guidance in interpretation is provided which is linked to evidence of validity.

A set of questions

1. No scientific rationale is presented.

2. The method of test construction is not described – someone just thought up the questions.

3. There is no evidence to show whether the final score is a measure of just one aspect of ability or a composite of various different ones.

4. Test administration procedures are undocumented and unstandardized.

(22)

WHAT ARE PSYCHOLOGICAL TESTS? 15 5. No information is provided on the scores obtained by other people.

6. No indication is provided of the accuracy or stability of the test scores.

7. No evidence is provided about the validity of the test. Validity is assumed on face value (‘face validity’) or what the test looks to be measuring.

8. Guidance in interpretation may be provided, but it is not linked to evidence of validity. Often, arbitrary ‘cut-off’ scores are provided with text explanations (e.g. ‘If you score between 10 and 20 you will find some tasks difficult to deal with’).

Collect examples of ‘tests’ from Sunday newspapers and other magazines.

Look at them carefully to see how close they come to meeting the criteria for a psychological test. Are they just lists of questions?

Summary

•

Information about reliability, norms and validity, as set out in the test manual, provides information about what the margin of error is on scores, how groups of people have performed on this test in the past and what their scores mean. This allows us to see how the score of the person being tested compares with those of others and what inferences we can draw from the test scores about other aspects of the person’s behaviour and performance.

•

Tests comprise a series of standardized tasks. They are designed so that everyone is given the same, or carefully matched, tasks to do and everyone is given a standard set of instructions for doing them.

In administration of a test, these standardized instructions must be followed carefully.

•

As a result, competent test users using well-designed tests can make reasonably accurate judgements about people’s capacities or potential to act or behave in certain ways. For instance, this might include the likelihood that an individual would be able to cope with the demands of a particular training course and their potential for success in certain types of work.

The technology underlying the development and use of psychological tests may seem complex and difficult to understand at first – but it is the body of information and statistics about people’s responses to its content, that gives a test its particular value and differentiates it from a set of questions.

(23)

16

Main Types of

Psychological Test

In general, tests fall into two broad categories:

1. There are those designed to assess characteristics or attributes, such as personality, beliefs, values, and interests, and to measure motivation or ‘drive’. These are known as measures of typical performance.

2. There are those designed to measure ability, aptitude or attainment.

These are known as measures of maximum performance.

This Module is concerned with psychological testing in general and with making you aware of the diversity of psychological tests. It deals with maximum performance tests (ability, aptitude or attainment) and certain types of measures of typical performance (personality, beliefs, values, interests and motivation).

These are the sort of tests suitable for group or individual administration and for assessing differences between people relating to occupational issues. In- depth, individual mental ability testing – carried out for diagnostic reasons – and some forms of personality or motivational assessment can require far more complex processes of administration and require additional specialist training if they are to be used effectively. These are not dealt with in detail in these Modules. The material presented here will provide a basic foundation in test administration in psychological testing on which you could later build further skills and knowledge in test use, and it will cover the requirements for the BPS Occupational Test Administration qualification.

Measures of typical performance

Measures of typical performance generally fall into three main categories: mea- sures of Personality, measures of Vocational or Occupational Interests and measures of Drive, Motivation or Need (see Table 1.2.1 for some examples).

Personality concerns the way we characteristically respond to other people and situations: how we relate to other people, how we tackle problems, our emotionality and responsiveness to stress, and so on. While interests are also related to personality, measures of interests focus more on what sort of activities we find attractive and which we would rather avoid. Measures of motivation and need focus on the factors which drive us to action (such as the need for success) or cause us to refrain from action (such as the fear of failure). Many personality and interest measures – either directly or indirectly – also provide measures of need.

1.2

(24)

MAIN TYPES OF PSYCHOLOGICAL TEST 17

(a) Personality inventories. Personality inventories are good examples of tests that assess our preferred or typical ways of acting or thinking. Items that test these characteristics do not have right or wrong answers. Rather, they attempt to measure how much or how little we possess of specific characteristics or sets of characteristics (e.g. gregariousness, empathy, decisiveness). Most instruments designed to measure such characteristics are administered without a time limit and stress the need for people to answer honestly and openly. But, in some situations, such openness may be difficult to achieve (for example, if it is perceived that one’s chances of being selected for a job depend on the results). However, in other situations such problems are less likely to arise where one can be sure that it is in the test taker’s best interests to co-operate and be honest (e.g. in vocational guidance).

( b) Interest inventories. Interest inventories are designed to assess in a systematic manner people’s likes and dislikes for different types of work or leisure activity. Satisfaction at work requires not only possessing the necessary skills to do the job competently but also having sufficient interest in it. Like tests of personality, these are not tests in the sense of having right or wrong answers, and hence they are very different from measures of maximum performance

(discussed later).

Interest inventories have an obvious application in guidance and in staff development assessment situations, where people may need help in sorting out what they do or do not want to do. They provide a means of exploring new options with people, of suggesting areas of work that they would not otherwise have considered. As with personality assessment, assessing interests may provide a useful, positive way of opening new doors for people in a career guidance context. Both forms of assessment are less well suited to

judgemental situations where any task a test taker is asked to complete will be perceived as a test of whether he or she has the

‘right’ qualities.

Both personality and interest assessment inventories are essentially different in kind from ability tests, even though the same psychometric principles apply (the need for reliability, validity and standardisation).

Such inventories are the means of providing a more qualitative description of people. Most of the available personality and interest tests are self-report or self-description instruments. That is, they are like a highly-structured, written interview that has been standardised and subjected to psychometric analysis. Hence, if properly used they can provide valuable sources of data about personality and interests to supplement information obtained from other sources (references, interviews, and the like).

(c) Measures of drive, motivation and need. People’s levels of drive or motivation can be thought of as having both state (moods) and trait ( permanent characteristics) components. Some people are characteristically more driven than others: some people always seem to be on the go, seeking more and more work or responsibility, while others are the opposite. At the same time, any individual will vary in their level of drive from time to time. On some days they will feel they

(25)

have more get up and go than on others. Many personality inventories measure aspects of trait motivation. These are often called needs:

the need for achievement

the need to be with other people the need to have approval from others the need to avoid failure

and so on.

Needs motivate us in that they tend to establish our priorities and our goals. Interest measures also provide some indication of motivation.

Generally, people strive hardest at those things that interest them most.

TABLE 1.2.1: Types of item used in personality and interest inventories The following are typical of the form and content of items you might find in personality inventories and interest inventories.

(Note: they are not taken from any actual inventories. In practice, the range of item formats and, of course, content is far wider than shown here.)

For each statement, choose (a), ( b) or (c)

1. Even when people are trying to be constructive, I find I get upset by their criticisms.

(a) Yes ( b) In between (c) No

2. If I could live my life again, I would do things very differently.

(a) Yes ( b) In between (c) No

For each pair of statements, tick the one most true of you: (a) or ( b) 1. (a) I tend to speak my mind whatever the consequences.

( b) I like to think through the effect of what I might say before I speak.

2. (a) I would rather work for a company with a long history of steady growth.

( b) I would rather work for an innovative new company that was prepared to take a few risks.

Rate each activity using the scale:

(1) Would dislike it a great deal (2) Would dislike it (3) In between (4) Would like it (5) Would like it a great deal

1 2 3 4 5

Working closely with people as part of a team [ ] [ ] [ ] [ ] [ ] Using a computer to analyse data [ ] [ ] [ ] [ ] [ ]

Arranging delivery schedules [ ] [ ] [ ] [ ] [ ]

Measures of maximum performance

Measures of maximum performance (ability, aptitude or attainment) measure how well people can do things, how much they know, how great their potential

(26)

MAIN TYPES OF PSYCHOLOGICAL TEST 19 is, and so on. Many of these measure general, rather abstract, characteristics (e.g. verbal fluency, spatial orientation, numerical reasoning) while others may seem more concrete and functional (clerical speed and accuracy, programming aptitude). The distinguishing feature about such ability tests is that they tend to contain questions, problems or tasks for which there are right and wrong (or good and bad) answers or solutions.

In addition, while tests of typical performance (personality, beliefs, values, interests, and motivation) are usually administered without any time limit on their completion, tests of maximum performance are usually timed. In some cases the time limitation is very strict and the emphasis is placed on how quickly a person can respond to the items. Tests which contain relatively easy items, but with a strict time limit are called speed tests. In other cases, the time limit is designed to allow most people to complete all the test items, and the focus is on how many they are able to get right. If the score you get is mainly affected by your ability to answer the questions – rather than your speed – the test is a power test. If the test is not automatically timed (as in a computer-delivered test) it is the responsibility of the test administrator to ensure that the timing is strictly adhered to (see Module 2 for practical ways of ensuring the timing is accurate). As you can see, for speed tests, making sure everyone has exactly the same time to do the test is critical if they are to be fairly evaluated.

Some examples of the enormous variety of item types used in maximum performance tests are shown in Table 1.2.2.

TABLE 1.2.2: Examples of the sort of items used in tests of maximum performance

The following examples give some idea of the sort of items used in tests of maximum performance. These are taken from tests of aptitude and ability. In all of these, the illustrations provided are those given in tests as examples or practice items for the test taker. For reasons of confidentiality, we have not included any that would count towards a person’s score on any test. As a test administrator, test users will provide you with access to such items as well as to information about how they are scored. Part of your responsibility as a test administrator is to ensure that the confidential nature of such information is maintained.

The tests illustrated in Figure 1.2.1, in order of presentation, are:

Two verbal tests:

1. The General Ability Tests (GAT) Verbal

2. The Advanced Managerial Tests (AMT) Verbal Analysis Two spatial (three-dimensional manipulation) tests:

3. The General Ability Tests (GAT) Spatial

4. The Information Technology Test series (ITT) Spatial Reasoning Two ‘abstract’ reasoning tests:

5. The General Ability Tests (GAT) Non-Verbal

6. The Graduate and Managerial Assessment (GMA) Abstract Two numerical tests:

7. The Advanced Managerial Tests (AMT) Numerical Analysis

8. The Graduate and Managerial Assessment (GMA) Numerical Reasoning

(27)

2. The Advanced Managerial Tests practice leaflet (AMT) Verbal Analysis item.

1. The General Ability Tests (GAT) Verbal scale example items.

Published by NFER-NELSON.

FIGURE 1.2.1: Example items from verbal, spatial, abstract and numerical tests

(28)

3. The General Ability Test (GAT) Spatial example item. Published by NFER-NELSON. Reproduced by permission.

FIGURE 1.2.1: continued

(29)

4. The Information Technology Test series practice leaflet (ITT) Spatial Reasoning item.

(30)

5. The Graduate and Managerial Assessment (GMA) Abstract example item. Published by NFER-NELSON.

Reproduced by permission.

5. The General Ability Tests (GAT) Non-Verbal example item. Published by NFER-NELSON.

(31)

36+

(8%) 0 – 5 (16%)

6 – 15 (28%) 26 – 35

(24%) Administration

(44%)

Production (56%)

16 – 25 (24%)

7. The Advanced Managerial Tests practice leaflet (AMT) Numerical Analysis item.

(32)

Pause and reflect for a moment on the differences between measures of maximum performance and measures of typical performance. (The following questions will help to structure your thoughts.)

1. What sorts of attributes are assessed by maximum performance measures and what sorts by typical performance measures?

2. How do they differ in the way in which they are timed?

Answers to SAQs and exercises can be found at the end of the Module.

The tests we have talked about so far all fall into the category of ‘paper-and- pencil’ tests – tests where the questions are printed and answers are in written form. Nowadays, you will find increasingly that such tests are also available in computerized (or partially computerized) forms.

Not all maximum performance tests are about measuring speed of mental oper- ations. There are also a number of different tests that are concerned with psycho- motor ability. For example, hand–eye co-ordination tests are commonly used to select people for training as pilots. Psycho-motor co-ordination tests and other newer types of maximum performance test cannot be carried out using paper-and-pencil technology. They require the use of various items of specialized apparatus, often in combination with a computer. Table 1.2.3 shows examples of these apparatus tests.

TABLE 1.2.3: Examples of apparatus tests

Some apparatus tests are designed to measure dexterity and hand–eye co- ordination. Figure 1.2.2 shows the Crawford Small Parts Dexterity Test (CSPDT).

Computer-based tests can provide a wide range of new types of assessment not possible with paper-and-pencil technology. The two screen displays are from MICROPAT – a series of tests used for selecting pilots for training. The first (called LANDING) is like a simple computerized flight simulator. The second (called SCHEDULE) requires the test taker to keep track of a changing complex display of information and to make quick and effective decisions.

Another large category of apparatus tests is that of work samples. These are described later in this Module.

8. The Graduate and Managerial Assessment (GMA) Numerical Reasoning example item. Published by NFER-NELSON. Reproduced by permission.

1.2.1 SAQ

(33)

FIGURE 1.2.2: The Crawford Small Parts Dexterity Test

(34)

FIGURE 1.2.3: Examples of computer-based tests

The two tests illustrated are called LANDING and SCHEDULE and form part of the MICROPAT battery of tests developed for use in the selection of people for training as pilots. Crown copyright.

(35)

Differences between ability and attainment tests

Tests of maximum performance can be divided into those which assess what we have learned and those which assess our potential for learning new things in the future. We call those which assess what we know or what we can do, tests of achievement and attainment, while those which assess our potential are called ability and aptitude tests.

Ability and aptitude tests are designed to provide an indication of a person’s potential to succeed in a wide range of different activities (e.g. coping with the academic demands of a degree course or being able to acquire the competences needed for a new job). Although such measures will depend somewhat on a person’s previous experience and learning they are used to draw inferences about the person’s potential.

Attainment and achievement tests, on the other hand, specifically assess what people have learned and the skills they have acquired (e.g. shorthand and typing tests; knowledge of motor mechanics).

What they have learned will, of course, depend partly on their ability – so scores on the two types of test are often related. However, the focus of attainment tests is on what has been learned and not on how or why this learning was acquired. School, college and university examinations are all methods of assessing achievement, and while people (such as potential employers) may draw inferences from them about a person’s ability or suitability for a job, the measures themselves are not designed or intended as measures of ability.

The main difference between ability and attainment tests lies in the way scores are used rather than in the actual test items.

Many aptitude tests contain items that look very similar to those one could find in attainment measures (e.g. vocabulary tests and mental arithmetic). However, attainment tests are retrospective:

they look back at what has been learned, what is known, what skills people have acquired. Ability tests, on the other hand, are prospective: they look forward to what people are capable of achieving in the future.

However some abilities cannot be measured until the test taker has a certain level of attainment. For example, the ability to reason using words (verbal reasoning) cannot be measured until the person is able to read. While ability is needed in order to attain new knowledge or skills, a test which shows that someone has reached a certain level of attainment does not tell us much directly about their ability. For example, writing an essay on Roman Britain requires the attainment of relevant knowledge and essay-writing skills. The ease with which this knowledge and these skills are attained will depend on a person’s ability. The quality of the essay, however, while providing a good indicator of the writer’s attainment, would not be of much use as an ability or aptitude measure. In the same way, tests of ability may provide very little direct information about a person’s level of attainment. For example, detecting regular patterns embedded in a background of confusing lines would be unlikely to serve a useful purpose as an attainment test, but it can provide very useful information about spatial ability.

(36)

MAIN TYPES OF PSYCHOLOGICAL TEST 29 People can only do verbal reasoning tests or numerical reasoning tests if they have learned the relevant language or number system. This might suggest that they are attainment tests, but this is not so, as they are not designed to measure how well we have learned our language but how good we are at reasoning with it. While they are dependent on the effects of experience, their function is to provide an estimate of potential, or in other words, ability. Test users have to be careful with such tests that they are fair to all those who are being tested by ensuring that the people they test have a sufficient knowledge of a particular language. Just as testers would want to ensure they could understand the test instructions, so they need to be confident that differences between the test takers on the test are due to differences in their abilities to reason with language. We don’t want to confuse differences in reasoning ability with differences due to problems of basic literacy or lack of fluency in the language being used for the test.

To summarize:

•

Attainment tests measure what has been achieved; ability tests measure what can be achieved.

•

Tests of attainment and tests of ability sometimes use the same items or content, but the scores are used differently.

•

Certain items are more relevant to attainment than to ability tests, and vice versa.

•

Some abilities cannot be measured until there is a certain level of attainment.

Work samples, trainability tests and job simulations

These require separate mention as they are of special interest in occupational testing and test administration. They cover the range from aptitude to attainment.

A work sample test is one in which the task has been taken from a job. All work samples assume that you are selecting experienced people. The task is done under standardized conditions. A typing test used for the selection of secret- arial staff is an example. It assumes that the applicant has some measure of typing skill, and sets out to see how much. So, it is clearly an attainment test.

Another example is the far more complex flight deck simulator check rides used for selecting pilots. These are only usable for the selection of qualified pilots – to select people for initial training as pilots you would have to use aptitude tests, not work sample tests.

Trainability tests, on the other hand, are designed to see whether someone is likely to be able to cope with the training required to do a job. Typically these consist of a highly structured short training course with a test of performance at the end. The test that comes after the training is very much like a work- sample test. Trainability tests have been developed for a range of occupations, from fork-lift truck driving and sewing machine operation to air crew training.

Job simulation exercises are typically met with in the multi-method, multi- dimensional, multi-assessor procedures that come under the general head- ing of the Assessment Centre Method. These kinds of procedure often form the basis of assessment for management selection and development and are widely for selection into the military or government service. The job simulations may take the form of in-tray exercises, analysis of complex