ACER Newsletter No. 31 December 1977

(1)

ACER

^Contents

NEWSLETTER

• New Directions and Perspectives

• The Parish Primary School Survey

No31

• Science Facilities Evaluation Project

DECEMBER 1977

• Diagnostic Tool Utilizes New

/

)

New Directions and Perspectives

During recent months the staff and Council of the ACER have been dis- cussing new directions and new perspectives for the program of research and development activities to be undertaken during the coming years. Traditionally the ACER has been active in the area of measurement and evaluation and has achieved a high degree of eminence both nationally and internationally in this field, particularly in the development of tests to serve educational and psychological needs. The ACER has also been increasingly engaged in under- taking studies that have a direct bearing on the work of the schools and on classroom practices. This is an area where the findings of research are of direct relevance to schools and classroom teachers as well as to college lecturers, inservice educators and curriculum co- ordinators and consultants.

There would appear to be a third area, which the ACER has touched on in the past, but without a clearly defined program of work being developed. This area may be best described as being concerned with studies in the social foundations of education. Changing governmental initiatives in education in recent years have highlighted the need for an ongoing program of studies in this area. Moreover, since some research studies will, of necessity, be. concerned with issues of educational and social policy it has been argued that such work is best undertaken by an independent research institution.

The ACER has recently advertised for three Assistant or Associate Direc- tors to join its staff to supervise the programs of research in each of these three areas referred to above: Measur- ment and Evaluation, Learning and

Teaching, and the Social Foundations of Education. It seems likely that appointments will be made in the first and last of these areas with the new members of staff taking up their appointments in the early part of 1978.

In all three areas a strong program of research will be developed which will continue work that is currently being undertaken and will include new studies relevant to the needs of individuals, the nees of educational institutions and the needs of Australian society.

In addition to these main areas of research it is envisaged that there will be several Units whose activities cut across the programs being planned for in the three areas. These Units would remain relatively small. They would be encouraged to provide services not only to research workers within the ACER but also to researchers, teachers and administrators outside it. In addition, the staff of these Units would be expected to carry out their own research studies.

Currently there would appear to be a need for four Units to provide specific services, namely, Survey Services, Statistical Analysis and Psychological Measurement Services, Library and In- formation Services and Advisory Services. Members of the ACER staff are already working in these areas, except perhaps in that of statisticai analysis and psychological measurement where the only studies currently being undertaken are associated with the validation of the Australian Scholastic Aptitude Test and the determination of cutting scores in criterion referenced testing.

In addition to identifying fields of research activity it is also of value to describe the studies being carried out by the ACER in terms of their functions.

Measurement Techniques

Eleven functions are envisaged for the research studies being undertaken by the ACER during the coming years:

1 Studies of educational and social policy,

2 Evaluation studies of particular policies and programs,

3 National studies of educational outcomes,

4 Studies of the education of special groups,

5 Studies of school and home practices, 6 The developement of tests for use by

teachers and guidance officers, 7 The development of tests and instru-

ments for use by psychologists, 8 Library and information services, 9 Testing services and programs, 10 Consultancy services and,

11 Advisory services.

The Director and staff of the ACER welcome contributions to their thinking on new directions and perspectives. In the past ACER has been aided by thoughtful suggestions regarding its projects and program and we hope that readers will continue to let us know what they think are important aspects of education which ought to be studied.

Future issues of this newsletter will outline the ACER research program in each of these new areas as new plans are made and new developments take

place.

m(1~lf tll[iill~UII l

(2)

The Parish Primary School Survey - 1976

Jack Darmody

The Parish Primary School Survey - 1976 was a joint project of the Catholic Education Office of Victoria and the Australian Council for Educational Research. The purpose of the survey was to describe the performance of children attending Parish Primary schools in Victoria with a view to identifying areas of relative need and making specific recommendations regarding additional services which might assist these schools in the prevention and remediation of learning difficulties.

Simple random samples of approximately 1000 children in each of three age groups (7-, 9- and II-year-old age levels) were selected in order to obtain reliable estimates of performance of the population of children in the three age groups attending the schools.

Children in the selected samples were given tests in reading, number skills, and general ability. At the same time background information was obtained by means of questionnaires completed by the children, their teachers and parents and by the school principals.

The background information varied from factual data to teacher and parent perceptions of the child's progress, and the need for, and availability of, re- medial assistance in reading, and mathematics. Factual data included the child's sex, age, grade, age of com- mencing school, pre-school experience, father's occupation and the extent to which mothers worked outside the home. One questionnaire (developed by Rutter, 1967) completed by teachers, supplied information on children's behaviour patterns in areas relating to children's general development and adjustment. Teachers gave each pupil a rating on a three-point scale (0, I, 2) on each of 26 items, yielding a total score in the range 0-52.

The tests of reading and numeracy administered to the 9- and 11-year -old children were those developed for the Australian Study of School Perfor- mance at ACER in 1975. Seven-year- old children were given the ACER Primary Reading Survey Tests AA and BB. A test of number skills for this age group was constructed by selection of a range of items from Key Math Diag- nostic Arithmetic Test (Connolly et al.

1971) which according to test norms were appropriate for investigating number skills of seven-year-old children.

The test covered the areas of content, operations and applications. The general ability tests used were all tests published by ACER (TOLA 4 and TOLA 6 for 9- and 11-year-old children and two subtests of the ACER

2

Lower Grades General Ability Scale for the 7-year-old children).

The results discussed here are a summary of findings available in full detail in the final report of the survey - The Parish Primary School Survey:

Final Report - which will be published at a later date.

The data gathered in this project were wide ranging and it is difficult to provide a simple picture of the cluster of variables that were found to be associated with performance in the classroom. Two major conclusions were evident from test scores and relation- ships between test scores and background variables. In the first place there was a significant correlation between all measures of classroom performance. Children who were disadvantaged in terms of their scores on one measure tended to be disadvantaged on all measures. Children of all ages with scores indicating below average general ability tended to have low scores in reading and number work as well as low behaviour ratings from teachers, suggestive of lack of adjustment to emotional and social require- ments of the classroom. There was little evidence of what has come to be referred to as specific learning disabilities.

Approximately one third of the lower 10 per cent in reading amongst 7-year -old children had general ability scores of average or better. However, in the I I-year-old sample, some of the lower 10 per cent in reading had general ability scores of average or better.

This would seem to indicate that early reading problems could have been related to developmental delays and gradually disappeared with increased maturity.

The second major conclusion is an extension of the first. Children whose performance scores indicated disadvantage in school achievement tended to be also disadvantaged on most background and personal variables examined.

Significant associations between performance measures and a variety of variables were found. Variables included with familiarity with, and use of, English in the home, availability of additional assistance whether at home or at school, and teacher perceptions of general performance in a variety of areas. Such areas included deviation from acceptable activity levels, ability to hold a pen or pencil sufficiently well to cope with classroom activities, speech, and reversals in reading and spelling simple words. Lower levels of reading and numeracy were also found to be related to socio-economic circum- stances as judged by father's occupa- tional levels. Children whose fathers had skilled, semi-skilled and unskilled occupations had lower performance in

reading and numeracy than children with fathers in professional, managerial and clerical occupations. The relation- ship between the variables and performance were complex because of relation- ships amongst variables themselves.

For example, father's occupation, language in the home, extent of pre- school experience of the child and the resources available for additional assistance outside school were all closely inter-related. Children from homes where little or no English was spoken more of ten than not were the children of semi-skilled and unskilled parents who had neither the financial nor human resources available to assist children in their school work. Such children also more often lived in sub- urban areas where there was a less than average use of kindergartens.

Generally speaking the study showed that children with lower levels of performance in reading and number shared a cluster of attributes which indicated disadvantage in a wide range of areas.

While it could not be claimed that such children shared all attributes, the fre- quency with which they occurred signi- ficantly suggested that a high proportion of such factors was associated with lower performance.

Bourke, S.F. & Keeves, J.P. (Eds).

The Mastery of Literacy and Nu- meracy. Australian Studies in School Performance Vol.I/1 Canberra: Aus-

tralian Government Publishing Ser- vice, 1977.

Connolly, A.J., et al. Key Math:

Diagnostic Arithmetic Test. Circle Pines, Min. : American Guidance Service, Inc., 1971.

Rutter, M. 'A Children's Behaviour Questionnaire for Completion by Teachers: Preliminary Finding'.

Journal of Child Psychology and Psychiatry, 8, 1-11.

Do you subscribe to the A.J.E.?

Australian Journal of Education

$10.00 p.a. (Australia) $11.00 p.a.

(Overseas)

The ACER Newsletter is published by the Australian Council for Educational Research, 9 Frederick Street, Hawthorn, Victoria 3122. Communications should be addressed to the Editor, Peter Jeffery at this address. The Newsletter is published approximately four times a year and is serially numbered. Each issue is serially dated according to its month of publication. It is not a quarterly.

ACER Newsletter No. 31 December 1977

(3)

Science Facilities Evaluation Project

John Ainley

The Science Facilities Evaluation Project set out to evaluate the influence of the Australian Science Facilities Program on science education in secondary schools. Under the Australian Science Facilities Program the Australian government made available some $123 million for the improvement of science facilities in schools. The money was spent on laboratories and apparatus and was applied in government and non-government schools. This program was the first major capital spending program on secondary education by the Commonwealth Government of Australia. An evaluation of the educational effects and impact of the program was therefore important to account for this spending and to provide information to guide future developments.

The Science Facilities Evaluation Project structured its investigations around five broad questions:

• What contribution did the Australian Science Facilities Program make to the availability and quality of science facilities in Australian schools?

• Did the Australian Science Facilities Program meet the needs of different types of secondary schools for adequate science facilities?

• Did the provision of more abundant and better quality facilities influence the way science was taught in secondary schools?

• To what extent did the provision of improved facilities contribute to the students understanding of, and attitude to, science?

• Were there concomitant developments (such as administrative procedures and the provision of laboratory assistants) affecting science education which were influenced by the Science Facilities Program?

The research study involved four phases of investigation. The first phase involved a survey of the facilities available for teaching science in a sample of about 400 schools and of the contribution of the Science Facilities Program to those facilities. The second phase was a study of the influence of science facilities on the teaching and learning process. Using the survey data a sample of l 00 science classes (from 30 schools) was chosen so that a wide range of standards of science facilities was represented.

All the students in each class were asked to complete a questionnaire about aspects of the science lessons which they experienced. In addition interviews were conducted with all the science teachers in each school. The third phase was a

study of administrative aspects of the Program and of system-wide concomitant developments in science education. To this end official records in Common- wealth and State departments were examined and a number of administrators were interviewed. The fourth phase was a study of some outcomes of science education: achievement in, interest in, and attitudes to science.

At the time of writing, data from the fourth phase of the study is still being analysed and will be reported subse- quently.

The Provision of Facilities

As the overall response to the survey questionnaire was 95 per cent it was possible to use the information from the survey as a basis for commenting on the science facilities in Australian secondary schools. It was found that the standard of science accommodation had substantially improved during the course of the Science Facilities Program.

Considering specifically those schools which already existed when the Program began, some three-fifths of the present science rooms have been added since 1964. Moreover teachers considered these rooms to be of better quality than others in which science was taught, while nearly two-thirds of such science rooms built in government schools were so funded.

At the conclusion of the Program in 1975 some deficiencies remained. About one school in 10 had insufficient science rooms to accommodate all its science lessons while a further one in 10 could only have accommodated all its science lessons if those rooms were used much more heavily than recommended by the Commonwealth Standards Com- mittee. In addition some science rooms (especially in government schools) were equipped only for teachers to demonstrate experiments and were not properly equipped for students to do experiments.

While the Science Facilities Program had improved the standard of facilities to an adequate level it was clear that some schools were still deficient.

From a number of indicators it was apparent that government schools were less well provided with science accommodation than non-government schools ..

A larger proportion of government schools had insufficient science rooms for the lessons they taught and more of them used demonstration rooms than fully equipped laboratories.

These conditions were reflected in the views of the science co-ordinators.

Only three per cent of those in non- catholic non-government schools regarded laboratory facilities as an impediment to experimental work. In Catholic schools 11 per cent of co-ordinators were of that opinion but 18 per cent of tho~e in government schools expressed that view.

Regarding the provision of apparatus in schools very few schools reported that either expendable or minor apparatus was deficient. In fact lack of apparatus was rarely regarded as a problem for practical work.

The Influence of Facilities on Science Teaching in Schools

From the study that was made of Year 9 science teaching in schools with varying standards of science facilities it seemed that better facilities were associated with an enriched learning environment and more active forms of science learning.

An enriched learning environment was one in which there was a greater involvement in purposeful activity, better organization and more stimulation in the variety of methods and materials used. Good quality rooms were most strongly associated with student reports of better organization, and more abundant apparatus was associated most strongly with students perceiving a more stimulating environment. Being frequently in science rooms and having sufficient apparatus were both associated with a reported greater involvement of students in learning activities. Student reports of active forms of learning, defined as more experimental work, less learning from text-books, and greater encouragement to explore, were also associated with better science facilities.

Observations made during visits to schools were consistent with the results of the statistical analysis of students' views. The standard of science facilities did influence the way in which science was taught. Even though changes were not dramatic they were observable.

Science teachers certainly found it more convenient and efficient to teach actively when they had good facilities.

While good facilities were neither a determining nor major factor shaping the science teaching patterns in schools, they were a component which set limits within which choice could be exercised.

Schools made different uses of the facilities available to them. Rooms and apparatus were used more efficiently when supported by skilled laboratory assistants, modern curriculum material and energetic science teachers.

It is necessary to exercise caution when generalizing from a cross sectional study to changes which occurred over time. However, it did appear that the provision of good facilities through the Science Facilities Program generally facilitated the development of enriched learning experiences in science and nurtured more active forms of science learning. The extent to which such development~ occurred also depended on other factors: good laboratory staff, good teachers and suitable curricula.

3

(4)

I

Implications of the Findings

The Australian Science Facilities Pro- gram had the immediate objective of improving facilities, the intermediate objective of influencing the processes of science teaching and the ultimate objectives of effecting such outcomes of science teaching as student achievement in, and attitudes to, science. It was found that the Program did substantially improve the standard of science facilities in schools but that some deficiencies remained at its conclusion. In addition the study provided evidence that improved facilities did influence the processes of science teaching.

As the cost of specialist facilities in school buildings continues to increase it will be necessary to demonstrate that those facilities provided in the past have been well used. The Science Facilities Evaluation Project has been concerned with material facilities and science education. On the basis of this study I argue that while the provision of good facilities is unlikely to change school practices dramatically, it can foster better quality science learning. Develop- ments in science teaching have assumed that science was best learned in an enriched environment which was well organized, involving and stimulating.

It has been assumed that science education should involve a variety of activities through which students parti- cipated in the direct observation of scientific phenomena. Good facilities, apparently, assist these approaches to teaching science. While additional resources alone may not make for good science teaching, good buildings which are well equipped make both the teacher's and the pupils' work easier and more agreeable. It is as important to consider the influence of facilities on the process of teaching as their influence on those immediate outcomes of learning which may be more readily assessed by achievement testing.

4

Peter Thomson joins VISE

After over six years with ACER, the last two as Chief Research Officer in charge of the Educational measurement Section, Mr Peter Thomson is leaving to join the newly formed Victorian Institute of Secondary Edu- cation. VISE was established as a statutory authority by the Victorian Government late in 1976. A major objective of the institution is to evolve procedures for the evaluation and assessment of the ability and achievement of students to assist in their transition to employment or further studies.

Mr Thomson's appointment is as head of the lnstitute's Curriculum and Assessment Branch and a major part of his responsibilities will be the co- ordination of planning and policy related to curriculum and assessment matters.

Diagnostic Tool Utilizes New Measurement Techniques Greg Cornish

The Operations Test of the ACER Mathematics Profile Series, to be available February 1978, is the first of a new series of tests concerned with the diagnostic evaluation of mathematical development. This test is an outcome of ACER's involvement in two areas;

1 the development of diagnostic tests in junior secondary mathematics, and 2 the development of tests based on the probabilistic measurement model of Georg Rasch 1 •

The decision to develop a series of mathematics tests in the junior secondary level was in response to a need identified in recent years by the Aus- tralian Association of Mathematics Teachers. A technique based on the 'Rasch model' was used to provide a flexible testing system to best service this need.

The Operations Test in this series consists of 60 multiple-choice items and can be used to assess a student's ability to handle familiar mathematical operations. The ⁶⁰ items in this test are based on those used by Collis²in his recent research on cognitive operational levels in mathematics. Collis found that the complexity of an item for students depended on an interaction between the structure of the operation and the nature of the elements operated upon. To facilitate interpretation the items have been classified on this basis as shown in Table 1.

The items shown in Table 1 indicate, in a general sense, the interaction between elements and structure.

This classification is also relevant

· when selecting a group of items for a particular group of students. As the technique used to construct the tests in the series enables different groups of items to be selected from the test, teachers can 'make up tests' that are appropriate and suitable for their particular group of students. This approach is illustrated in the Operations Test Teacher's Handbook which recommends that 30 or 40 item 'tests' be selected from the 60 items. The selection of item groups depends on the range of mathematical development of the

students, their mathematical background (for example, 'should pronumerals be used?) and the time available for testing.

The Operations Test Teacher's Hand- book also details quantitative methods for selecting suitable items and inter- preting the information gained by testing.

A major feature of this series is the flexibility it provides for testing. Using the Operations Test items teachers may tailor 'tests' to better suit their particular students, and so gain information that is more relevent for diagnosing difficulties.

As noted at the beginning of this article, the ACER Mathematics Profile Series - Operations Test will be published in February 1978. Pre- publication enquiries should be directed to Education Advisory Services.

' Rasch, G. An individualistic Approach to Item Analysis. In Readings in Mathematical Social Sciences. Eds. Larzsfeld and Henry. Chicago:

Science Research Associates Inc. 1966, 89-107.

'Collis, K.F. A Study of Concrete and Formal Operations in School Mathematics: A Piagetian Viewpoint, Hawthorn, Victoria: ACER, 1975.

NOW AVAILABLE

Australian Studies in School Performance Volume III Final Report

The Mastery of Literacy and Numeracy S.F. Bourke and J.P.Keeves

ERDC Report No. 13 AGPS, Canberra, 1977.

A four page article summarizing this important contribution to knowledge of education in Australia was published in ACER Newsletter No.29, July 1977.

The full volume of 334 pages contains details of the performance of various groups of students studied and a dis- cussion of methodological and other issues raised by the study. The study utilized new techniques and has created interest in Australia and overseas.

Available from ACER or AGPS at

$8.25 per copy.

JUST OFF THE PRESS

Aspiration and Attainment: The Mea- surement of Professional Self Perception in Student Teachers by Gerald R.

Elsworth and Frank Coulter (Occasional Paper No.II) ... $4.00 Rights in Education: The Australian Conundrum by I.K.F. Birch (Australian Education Review No.9) ... $3.50 Table l. Classification of Operations Test items

ELEMENTS

Small numbers Large numbers Pronumerals Concrete Items 1-10 Items 21-30 Items 41-50 STRUCTURE

Formal

e.g.l 3+4=4+L',. e.g.21 123+456=456+L',. e.g.41 p+q=p+L',.

Items 11-20 Items 31-40 Items 51-60

e.g.11 (8-4)+4=L',. e.g.31 (987-321)+321=L',. e.g.51 (j-k)+k=L',.