Support for this project was provided by the Australian Council for Learning and Teaching, an initiative of the Australian Government's Department of Education, Employment and Workplace Relations. Support for this report was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government's Department of Education, Employment and Workplace Relations. 22 4 Table 4: Number and percentage of comments made about using RSD in ways that.
1 INTRODUCTION
- Project team members
- Project aims
- People involved
- Dissemination methods
- Stakeholders
- Related ALTC projects
Project team members disseminated their research and the RSD framework through two keynote addresses, three conference papers, and 15 national and international conference presentations. A website presenting the RSD framework and the project was developed in 2006 and has been regularly updated throughout the life of the project. A book proposal focusing on the RSD framework and its use, containing chapters by members of the project team and other international academics, is currently at Routledge, UK.
2 PROCESS
Initial development of the Research Skill Development Framework
This team submitted a proposal for a follow-up project and invited a member of the RSD project team to be part of their reference group. Dr Sue Bandaranaike, James Cook University, adapted the RSD framework to suit WIL, and kept Ms Patrick informed of its development and evaluation.
Methodology
- Awareness raising about, and informing the use of, the Research Skill Development framework
- Implementation of the project’s RSD approach
- Evaluation of implementation of project’s RSD approach
- Incorporation of alternative approaches to using the RSD
Generate evaluation criteria for a summary evaluation typically marked up to RSD "Level 3" or "Level 4". During 2008 and 2009, both project team members and academics working independently of the project developed several alternative approaches to using the RSD framework. This increased the project team's understanding of the ways in which RSD could be used.
3 OUTCOMES AND DELIVERABLES
Project outcomes
- Diagnostic and summative assessment of student research skills informed by the RSD framework embedded in the assessment regime of coursework students
- Measurable improvement in student research skills produced in each course
- Students in involved courses report being better prepared for and more interested in research generally and in higher degrees by research
- Undergraduate students in each course report a more satisfying learning environment and greater skill development compared to courses not using the RSD
- RSD approach trialled in eight courses in addition to those named in the original application by the end of the project
- An understanding developed of how to effectively transfer the RSD
Data from student interviews also suggest that students' perceptions of research competency development are greater when students have completed RSD-based courses, compared to their perceived development in non-RSD-based courses (see Part 4: Results and Analysis: Student Interview Data ). In addition to the nine courses mentioned in the original project proposal, 22 courses tested the use of RSD-based approaches in 2008 and 2009, as listed above. The availability of the published conceptual framework and easily accessible and applicable examples of RSD-based rubrics from a range of disciplines has enabled academics and others at Australian Technology Network universities (Queensland University of Technology, Royal Melbourne Institute of Technology), an Intensive Research University (James Cook University), and the University of Southern Queensland and Victoria University to adopt and apply RSD.
Project deliverables
- Workshops run twice per year in schools housing courses involved in the project across the four original universities, and in other universities
- Interactive web portal presenting examples of practice and assessment of RSD in a minimum of eight disciplines
- Peer-reviewed articles by some project team members on discipline-based RSD approaches, and conference presentations, accepted for publication one year
Twenty dissemination seminars, followed up with workshops or one-to-one support, were offered at the University of Adelaide in ten schools, two centres, one faculty and at the Barr-Smith Library. Eleven combination seminars and workshops were offered at other partner universities: three at Monash University in the Faculty of Business and Economics; one in a research teaching group; one in a library and academic language support group; two at the University of Melbourne; two at Macquarie University; and two at the University of South Australia. The University of Southern Queensland held three; Queensland University of Technology, two; James Cook University, two; and Trinity College Dublin, two.
4 RESULTS AND ANALYSIS
- Pre-course and post-course questionnaires
- Scores
- Analysis
- Academics’ assessment of student research skills
- RSD score generation and preliminary analysis
- Comprehensive analysis of scores
- Student interviews
- Interview protocol
- Analysis of interview transcripts
- Categories A and B: Advantages and disadvantages of explicitly developing research skills
- Factors that support or hinder develop student research skills
- Academics use of the RSD and interviews with these academics
- Ongoing RSD use by academics
- External Reviewer’s Interviews with Academics
- Summary of analysis
These results demonstrate substantial results from academics' use of the RSD to inform the development of students' ability to identify, evaluate, and use valid sources. This is consistent with students' self-assessment of research skills as discussed in section 4.1 above. A number of further assessments were reformulated in 2006, including the literature review assessment task, which used Level 3 of the RSD.
After the introduction of RSD, scores were assigned by summing the levels of RSD achieved on each page. In the initial analysis of the 2008 interviews, the project team used these questions to guide the categorization of comments. All but one of the positive comments therefore focused on the significant educational benefits gained through the explicit development of research skills.
However, only 30 percent of students reported an improvement in their ability to formulate a research question in the interview, but eight of the course's 10 questionnaires were found to be significant. Another difference in the data sets similar to the one above is shown by the category reporting the results of their research, with 30 percent of students noting this in interviews, but statistically significant perceived changes in seven out of 10 subjects. Of the four subjects that showed statistically significant changes in the information organization questionnaires, three were business subjects, and the fourth was the second semester of human biology, which followed the previous semester of explicit development of research skills.
When considering components of interviews other than specific skills, 75 percent of the students interviewed said that research skills they learned in RSD-based courses were useful in other university courses, and 89 percent said that those research skills are relevant to the workforce (yet only 24 percent of students said that research skills would make them more employable). Of all unique-to-student comments made, 147 of the positive effect comments (44 percent) stated that the ways in which the RSD was used were beneficial to their development of research skills, and 89 of the barriers comments (40 per cent) said that the ways in which the RSD was used was detrimental to their development of research skills. Sixty unique-to-student comments (18 percent of positive factors identified) identified aspects outside the course domain that had a positive impact on their research skill development, while 79 (35 percent of negative factors identified) discussed aspects outside the course. the course domain that had a negative effect.
5 UPTAKE OF RSD APPROACHES
- Sustainability
- Reproducibility
- Efficiency
- Variety of approaches
Curriculum redesign: In this approach, all available knowledge tests, laboratory assignments, field components, etc. Resource Module Structuring: This approach was developed at the Queensland University of Technology to organize existing interactive online modules to develop aspects of library research skills along a four-level continuum (see Appendix 9). PhD Bridging Program: This approach, developed at the University of Adelaide, uses the newly developed RSD7 (see www.adelaide.edu.au/clpd/rsd/rsd7 and Appendix 5) to structure the marking rubric for draft research proposals that was written by a Ph.D. applicants who are new to the university.
In this approach, students self-assess their work and supervisors assess the proposal against the rubric, which then forms the basis for a discussion about differences and expectations. Analysis of Existing Assessment or Curricula: This approach was used to assess a medical school's problem-based learning curriculum to determine the level of autonomy students needed to achieve during their studies. Starting point: this approach was developed in the Faculty of Humanities and Social Sciences at the University of Adelaide.
In it, the RSD framework serves as a basis for the development of structured methods for marking analytical works, but its structure was then adapted to meet the demands of individual disciplines. Development of student-negotiated marking criteria: in this approach, students in a school of education were given specific marking criteria for three facets of the RSD, but were expected to write and negotiate criteria for the other three facets, which were then used to assess their research evaluation. Policy guidance: Maastricht University in the Netherlands has embedded the RSD in their Dutch-language policy document.
6 DISSEMINATION
- Project website
- Seminars and workshops
- Papers and conference presentations
- Book manuscript
- Word of mouth
Members of the project team have published papers discussing their use of the RSD, and given presentations at local, national and international conferences (see Appendix 2 for a complete list). A full book proposal is at Routledge UK, with most of the original members of the RSD project team involved as authors of chapters. Although no systematic records of word-of-mouth dissemination were kept, anecdotal evidence from private communications and conversations suggests that word-of-mouth was a significant mode of dissemination, with individual contacts between project team members and other academics probably the interest in interacting with and applying the RSD framework.
7 CONCLUSION
8 RECOMMENDATIONS
The project team recommends that the RSD framework be explored by universities as a way to support a consolidating agenda and bring together the sometimes conflicting agendas of education and research. The RSD framework can help bring together multifaceted teaching and learning agendas, including policies for well-framed course outcomes, identifying program graduate characteristics, and assessment and feedback policies. Because the RSD describes a continuum of autonomy, it would be inappropriate for academics to be forced to adopt this as a matter of policy.
During the project's lifetime, the project team's openness to new approaches and respect for individual autonomy allowed academics to develop new, effective uses of the RSD for specific purposes. The project team recommends that academics and universities take a practice-driven approach to using the RSD framework, where policy supports the emergence of context-oriented practice. As a conceptual framework, the RSD can provide individual academics, departments, schools and universities with a vision for action, and the role of policy in this case can be to support academics' initiatives.
Diagnostic assessments in some of these often reveal a significant proportion of students struggling with basic research skills. If the pedagogical relationship between high schools and universities were more coherent and developed, the skills that are relevant for research at all levels could be explicitly learned and developed much earlier in the education process. Schools of education are crucial in this process, as they have the capacity to develop both teachers' (those in-training and those in-service) research skills and to enable teachers to explicitly transfer those research skills to primary and high school students.
The project team recommends that special attention be paid to supporting schools of education in adopting and teaching RSD approaches, due to the potential long-term benefits of increasing students' research capabilities across all faculties of the university.
9 APPENDICES
List of RSD Seminars and Workshops
Graduates: Ready for Research?” Presented at the Faculty of Civil and Environmental Engineering, University of Adelaide, November 2007.
Journal publications, conference papers and conference presentations
Paper presented at the National Summit on the Integration of Teaching, Research and Learning, Sydney. Preliminary results of the explicit development of research skills for all students during the teaching of ten disciplines. Paper presented at the Threshold Concepts: from Theory to Practice Conference, Queen's University, Kingston, Ontario, Canada.
Paper presented at the Council on Undergraduate Research Biennial Conference, College of St Benedict, St. Pauls, Minnesota, USA. Keynote presentation at the 2009 All Ireland Society for Higher Education Conference, National University of Ireland, Maynooth, Ireland. Invited presentation at the South Australian Round Table for Integration of Teaching, Research and Learning, Adelaide.
Research Skill Development framework
Example marking rubric framed by the RSD
Researcher Skill Development framework (RSD7)
QUT Resource Module Hyperlinked Organiser
Work Skill Development framework
Example interview protocol
RSD student self-assessment questionnaire
External review of the project
Bibliography
