Higher Education Pedagogy

(1)

Conference on

February 4-6, 2015

The Inn at Virginia Tech and Skelton Conference Center Virginia Tech, Blacksburg, Virginia

(2)

(3)

iii Letter from the Conference Committee

The 7

^th

annual Conference on Higher Education Pedagogy is focused on higher education teaching excellence and the scholarship of teaching and learning. The conference provides a forum for faculty members, researchers, administrators, and graduate students to showcase the latest in instructional practice and educational research; a mechanism to network with like- minded educators regarding higher education instruction; and, an opportunity to expand one’s understanding of and motivation for learner-centered instruction.

The conference has grown over its seven-year history from a one-day to a three-day event involving 250 participants all from Virginia Tech in 2009, to 1000 participants from 335 institutions, 46 states plus the District of Columbia, and 50 countries in 2015. This growth has been accomplished without a conference fee thanks to our academic sponsors and corporate sponsors.

In this time of economic challenge, it is essential that we maintain the highest standards for higher education and continue to increase the effectiveness of instruction and the depth of student learning. We are pleased to join with our colleagues to foster educational excellence through the Conference on Higher Education Pedagogy.

Peter E. Doolittle Danielle L. Lusk

Director, CIDER Assistant Director, CIDER

[email protected] Conference Chair

[email protected]

(4)

Conference Sponsors

Conference Event Sponsors General Sponsors

Undergraduate Academic Affairs Rachel Holloway, Vice Provost for Undergraduate

Academic Affairs Graduate School

Karen DePauw, Vice President for Graduate Education Vice President for Finance

Dwight Shelton, Vice President for Finance and Chief Operating Officer

Office of the Vice President for Information Technology

Scott Midkiff, Vice President for Information Technology College of Architecture and Urban Studies

Jack Davis, Dean College of Engineering

Richard Benson, Dean

College of Natural Resources and Environment Paul Winistorfer, Dean

College of Science Lay Nam Chang, Dean

Virginia-Maryland College of Veterinary Medicine Cyril R. Clarke, Dean

Office of Assessment and Evaluation Steve Culver, Executive Director Office of First Year Experiences

Mary Ann Lewis, Director Office of Undergraduate Advising

Kimberly Smith, Director Outreach and International Affairs Guru Ghosh, Vice President for Outreach

and International Affairs

Institute for Critical Technology and Applied Science Roop Mahajan, Director

University Honors Christina McIntyre, Interim Director

Pamplin College of Business Richard Sorensen, Dean

Virginia Tech Carilion School of Medicine Cynda Johnson, Dean, & Daniel Harrington, Senior Dean

Agricultural Technology Program Pavli Mykerezi, Director

Via Department of Civil and Environmental Engineering

Sam Easterling, Department Chair Department of English Joe Eska, Department Chair

Department of Fish and Wildlife Conservation Joel W. Snodgrass, Department Chair

Department of Psychology Bob Stephens, Department Chair Department of Sustainable Biomaterials

Robert Smith, Department Chair Department of Finance Vijay Singal, Department Chair

Department of Management Kevin Carlson, Department Chair

Department of Human Nutrition, Foods, and Exercise Matthew W. Hulver, Department Chair

Department of Plant Pathology, Physiology and Weed Science

Elizabeth Grabau, Department Chair Department of Political Science Timothy Luke, Department Chair

Department of Chemistry James M. Tanko, Department Chair

School of Education Steven M. Janosik, Director

(5)

v

Corporate Sponsors

KEYNOTE SPONSOR

School of Education

(6)

Conference on Higher Education Pedagogy

Planning Committee

Peter Doolittle, Executive Director Danielle Lusk, Chair Jackie Woodyard, Co-chair

Hosted by

Center for Instructional Development and Educational Research (CIDER)

Peter E. Doolittle Executive Director

Danielle Lusk Assistant Director

Tiffany Shoop Assistant Director

Patricia Sandler Senior Associate

Bonnie Alberts Administrative Assistant

Stephan Munz

Professional Development Fellow Jackie Woodyard

Professional Development Fellow

(7)

vii

Opening Keynote Address Wednesday, February 4, 2015

8:45 – 9:45 am Terry Doyle

Professor of Reading Ferris State University

A New Paradigm for Student Learning

Dr. Doyle will address A New Paradigm for Student Learning. Terry Doyle is an author, nationally recognized educational consultant and Professor of Reading at Ferris State University where he has worked for the past 38 years. From 1998 to 2009, he served as the Senior Instructor for Faculty Development and Coordinator of the New to Ferris Faculty Transition Program for the Faculty Center for Teaching and Learning at Ferris State. He is the author of the book Learner Centered Teaching: Putting the Research on Learning into Practice which was featured in the Chronicle of Higher Education's Selected New Books in Higher Education in

2012 and the book Helping Students Learn in a Learner Center Environment: A Guide to Teaching in Higher Education, published by Stylus in 2008. He is the co-author of the book New Faculty Transition - An Ideal Program published in 2004. His newest book published in August 2013, co-authored with Dr. Todd Zakrajsek, is titled The New Science of Learning: How to Learn in Harmony with Your Brain and is written for college and high school students. It has been described as breaking new ground in helping students understand how learning happens and suggests a new paradigm for how students should prepare themselves for learning.

Closing Keynote Address Friday, February 6, 2015

12:30 – 1:30 pm Paul Hanstedt

Professor of English Roanoke College

Reconsidering “Whole Person” Education:

What Do We Really Want for Our Students—and How Can We Get Them There?

“Whole Person Education” is a tired old workhorse of a cliché, probably more fit for pasture than a conversation among academics intent upon moving their institutions and their students forward in dynamic ways. This talk seeks to revitalize the concept by moving from quantitative to qualitative terms and focusing on the end-game: how can we use this language to capture what it is we’re really after for our students? And what does this mean for the kinds of education we provide—at the institutional, course, and even assignment level? And, finally, is it even possible to assess this?

Paul Hanstedt is a professor of English at Roanoke College, where he led a revision of the general education program that resulted in a theme-based core emphasizing writing-, speaking-, and quantitative reasoning-across-the curriculum. He is the recipient of a Fulbright to aid general education reform in Hong Kong, a FIPSE grant for sustainable faculty development, and several teaching awards including a 2013 State Council for Higher Education in Virginia Outstanding Faculty Award, and the 2014 CASE Virginia Professor of the Year Award. His essays regularly appear in the Chronicle of Higher Education, Liberal Education, and on Virginia Public Radio. He is the author of two books: General Education Essentials, a faculty introduction to current trends in liberal education, and Hong Konged, a travel memoir about spending a year abroad with three kids under the age of ten.

(8)

Wednesday

February 4, 2015

Presentation Sessions

http://www.cider.vt.edu/conference/

(20)

Wednesday February 4, 2015

Session 1

10:00-10:50 AM

http://www.cider.vt.edu/conference/

(21)

3 A Collaborative Approach to Improving First Year Students’ Information Literacy Skills

Maria Stack, Kathryne McConnell, Carolyn Meier, and Rebecca Miller Virginia Tech

Abstract: This study is an example of an institutional-level learning improvement in students’

information literacy skills resulting from an innovative collaboration between an assessment office and university librarians supporting a first-year experience (FYE) at a Research I university. The power of the collaboration revealed itself iteratively each year as the team worked together to

“close the loop” by implementing changes not only to the programs based upon the assessment data, but also through the systematic improvement of the assessment instrument – a locally developed Information Literacy Test (ILT). The annual revisiting of the results and the methods that generated them has helped advocate for more significant changes to the instructional design of the first-year experience vis-à-vis information literacy, moving from the standard, “surface level”

library tour and the one-off library lecture to the authentic embedding of librarians as instructional partners in the FYE classrooms. In short, this paper addresses collaborative process as a necessary component for effecting change in student learning.

Literature Review

Broadly, this study is grounded in the assessment for improvement paradigm (Ewell, 2009), which – in contrast to assessment conducted solely to comply with accreditation or accountability mandates – eschews positioning data in the best possible light in order to reveal gaps or flaws in students’ thinking. When asked about incoming first-year students’ information literacy skills, faculty often bemoan their students’ lack of knowledge, from how to conduct a simple search or identifying peer-reviewed sources to troubles with knowing how and when to cite sources

appropriately. Regardless of previous experience, it is fair to say that the library students encounter at a Research I institution is far more complex than any they have been required to use previously. Coupled with faculty’s increased expectations of students and institutional efforts designed to engage students in high-impact practices like

undergraduate research earlier and more often (Kuh, 2008), it is critical that deficiencies in students’ information literacy skills be (1) identified early and (2) corrected and reinforced throughout their undergraduate careers.

The Association of College & Research Libraries (ACRL) defines information literacy as a set of abilities requiring individuals to "recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information (as cited in ALA, 2000).” As an organization, the ACRL has shown increased interest in assessment (e.g., creating the Assessment in Action grants), and is currently reviewing and revising these standards.

University librarians, the content experts for information literacy, are feeling the pressure to hold themselves accountable to assess student learning. However, “to many librarians, the mandate for information literacy

assessment is a crisis; they need to measure information literacy skills but feel unprepared to do so” (Oakleaf, 2008, 234). Collaborating with assessment experts, however, brings together the necessary content and

methodological/process skills.

Methodology

The Information Literacy Test (ILT) was given to 1,847 freshman students during the 2013-2014 school year who were enrolled in one of the university’s First Year Experience (FYE) courses. The test, which contained 24 multiple- choice questions, was given to students during their first week in the course and then an equivalent test was given as they completed the course at the end of the semester. The post-test ILT also contained seven open-ended questions, of which three focused on research competencies, while the other four focused on academic integrity. For purposes of this study, only two of the open-ended questions related to academic integrity were analyzed.

First, we analyzed the multiple-choice section of the ILT, focusing on the change in students’ scores from pre- to post-test. In the next phase, we qualitatively analyzed the open-ended responses on the ILT. We were not looking to assess the correctness of the responses to the open-ended questions alone, but also were interested in revealing any patterns in reasoning revealed by the students, and were particularly interested in identifying whether or not there were shared misconceptions or faulty logic exhibited by the students in their answers.

(22)

Results

On the multiple choice portion of the ILT, a paired-samples t-test indicated a significant improvement from the students’ pre-test scores to the post-test scores (t(1,531) = -22.60, p < .001). In looking at the multiple choice questions specifically related to academic integrity, a majority of the students answered the questions correctly on both the pre-test (92.7%) and the post-test (95.9%). On the post-test, students were also asked several open-ended questions relating to academic integrity. (We are still analyzing this data; results reported here are preliminary in nature). For the first question, students were asked to list reasons it is important to cite sources. Almost all of the students were able to correctly articulate at least one reason it is important to cite sources. Many students listed three plausible reasons to cite sources. The most common themes were 1- giving credit to the original author, 2-

establishing credibility in your own work, and 3- differentiating your contribution from previous work.

For the second open-ended question, students were asked to read two passages and to choose and explain which one had been correctly paraphrased. 78.13% of students correctly identified which passage was paraphrased. Regardless of whether they chose the paraphrased passage, students’ justifications for their answer varied. From their responses, several themes emerged. First, many study were able to articulate that one passage was paraphrased, while the other passage was clearly plagiarized as it followed too closely to the original. Students responding in this manner understood and were able to explain the concept of plagiarism.

Other students, who also chose the correct response, supported their response by focusing on the grammar and sentence structure, rather that the issue of plagiarism. Although the students were able to identify the passage that was correctly paraphrased, they did not appropriately justify their answers. Lastly, for the students who chose the incorrect answer, they reported that this passage was closer to the original idea, while the other student had strayed too far from the original and adding in opinions, as demonstrated here: “Student 1 is correct because the information he pulls out is directly from the passage and not extrapolated. Student 2 makes the assertion that the children are the focus of the book however the passage does not talk about this. Even though the book might be about this, the passage does not explicitly define this.”

Discussion/Conclusion

This study holds significance for several reasons. First, it represents a methodologically rigorous assessment process that has facilitated learning improvement within an institutional-level curricular priority, the University’s first year experience. Second, the authors assert that this assessment process would not have been successful were it not for the collaborative nature of the effort, one characterized by sustained engagement on the part of both content and assessment/methodological efforts. Finally, the ability to capture evidence of students’ reasoning and/or misconceptions associated with the application of information literacy skills may lead to more sophisticated and nuanced instructional interventions that may lead students to “a broader view of information (that)… will serve our students better in the long run than precise expertise in the more mechanical aspects of information literacy” (Scales

& Lindsay, 2005, p. 521).

References

American Library Association [ALA]. (2000). Information literacy competency standards for higher education.

Chicago: The Association of College and Research Libraries.

Ewell, P. T., (2009). Assessment, accountability, and improvement: Revisiting the tension (Occasional Paper No. 1).

National Institute for Learning Outcomes Assessment.

Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter.

Association of American Colleges and Universities.

Oakleaf, M. (2008). Dangers and opportunities: A conceptual map of information literacy assessment approaches.

Libraries and the Academy, 8(3), 233-253.

Scales, B. J., & Lindsay, E. B. (2005). Qualitative assessment of student attitudes toward information literacy.

Libraries and the Academy, 5(4), 513-526.

(23)

5 Faculty Perceptions of Course-Level Assessment in Higher Education

Barbara Howard, Sara Olin Zimmerman, Anthony Jones, & Amy Trawick Appalachian State University

Abstract: Institutions of higher education have traditionally used assessments for accreditation and for program accountability to show evidence of effectiveness at a campus level. These actions may be reducing assessment discussions to a record of accomplishments and discouraging more in-depth conversations regarding assessment at the class and student levels. For decades research has shown a myriad of successful uses of types of assessment including formative (Sadler, 1998;

Hounsell, 2003), journals and portfolios (Woodward, 1998; Dyment & O’Connell, 2010), and constructive alignment (Boud & Falchikov, 2006) as important initiatives that promote student learning in higher education. And yet conversations may not be occurring regarding the commonalities of assessment practices, the focus on course content or student learning, and the use of constructive alignment. To promote conversation among faculty and to gain valuable information for improving practice, a survey was distributed to determine types of assessments typically used, purposes for assessments, scoring methods and the use of technology in assessment. The results of this survey give a glimpse into the perceptions of one campus’ faculty on their own assessment practices.

Literature Review

Assessment practices in higher education classrooms may be nebulous at best, capricious at worse. Grading practices that may be poorly defined and implemented often result in hotly debated controversies surrounding grade inflation. Thus far, higher education classrooms have escaped the types of scrutiny currently leveled at k-12 classrooms. Yet, the anvil of accountability is steadily approaching graduate programs, particularly those at state universities where state legislators and general administrations demand evidence of effectiveness. This scrutiny may be warranted in the area of assessment. Clearly higher education relies on end of course assessments even though research on summative assessments has not supported these practices as positively affecting student learning (Eccleston, 1999). What are faculty perceptions of assessment practices? Do faculty have the knowledge base to align the curriculum with appropriate forms of assessment to provide feedback to improve student learning? Is there a common understanding of assessment among faculty? Is the emphasis more on content and less on student learning? Is the paradigm of academic freedom impeding efforts to engage in more meaningful assessment? Do current assessment practices equip students well for a lifetime of learning and future assessment challenges? These are all questions that need to be constantly checked to determine the pulse of assessment practices in higher education.

In the past, studies have not focused exclusively on the alignment between approaches of teaching and different features of assessment (Fanghanel, 2009). Academic disciplines may have different structural practices in teaching and assessment. Traditionally labeled “hard disciplines” such as math, physics, and engineering often use lectures and problem-based learning while “soft disciplines” such as history, philosophy and education use more discussions and debates in their teaching approaches (Trigwell & Prosser, 2014). Therefore it makes sense that different approaches are used with assessment to document learning. This is the concept of signature pedagogies in which unique disciplines use their own teaching methods and therefore their own assessments (Shulman, 2005).

An old concept in education, but a relatively new paradigm in higher education is that of constructive alignment.

This is the idea that for teaching to be beneficial there must be an alignment between content, learning outcomes, teaching and assessment (Boud & Falchikov, 2006; Fanghanel, 2009). In this paradigm there are two distinct views of teaching and assessment. One is the view that knowledge is created externally from the student and teaching becomes teacher-centered dissemination of knowledge. In contrast, the other views acquiring knowledge as student- centered and therefore teaching focuses on conceptual change and the development of information. These

approaches result in distinctly different approaches to assessment. The teacher-centered approach regards

assessment as separate from teaching and the student centered approach views assessment as an integral part of the teaching and learning process (Trigwell & Prosser, 2014).

(24)

Methodology

This study, conducted at a mid-sized state university in the Southeastern United States surveyed 901 faculty that teach in more than150 programs of study. The general framework for data analysis was descriptive. Along with demographics, questions were asked regarding types of assessments typically used, purposes for assessments, scoring methods and the use of technology in assessment. This survey was viewed as an impetus for promoting conversation among faculty in the hopes of recognizing research based practices as models and identifying areas of assessment to strengthen with professional development. The survey was digitally disseminated through the university’s faculty development center.

Data Analysis and Discussion

Results of the survey will be presented along with a discussion of possible reasons for the responses. A descriptive analysis was completed to analyze results at the content (college) level, the course levels taught, current faculty rank, total years of teaching experience in higher education, and the delivery mode of the courses. The responses were also categorized in terms of a theoretical framework for faculty willingness to engage in assessment:

institutional culture of assessment, perceived benefits, perceived time investment and perceptions of assessment as a scholarly activity (Wang & Hurley, 2012).

References

Boud, D. & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education, 31(4), 399–413.

Dyment, J.E. & O’Connell, T.S. (2010). The quality of reflections in student journals: A review of limiting and enabling factors. Innovative Higher Education, 35(4), 233-244.

Ecclestone, K. (1999). Empowering or ensnaring?: the implications of outcome-based assessment in higher education, Higher Education Quarterly, 53(1), 29–48.

Fanghanel, J. (2009). The role of ideology in shaping academics’ conceptions of their discipline. Teaching in Higher Education, 14, 565–577.

Hounsell, D. (2003). Student feedback, learning and development, in: M. Slowey & D. Watson(Eds)

Higher education and the lifecourse (Buckingham, Society for Research into Higher Education and Open University Press), 67–78.

Kember, D., & Kwan, K.P. (2002). Lecturers’ approaches to teaching and their relationship to conceptions of good teaching. In N. Hativa & P. Goodyear (Eds.), Teacher thinking, beliefs and knowledge in higher education,

Chap. 9 (pp. 219–239). London: Kluwer Academic Publishers.

Sadler, D. R. (1998). Formative assessment: revisiting the territory, Assessment in Education, 5(1),77–84.

Shulman, L.S. (2005). Signature pedagogies in the professions. Daedalus 134(3) 52-59.

Wang, X. & Hurley, S. (2012). Assessment as a scholarly activity? Faculty perceptions of and willingness to engage in student learning assessment, The Journal of General Education, 61(1), 1-15.

Woodward, H. (1998) Reflective journals and portfolios: Learning through assessment, Assessment and Evaluation in Higher Education, 23(4), 415–423.

(25)

7 Concepts, Connections, and Constructivism: Mind Mapping for Pre-service Teachers

Keonya Booker, College of Charleston

Abstract: Constructivist theory asserts that students attach meaning to their learning by way of predicting, organizing, and evaluating information. Instruction in constructivist-based classes should support students as they actively connect new ideas with prior knowledge. Mind mapping allows students to make associations between both abstract and concrete types of information in creative and imaginative ways. This practice session will explore the use of a popular software program to help pre-service teachers understand major theoretical perspectives in a human development course. Particular attention will be paid to student perceptions of both the tool’s functionality and benefit to learning.

Literature Review

Brainstorming is a concept that teachers have used for decades. Conventional directions for a brainstorming activity would involve the teacher giving minor prompts to students and leaving them to figure out

connections on their own. Research shows that students are more actively engaged and retain information when they make associations with prior knowledge (Davies, 2011). Mind mapping, in more recent times, is a systematic way of organizing images with text in order to facilitate a deeper understanding of information (Johnson, Podjed, & Taasan, 2013). Students can formulate projects, portfolios, and presentations all with the use of a concept map.

Mind maps that are interactive also show great promise. When students collaborate, there seems to be a multiplicative effect whereby they report being better able to process information and elaborate on specific ideas (Naykki & Jarvela, 2008). The editorial nature of mind maps also lets students reflect on their comprehension of particular ideas before committing to a position. If students work together on one mind map, they show a greater shared understanding of task requirements and course objectives (Naykki &

Jarvela).

Goals and Objectives

This practice session will begin with a review of major principles of constructivist education and how such an approach reinforces the use of mind maps. We will then turn our attention to how mind mapping supports student learning outcomes in a core class for pre-service teachers. The session will continue with a discussion of Popplet, a web-based application that helps students make visual and text connections between ideas. Collaboration with others is another advantage when using Popplet. At the conclusion of the session participants will have learned how to leverage an easy and straightforward tool to help students build on prior knowledge stores. Implications for learning styles will also be addressed.

Description of Practice

Mind mapping has myriad uses in education and there are several tools instructors can use to support student learning. At the College of Charleston all elementary and secondary pre-service teachers must successfully complete a Human Development course prior to their fieldwork experience. Since lifespan development traditionally takes a survey approach, breadth of information is emphasized, not necessarily depth. Because we want students to have a strong understanding of a particular topical issue prior to their practicum, the cumulative assignment was developed. For the cumulative assignment, each student group is responsible for exploring a developmental theorist (e.g., Piaget) or current educational issue (e.g., Common Core) and then presenting to the rest of the class. Students are required to use Popplet to coordinate their work and show connections between contrasting ideas. Uses for Popplet include editing, organizing, and drafting mind maps which will be demonstrated in the session. Participants will also see examples of student work and hear how students evaluated the use of the tool in terms of functionality and worth to the overall project.

(26)

Discussion

Constructivist principles stress not only the product of student learning, but the process as well. Instructors should encourage students to use innovative ways to display their understanding of course concepts. Mind mapping is a useful tool for instructors to employ when introducing complicated or abstract information.

Having students collaborate while constructing a mind map shows an additional benefit as ideas and connections must be evaluated, judged, and defended (Fonteyn, 2007). Applications of this approach can be realized in a multitude of disciplines and areas of inquiry.

References

Davies, M. (2011). Concept mapping, mind mapping, and argument mapping: What are the differences and do they matter? Higher Education, 62, 279-301.

Fonteyn, M. (2007). Concept mapping: An easy teaching strategy that contributes to understanding and may improve critical thinking. Journal of Nursing Education, 46, 199-200.

Fortson, K. (2012). Six creative storytelling tools that will make you wish you were a student again. THE Journal, 39, 8.

Johnson, M. L., Podjed, S., & Taasan, S. (2013). Engaging honors students in purposeful planning through a concept mapping assignment. Honors in Practice, 9, 73-84.

Naykki, P., & Jarvela, S. (2008). How pictorial knowledge representations mediate collaborative knowledge construction in groups. Journal of Research on Technology in Education, 40, 359-387.

Wanago, N. (2013). Effective web 2.0 tools for your classroom. Connecting Education and Careers, 88, 18-21.

(27)

9 Measuring the Promise: Assessing Syllabi Using a Valid & Reliable Rubric

Michael S. Palmer, Dorothe J. Bach, and Adriana C. Streifer, University of Virginia Abstract: This practice session will provide an introduction to and hands-on practice with a newly developed syllabus rubric that assesses the degree to which a syllabus achieves a learning-focused orientation. The rubric provides qualitative descriptions of 16 components and uses a quantitative scoring system that places syllabi on a spectrum from content- to learning-focused. It is nuanced enough to provide summative information to developers using the tool for assessing their course design initiatives and formative feedback to instructors interested in gauging the focus of their syllabi. After a brief introduction of the tool, participants will have a chance to test its functionality and usability by applying it to sample syllabi. At the conclusion of the session, the presenters will briefly describe the results of scoring over 50 pre- and post- syllabi pairs collected from instructors that participated in our week-long Course Design Institute.

Introduction and Review of Prior Literature

Guided by Kreber and Brook’s six-component impact model [1], we have developed a series of assessment

instruments to systematically investigate the arc of the educational transformation process associated with our week- long course design institute (CDI). In general, CDIs aim to help instructors create rich, active classroom

environments grounded in evidence-based practices; expand their pedagogical content knowledge; become reflective practitioners; and foster teaching community and personal growth [2]. In addition to these goals, our own week-long CDI is designed to help participants create what Ken Bain calls a “promising syllabus” [3]: a learning-focused document that communicates clearly and compellingly what students will gain from the course, what they will do to achieve the promise it lays out, how they will know whether they are getting there, and how to best go about studying. We emphasize the syllabus because of its capacity to serve as a framework for designing meaningful learning environments.

In this session, we will introduce—and offer participants a chance to use—one of our assessment tools: a valid and reliable rubric designed to assess the quality and alignment of learning objectives, assessments, activities, schedules, and overall motivational tone in course syllabi (see http://trc.virginia.edu/resources/syllabus-rubric/) [4]. The development of the rubric was guided by the literature on learning-focused course design, teaching, and student motivation and influenced by existing literature on syllabus construction and syllabus components [see, for example, reference 5-7].The rubric accounts for the nuances and variety of shapes of learning-focused syllabi by subdividing the five major criteria into 16 distinct components. Each component is scored based on the strength of supporting evidence: strong, moderate, or low. A multi-directional weighting scheme is applied in order to ensure that the final score reflects the presence and quality of essential components (e.g. learning objectives). The maximum score a syllabus can achieve is 58; content-focused syllabi fall in the range 0-18, transitional 19-40, and learning-focused 41-58.

We have scored over 50 pre- and post-CDI syllabi pairs collected from participants during the seven years we have offered our Institute. Our preliminary results indicate that all instructors move toward the learning-focused end of the scale after participating in our CDI. Instructors who initially had highly content-focused syllabi and scored below 15 on our rubric gain as many as 20-30 points following CDI. Instructors who began with more learning- focused syllabi also improve but exhibit smaller gains.

Goals & Objectives for the Practice Session

This practice session will focus on providing participants, both instructors and educational developers, with a basic understanding of the syllabus rubric and an opportunity to learn to use it as a formative assessment tool.

Upon completion of the session, participants will:

• be able to articulate the basic purposes, functions, and limitations of the syllabus rubric;

• be able to use the syllabus rubric to score a range of syllabi;

• be aware of the basic findings of our ongoing study assessing the impact of our CDI;

(28)

• consider how they may use the rubric to create learner-focused syllabi (faculty) and/or assess their course design programming (educational developers).

Session Outline and Materials

After a brief introduction to our overall CDI assessment project and the syllabus scoring system, each participant will receive a copy of the syllabus rubric, the quantitative scoring tool, a user guide and two sample syllabi (10 min).

Participants will spend the majority of the time learning to effectively use the rubric and score the sample syllabi, placing each on a spectrum from content- to learning-focused (18 min). Following the individual work with the rubric, participants will debrief as a large group (12 min). The session facilitators will then compare the participants’

scores to those given by trained raters and briefly summarize the results of scoring over 50 pre- and post- syllabi pairs collected from instructors that participated in our week-long CDI (5 min). The final portion of the session will be devoted to Q & A and discussing ways that the syllabus rubric could be useful within the participants’ unique professional roles and institutional contexts (5 min).

References

Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press.

Doolittle, P. E., & Siudzinski, R. A. (2010). Recommended syllabus components: What do higher education faculty include in their syllabi? Journal on Excellence in College Teaching, 21(3), 29–61.

Harnish, R. J., & Bridges, K. R. (2011). Effect of syllabus tone: Students' perceptions of instructor and course.

Social Psychology of Education, 14(3), 319–330.

Johnson, T., Nelms, G., Linder, K. and Palmer, M. (2012). Exploring the range of multi-day course design institutes.

Presentation at 2012 POD Conference, Seattle, WA.

Kreber, C., & Brook, P. (2001). Impact evaluation of educational development programmes. International Journal for Academic Development, 6(2), 96-108.

O’Brien, J. G., Millis, B.J., & Cohen, M. (2008). The course syllabus: A learning-centered approach (2nd ed.). San Francisco: Jossey-Bass.

Palmer, M. S., Bach, D. J., & Streifer, A. C. (2014). Measuring the promise: A learning‐focused syllabus rubric. To improve the academy: A journal of educational development, 33 (1), 14-36.

Higher Education Pedagogy - UBBG Institutional Repository

Conference on