Computing Education Research
Anders Berglund Informationsteknologi Uppsala universitet Uppsala
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Computing education research at
Uppsala University
How do our students understand computer
science concepts?
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Computing Education Research
This talk
What can we say about how grading in an
internationally distributed project-based course?
and
Why and how to do research in computing
education?
Two parallel stories
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
1. Background
• A research approach? • Phenomenography
2. Results concerning grading
• The setting
• The teachers’ grades
• The students’ peer evaluation • The experience of being graded • Results concerning grading
3. Summary
• Computing Education Research • Some research results
• Choosing a research approach
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
What does it mean to learn
something?
Unfortunately (?)….
General case:
A “meaningless” question
It all depends on
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Quantitative/Qualitative research
Quantitative research is grounded on
• “… the assumption that features of the social environment
constitute an objective reality … collecting numerical data on observable variables”
Qualitative research is grounded on
• “… the assumption that individuals construct a social reality
in the form of meanings and interpretations. … studying … intensively in natural settings”
Implications for the role of the researcher, the concept of evidence, interpretation etc.
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
A research
approach/methodology/
framework
Offers a way to perform research in
learning
Organizes “ways to see things”
A lens with a certain focus
With a specific research approach:
Some issues get clearer, others blurred
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Research approach/methodology/framework
Phenomenography
(Marton & Booth, 1997)
Takes the learners’ perspective.
Aims at analysing and describing the variation
in students’ experience (understanding, learning).
A empirical, qualitative research approach Data is often collected through interviews
Outcome: A few qualitatively different ways, in
which something is understood within a student cohort
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Phenomenography
Students taking a course
TCP
Researcher
The researcher studies the different ways in which the students understand TCP
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Why the learners’ perspective?
A teacher’s understanding of how her
students understand and learn about
something (CS concepts, for example TCP)
is a good tool for improving teaching.
A change that is not perceived as “good”
by the students does not improve
learning.
Example: Grades are not the driving force for
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
1. Background
• A research approach? • Phenomenography
2. Results concerning grading
• The setting
• The teachers’ grades
• The students’ peer evaluation • The experience of being graded • Results concerning grading
3. Summary
• Computing Education Research • Some research results
• Choosing a research approach
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Background:
The Runestone initiative
A project course in computer systems
A project-based course in distributed systems,
real-time programming and computer communication.
Third/fourth year students majoring in CS.
International collaboration for students who do
not go on exchanges.
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Project course in computer systems
The Runestone project
USA Sweden
• 3 + 3 advanced CS students per team • 16 teams in total
• No lectures
• Tutoring by e-mail and chat
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Student project
Student project: Produce a software system to control a
(modified) Brio labyrinth from any Web-browser.
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Both the process and the project are graded, in
relation to the teams’ own plan
Process grade is based on weekly meetings Components of grade:
• Team performance, in relation to the team’s own plan • Individual contribution
• Peer evaluation
• The instructor’s decision.
Team members are graded by “their” instructor Different grading schemes in Sweden and US
• Sweden: pass/fail • US: A to E
Grading in Runestone
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Analysing the grading in Runestone
Teachers’ distribution of grades (quantitative)
Peer evaluation
The students’ evaluation of each others’ contributions
(quantitative)
Students’ experienced purpose of being graded
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Grades
Grades awarded by the instructors, according to
the Runestone scheme (Max = 100, Pass ≈ 60)
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Peer evaluation
From Swede to Swede 22,25
From Swede to American 18,79
From American to American 20,07
From American to Swede 20,07
Each student awarded USD 120.- to his
team-mates
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
The experienced purpose of being
graded
Cat Getting a good grade …
1.
2.
3.
… has a value on its own
… is a tool to reach other aims
… is sub-ordinated to other aims
Focus is on
The grade per se
The benefits of a good grade
Me and the team My team and
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Results on grading
Getting a good grade is not the driving force for
most students in this project.
”Me in the team” or ”My team in front of other
teams” is often important.
How generalizable are these results?
• Research in computing education research are normally
situationally bound
How can we use this in our teaching?
• Both ”hard” results and insights gained by doing the
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
1. Background
• A research approach? • Phenomenography
2. Results concerning grading
• The setting
• The teachers’ grades
• The students’ peer evaluation • The experience of being graded • Results concerning grading
3. Summary
• Computing Education Research • Some research results
• Choosing a research approach
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
On Computing Education Research
Theoretically sound research in students’
learning in Computing can serve to improve teaching.
Different research approaches offer various
contributions to our understanding of students’ learning.
The perspective on ”reality”, what can be
studied, what can be known, what the
researcher’s role is, how research is performed etc. varies.
This talk: Some examples of
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Example, research approach
Constructivism
A family of traditions
Jean Piaget, 1896 - 1980
• Reality is rejected or irrelevant
• Knowledge is constructed by each individual • No firm methodology
• Passive learning will fail
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Empirical results from constructivism
in CS Education
Students construct rules for parameters. They
are only sometimes successful. (Fleury, 1991)
Students construct their own understanding of
variables. (Paz, 1996; and others)
Software visualization in itself does not help
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Applications of constructivism in
Computer Science Education
Think twice when using visualizations
Explicitly teach the model of the computer Don’t start with abstractions
Teach planning, teach to avoid “bricolage” Don’t run to the computer
Organize “closed labs”
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Example, research approach: A
socio-cultural perspective
Thinking/Learning is not influenced by the environment
⇔
Thinking/Learning is an interaction between the individual and the environment
A family of traditions in research into learning Common “source of inspiration”: Lev Vygotsky
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Applications of the soci-cultural
tradition
Why do teams of students interpret a
programming task so differently? (Holland & Reeves, 1996)
Why do our students hand in “incorrect”
programs? (Ben-David Kolikant, 2005)
The example: Open source community
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Example, research approach:
Critical enquiry
Critical enquiry (research with “a mission”, often
to address power imbalances)
• Feminist research
Cannot be defined in ontological stand or
research methodology
Instead, it questions fundamental principals and
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Feminist research, an example of
critical research
Are there factors within computer science itself,
that preserves the currently dominating gender structure? (Björkman & Trojer, 2002)
“We consider it of vital and decisive importance that gender research is done from within computer science.”
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Why phenomenography? A
personal view
Appropriate for the research questions
• “Complex” “answers” desirable
• Statistical “answers” hard or impossible to get
“Close to” learners
• Students’ perspective
• A way for students to “talk” to teachers/organisers • Data stems from individuals
Computer science is in focus
• Results “talk to” computer scientists
• Supports deployment of results in teaching
• Relevant for recognition of CSE research within CS
Competence at hand/tradition
• Shirley Booth
I like it
Selecting a research approach
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
Literature
Clancy, M., Stasko, J., Guzdial, M., Fincher, S., & Dale, N. (2001).
Models and Areas for CS Education Research. Computer Science Education, 11(4), 323-341
Fincher, S., & Petre, M. (2004) Computer Science Education
Research, London, UK: Taylor & Francis
Berglund, A., Daniels, M. and Pears, A. (in press). Qualitative
Research Projects in Computing Education Research: An Overview. To appear in the Proceedings of the 8th Australasian Computer Science Education Conference, Hobart, Australia.
Berglund, A. 2005. Learning computer systems in a distributed
project course: The what, why, how and where. Acta Upsaliensis Universitatis. Uppsala Dissertations from the Faculty of Science and Technology 62
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
i
CeTUSS
CeTUSS ska verka för anpassning av
teknikutbildning till samhällets och
studenternas behov.
Genom att utveckla och sprida
information om lärandemiljöer som är
• Personligt meningsfulla • Socialt relevanta
• Gränsöverskridande
• Baserad på samarbete (lokalt och internationellt)
Workshops, kurser etc.
In
fo
rm
a
ti
o
n
st
e
kn
o
lo
g
