Pilot Study on ‘Where-To-Next’
Feedback in the New Zealand Diploma of Business.
Kearns, N., Wessels, A. & Robertson, C.
School of Applied Business
Te Pukenga–NZIST (trading as) Unitec UREC approval 2022-1039
“Feedback is essential for student learning provided that feedback is accurate, timely and specific. . . . Feedback is among the greatest influences on student achievement”
Hattie &Timperley (2007).
“The major finding was the importance of ‘where to next’ feedback which led to the greatest gains from first to the final submission.” Hattie et al (2021).
Why use where-to-next feedback?
• Could WTN feedback be used to help close the SCC parity gap?
• Ongoing issue of an SCC parity gap of 10 – 20% for priority groups in NZDB and BBS courses.
• First semester of study. . .so early ‘feedback literacy’ development could be very effective.
• Where-to-next feedback has three components:
• Issue
• what is wrong or missing / what can be improved here.
• Relevance
• why it is wrong / what needs to be improved.
• Action
• what you should do to fix it / how to improve the answer.
The research question & context
What students get as feedback
(Hattie et al 2021)31
50
19
0 10 20 30 40 50 60
feedup feedback feedforward
What students get in feedback (%)
What’s the goal? How have I done so far? Where to next?
What students want in feedback
(Hattie et al 2021)0 10 20 30 40 50 60 70 80 90
“least useful”
“most useful”
0 10 20 30 40 50 60 70 80 90
0 10 20 30 40 50 60
feedup feedback feedforward
. . . combining the graphs. . . WTN is the sweet spot
• WTN feedback explained in first two classes.
• First assessment used as WTN opportunity.
• Assessment ran for ten weeks.
• Four opportunities to submit and receive WTN structured feedback – weeks 6, 7, 8, 9.
• Strong promotion/encouragement of WTN opportunities in class.
• “Opt in” or “opt out” at any time.
• Class was surveyed ‘before’ and ‘after’ the WTN experiment.
Method
Demographic class data (50% in BBS / 50% in NZDB)
33%
6%
33%
10%
4%
14%
Class by Priority Groups
U25 Maori Pacifica International Pakeha
Other 70%
8%
12%
8% 2%
Class by age group
U25 26 - 30 31 - 40 41 -50 50+
Lifestyle class data
4% 11%
46%
30%
9%
Study & other commitments
Study part time /other part time (4%) Study part time / other full time (11%) Study full time / other part time (46%) Study full time / other full time (30%) Study full time / no other (9%)
76% of class were in . .
full time study / full time other full time study / part time other 14 of 18 Pacific students in this group
Results: Draft submissions for WTN feedback
17 students participated making 25 submissions in total.
3 3
6
13
WK 6 WK 7 WK 8 WK 9
Number of submissions by week (Due date week 10)
11
4
2
0
ONE TWO THREE FOUR
Number of submissions per student
Results
29%
7%
14%
43%
7%
Students participating in WTN Average mark = 66.4%
Intl U25 Pac U25 U25
not priority Maori
14%
36%
36%
7%
7%
Students not participating in WTN Average mark = 44.9%
Intl U25 Pac U25 U25
not priority Maori
Participating students’ views of WTN
(7 BBS students and 10 NZDB students participated.)
20%
53%
27%
Impact on marks
no impact improved
improved more than expected
33%
67%
Impact on understanding assessments
no impact on understanding improved
Some participating students’ comments. . .
“ was useful in pushing me to work on my assessment each week, and gave me a better understanding of what was required”
“very interesting and super helpful in understanding the assessment and getting better marks”
“provided details of what is the problem and actions to fix it”
Two WTN feedback examples.
on strategy
Issue: too many directions for the strategy in above three para.
Relevance: unfocused strategies don’t deliver.
Action: either align the three strands somehow, or drop one of the strategies. (Don’t need to be “all things to all people".)
on assessment processes / skills
Issue: diagram is not related to your choc business (statements are too general)
Relevance: you need to show a case for change for the business you are using in this assessment, so words on all diagrams need to relate to this.
Action: please modify the diagram to refer to your choc business.
Why not participate?
7%0%
86%
7%
Reasons for not doing WTN
didn’t understand FFWTN
didn’t see how it would help
didn’t have time for early draft
only want to pass
“ I wanted to submit an early draft . . . however I was too busy at work”
“I was sick and got an extension, and missed the early draft dates”
Final comments – on reflection
• Using a WTN s
tructure . .• gives more consistent feedback.
• Focuses your attention on ‘what am I looking at?’/‘how to improve’ /‘what to say here’.
• Supports excellent conversations in class following feedback.
• Increases marks for participating students – but takes more time.
• The barriers to WTN for time-poor priority groups exist outside the academic scope.
• Provides further support for flipped learning.
• Less ‘delivery’ / more ‘feedback’
Next steps
• Expand the study to level 6 and 7.
• Refine the approach with purpose designed assessments.
• Looking for collaborators. . .please get in touch.
• Nick Kearns [email protected]
Thank You for your time Any questions?
Kearns, N., Wessels, A. & Robertson, C.
School of Applied Business
Te Pukenga–NZIST (trading as) Unitec UREC approval 2022-1039
References
Hattie, J., Crivelli, J., Gompel, K., West-Smith, P., & Wike, K. (2021) - Feedback That Leads to
Improvement in Student Essays: Testing the Hypothesis that “Where to Next” Feedback is Most Powerful.
Frontiers in Education 6, 1 – 9. https://doi.org/10.3389/feduc.2021.645758
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.
https://doi.org/10.3102/003