From Assessment to a Learning Experience: A Transformational Journey with RM Compare

The Challenge

Purdue University is a public research university in Indiana with over 44,000 students. The course of ‘Design Thinking in Technology’ offered here found that learners often struggle with a point-of-view (POV) task that encourages them to approach from the end user’s perspective. As a result, Dr Scott Bartholomew, a technology educator at the university, gave students the opportunity to view and assess work from previous students to see if it would help in their understanding of the POV task and ultimately improve their work.

You can read the full case study HERE.

The Chosen Digital Assessment Solution

RM Compare is designed to enable a flexible approach to formative assessment and collaborative learning. Its adaptive comparative judgement (ACJ) technology is grounded in the Law of Comparative Judgement that proves people can naturally make better paired judgements than absolute ones. Further to this, its cloud-based software renders it highly scalable: half of the 550 first year students in the Design Thinking in Technology course at Purdue University were randomly selected to participate using RM Compare or as part of the control group. This is the largest-ever study of ACJ to date.

The Logistics

The ACJ group could view work submitted by previous students who had undertaken the POV task with a different assignment brief. This was achieved using RM Compare to display two anonymised pieces of work at one time. ACJ students then evaluated which one more closely met the criteria, while the control group assumed traditional classroom approaches of teacher-led discussions and direct sharing of feedback. For validity, all other elements of the course such as teachers and rooms were unchanged.

Following this, work from both groups were loaded into a second ACJ session using RM Compare where teachers collaboratively evaluated the work through comparison. Drawing on the innate nature of the comparison process, both students and teachers found the technology intuitive and enjoyable to use, and students additionally found it beneficial to view a wide range of work. This is particularly useful in open, creative subjects like English, Art or Design Technology.

Within the technology, there is a unique algorithm that intelligently selects and pairs work based on previous judgement. This significantly reduces the time it takes to accurately rank work through a professional consensus. This rank can be used an informative learning tool: displaying where work sits within the cohort and starting engaging discussions with students about results.

The Results

“Interestingly, the use of ACJ technology did not just benefit either the highest or lowest achieving students. It actually boosted the attainment of all the students who used RM Compare as a learning intervention”

• Seven out of the top ten highest performers on the task were part of the ACJ group that used RM Compare.
• The rank order showed students using RM Compare to evaluate earlier work performed significantly better than their peers using traditional methods.

By exposing students in the ACJ group to a broad range of work from former students, it helped them to identify what a ‘good’ piece of work looks like. These comparative decisions helped to also internalise the learnings of the exercise and course, and these learnings are then solidified as students verbalise their feedback to peers. This feedback becomes a formative learning opportunity that they can apply to their work, that positively impacts attainment for students at various stages of their learning journey.