Research: Unveiling the Potential of Peer Assessment in Mathematics Through Adaptive Comparative Judgement in HE

By Mark House

Peer assessment has long been a valuable tool in the educational toolkit, offering students the chance to engage with the grading process and learn from their peers. But what happens when we strip away the traditional assessment criteria and dive into the world of comparative judgement? A fascinating study by Ian Jones and David Sirl provides some compelling insights.

The Power of Comparative Judgement

The study, conducted with first-year undergraduate students, delved into the realm of calculus, a subject where conceptual understanding is as crucial as procedural knowledge. The researchers employed a method known as comparative judgement (CJ), which eschews detailed rubrics in favor of direct, holistic comparisons of student work.

The Study at a Glance

  • Participants: 141 first-year mathematics undergraduates.
  • Method: Students were given a test question in advance and wrote their answers under exam conditions. These were then uploaded to an online CJ engine.
  • Process: Students completed at least 19 pairwise judgments to determine which response showed better understanding.
  • Outcomes: The study reported high reliability and validity, with a Scale Separation Reliability (SSR) of 0.89 and a median inter-rater reliability of 0.80.

What Makes Comparative Judgement Different?

CJ taps into our innate ability to make consistent comparisons. It's like judging which of two objects is heavier – a task we can do with surprising accuracy. This approach has been made possible in educational settings thanks to technological advancements, including the internet and the Rasch model, which help scale judgments and produce reliable scores.

The Results Are In

The study's findings were clear: peer assessment using CJ can produce valid and reliable outcomes suitable for summative grading. It also shed light on how students make judgments without explicit criteria, focusing on presentation, understanding, and technical aspects.

The Takeaway for Educators

This research suggests that CJ can be a practical and effective method for peer assessment in mathematics. It offers a fresh perspective on assessment, one that could transform how we evaluate student understanding in complex subjects like calculus.

Looking Ahead

As we continue to explore innovative assessment methods, studies like this one by Jones and Sirl are invaluable. They not only demonstrate the feasibility of new approaches but also their potential to enhance educational outcomes. It's an exciting time for educators willing to step outside traditional boundaries and embrace the possibilities of comparative judgement.

Stay tuned to RM Compare's blog for more insights into the future of assessment and how tools like RM Compare can support these innovative practices. For more information on how RM Compare is revolutionizing assessment, visit our blog and discover a world where assessment is less about ticking boxes and more about understanding and improvement.

Reference

Jones, I., & Sirl, D. (2017). Peer assessment of mathematical understanding using comparative judgement. Studies in Higher Education, 39(10), 1774–1787.