- Product
An introduction to Adaptive Comparative Judgement (ACJ)
When you go to get your eyes tested, how does it work? Why aren’t you just passed a pair of glasses and asked how good they are? As anyone who have been and done this will know, it is far easier to make judgments when we have two things to compare. This concept was explored by a psychometrician called Louis Thurstone in the 1920s. Thurstone studied how people measured the severity of crimes and found that, whilst they struggled to evaluate the severity of a crime in isolation, they could easily compare two crimes and say which was more serious. From his work, he formulated the ‘Law of Comparative Judgement’, which found that humans are more accurate, exact and reliable when comparing two stimuli, rather than passing absolute judgement on just one.
As a method of assessment for learning, the benefits of using Adaptive Comparative Judgement (ACJ) are clear. It leads to a richer understanding of pupil performance, less time spent on the overall assessment process (marking and moderation), and clearer insight for both learners and teachers into the quality of work, from poor to good and, indeed, exceptional. Deployments of ACJ in schools, colleges and universities, both in the UK and overseas, have proven that its application leads to improved learner attainment.
Comparative Judgement as an assessment method
In an assessment context, rather than marking individual pieces of work against a set of marking criteria, comparative judgement allows assessors to compare two pieces of work side-by-side and decide which is better. This is done multiple times by multiple assessors to ensure a high level of reliability. Adaptive Comparative Judgement makes this process even more efficient, as an algorithm adapts how often each piece of work needs to be seen and judged based on the results of previous rounds of comparison. This helps achieve reliable professional consensus on the rank order of all the pieces of work being assessed much more quickly.
The comparison process – together with the collaborative, multi-assessor approach that allows comparisons to be made on a national and international scale – improves teachers’ assessment skills and enables teachers and learners alike to understand how criteria for success is interpreted. In a recent study conducted by Dr Eva Hartell and Professor Inga-Britt Skogh, looking at the use of comparative judgement in Swedish technology education, the participating teachers reported that they liked being exposed to work other than just their own students’. It made them consider their classroom practices and refine how they articulated criteria for success – ultimately benefitting their own pupils.
Comparative Judgement as a learning tool
For learners, ACJ can help learning goals to be understood more clearly. By going through a process in which teachers can provide more clarity on what they expect of their students, and what good work looks like, learners can understand what direction they need to move in. Previous deployments of ACJ have shown it to be particularly beneficial for lower achievers, helping to bridge the gap between those at the top and bottom of the class.
When used in a peer-to-peer assessment context, ACJ can also offer clarity of learning goals, allowing students to identify the key components which make a successful piece of work. In a study by Seery et al (2011), conducted by the Irish Technology Education Research Group (TERG) and the University of Limerick, technology education students were asked to participate in a comparative judgement peer review process before final assessment. Afterwards, the students reported that the intervention improved their understanding of what successful work looks like and was far more effective than referring to rubrics.
The human brain is more capable of assessing and forming valid judgements when it is comparing two stimuli. Within education, ACJ can play a crucial role in assessment for learning, helping teachers clarify, validate and moderate their own interpretations of ‘good’ and shed light on areas they may need to focus their pupils on. For learners, ACJ can help to demystify learning, identifying and paving the way to success.