- Product
Workload reduction and Adaptive Comparative Judgement
Assessing student work is expensive. For teachers it's generally an opportunity cost - there are always plenty of other things to do. For examination boards the cost could be seen a purely financial - hiring markers for example. In both cases the desire to get 'more for less' is paramount when considering a new approach like RM Compare.
Recently there has been a bit of a buzz in education circles following the release of GTP-3 which included some impressive auto-marking abilities. Time will tell on its lasting impact and implications, however the clamour for time-saving solutions from assessors has been plain to see.
Reducing workload is a major focus for RM Compare. We have written a lot recently about the potential of RM Compare On-Demand to reduce the number of judgements required. Right now a typical RM Compare session requires around 12 judgements pers script. On-Demand will reduce this by around 50%.
The challenge of complex items
Item complexity can differ substantially in any assessment. Long form written submissions for example have the potential to be more complex than shorter ones. The same is true for multi-page portfolio's and coursework.
As things get more complex so the cognitive load (the amount of working memory) placed on an assessor increases. The implication of item complexity on the comparative process, including workload, has been investigated many times. In 2022 for example Qualification Wales focused on its impact in GCSE History, a subject that elicits relatively long written submissions for assessment.
Reducing workload by chaining items
RM Compare's unique chaining functionality helps when dealing with more complex items. In this example we can see how chained items appear sequentially in subsequent judgements. For the judge this means that having achieved familiarity with an item they can then use this knowledge again on the very next judgement. This simple process reduces the cognitive load by up to 50% as only one item in a pair has the potential to be unfamiliar.
Adaptivity, chaining and cognitive load
The chaining process is especially important as the adaptivity in the system takes effect. The intelligent pairing of items optimises the decision making process by avoiding the random comparison of items that are very different in quality. In this regard then adaptivity increases cognitive load and at the same time chaining reduces it.
Working together adaptivity and chaining optimises the assessment process, reducing workload even for complex items.