Artificial intelligence is becoming deeply embedded in how we teach, learn, and assess. But as LLMs and automated marking tools step into spaces once reserved for humans, a fundamental question emerges: can AI truly understand what we mean by quality and value?
Can LLM's understand concepts of 'value' and 'quality' in the same way as humans do? If not what does this mean for AI assessment? We completed a short study to explore more, and to think about some of the implications.
In today’s rapidly changing educational landscape, the key challenge isn’t just whether AI can mark student work, but how to ensure every mark is reliably fair. With that mission in mind, our latest proof of concept was designed to demonstrate why RM Compare is uniquely positioned as the foundation for trustworthy, scalable automated assessment.
The foundations of modern assessment are shifting. Not long ago, RM Compare introduced “ranks to rulers”: a pioneering process that transformed human judgments—ranking student work by quality—into reliable, calibrated measuring scales that educators and students could trust. Now, new AI validation methods are taking these ideas further, making instant, trustworthy assessment a reality for everyone.
We have had a bit of a tidy up to prevent users from tripping and falling. This is the BIGGEST refresh we have ever undertaken on the RM Compare User Interface
There is a renewed focus on holistic assessment recognising the need for a broader approach that can better appreciate the whole learner. The possibilities are exciting - they are also complex.
It's not quite spring yet but change is in the air. Read more about some imminent changes to the RM Compare user experience as well as other news.
In RM Compare sessions, parameter values are numerical representations of the relative quality or difficulty of the items being judged.
In the context of comparative judgement, the term "misfit" refers to a judge or an item that consistently deviates from the expectations of the model being used. Misfit can occur in two main contexts: judge misfit and item misfit.
In 2023 RM Compare was used in every continent (ok - with the exception of Antarctica!). In doing so it helped to validate one of our key hypotheses.
2024 is going to be an even more exciting year for RM Compare.
How long will it take for Judges to make their decisions? How will this influence the Judge workload? It depends!
Did you know that you can use your AI assistants to help you get even more from RM Compare?