RM Compare and AI

By Mark House

We get asked a lot of questions about RM Compare and AI - here are some of the common ones together with our responses

Does RM Compare use AI?

Yes, RM Compare uses artificial intelligence, specifically machine learning, at the core of its system. The adaptivity that makes RM Compare’s comparative judgement process powerful is delivered by a machine learning algorithm. This enables the platform to efficiently rank, assess, and standardise various types of work, such as documents, audio, and video, for educational and professional purposes

Will RM Compare replace human judges with AI judges?

RM Compare cannot fully replace human judges with AI judges. While RM Compare uses AI—specifically, machine learning—to facilitate adaptive comparative judgement and efficiently manage large-scale assessments, the core process still relies on human judgement to evaluate and rank work. The AI in RM Compare is designed to optimize the workflow and support decision-making, not to autonomously make evaluative or ethical judgements in place of humans.

Recent research in judicial contexts underscores several key limitations of AI as a replacement for human judges:

  • Lack of Human Reasoning and Empathy: AI can generate decisions that resemble human reasoning but does not engage in genuine reasoning or feel empathy. This limits its ability to handle cases where moral, emotional, or social context is crucial.
  • Objectivity vs. Human Values: Studies show that while AI can apply legal rules with consistency, human judges often weigh emotional and moral factors, especially in complex or sensitive cases. This human element is difficult for AI to replicate.
  • Perceptions of Fairness: People generally perceive human judges as fairer than AI judges, particularly because human judgement is informed by shared social and moral codes.
  • Hybrid Systems: Some research suggests that combining AI and human judgement—rather than replacing one with the other—can offer complementary strengths. AI provides objectivity and consistency, while humans contribute empathy and nuanced understanding, especially in cases involving mitigating circumstances.

In summary, while RM Compare leverages AI to enhance and scale the assessment process, it is not designed to, nor can it currently, replace human judges with AI judges. The platform’s effectiveness depends on the collaboration between AI-driven adaptivity and human evaluative expertise, reflecting broader trends and limitations observed in the use of AI for judicial decision-making

Can RM Compare assess AI generated content?

Yes, RM Compare can assess AI-generated content. The platform is designed to facilitate comparative judgement on a wide range of item types, including documents, images, video, audio, and web pages—regardless of whether the content is created by humans or AI. Educators and assessors can set up sessions that consist solely of AI-generated work, a mix of AI and human work, or any combination that suits their assessment goals1.

How might RM Compare handle AI-Generated Content

  • Learning by Evaluating (LbE): RM Compare supports the use of AI-generated content in "Learning by Evaluating" sessions, where students compare and judge the quality of work. This helps students understand what constitutes high-quality work, even when the examples are produced by AI tools.
  • Flexible Content Types: The system accommodates various content formats, allowing AI-generated text, images, and other media to be compared alongside or against human-created work.
  • Research and Innovation: RM Compare is actively exploring how the presence of AI-generated content impacts assessment, originality, and value. For example, recent research has examined the relationship between content originality and value in an "AI world," using RM Compare to judge both human and AI submissions.

What are the key considerations when thinking about AI generated content in RM Compare?

  • Human Judgement Remains Central: While RM Compare can assess AI-generated content, human expertise is still crucial for nuanced evaluation. Studies have shown that AI-generated work often lacks the depth, context, and nuance found in human-created content, reinforcing the importance of human judgement in the assessment process.
  • Encouraging Originality: The platform is being used to help educators and institutions think critically about what they value in student work, particularly in distinguishing and rewarding originality in an era where AI-generated content is increasingly prevalent.

How might RM Compare encourage innovation with AI generated content?

RM Compare not only supports the assessment of AI-generated content but also encourages innovative approaches to evaluating and understanding the value of both AI and human work. Human judgement remains at the heart of the process, ensuring that assessments are meaningful and contextually aware