- Opinion
Comparative Judgement and the digital assessment journey
In our global interactions with assessment providers, we've identified a typical digital assessment journey that comprises five stages: Process Enhancement, Digital Transition, Digital Integration, Assessment Transformation, and Learning Transformation. This framework is designed to help organisations comprehend the intricacies of the digital journey and to devise a strategy that aligns with their specific needs and aspirations.
The journey towards digital assessment can be complex and intimidating for organisations, especially when considering novel assessment methods like comparative judgement. As a global leader in assessment, RM has collaborated with organisations worldwide. Our extensive expertise in Comparative Judgement empowers organisations of all sizes to confidently explore this innovative assessment method.
This discussion explores some of the unique challenges organisations often encounter when considering comparative judgement. The narrative begins with a stool and concludes with lemonade, a progression that will make sense in the end.
The 3 legged stool
Assessment is often likened to one component of a three-pronged approach, with curriculum and pedagogy forming the other two elements. The concept of assessment washback, or the influence of assessment on teaching and learning, is a significant factor in this model. It's not uncommon to encounter curriculum and pedagogical strategies that haven't been developed with a comprehensive perspective. However, this doesn't have to be a hindrance. In fact, it's a situation we frequently encounter and assist organizations in navigating to determine their optimal path forward.
Apples and oranges
In an ideal scenario, assessment tasks should be crafted in a way that they are well-suited for evaluation through Comparative Judgement. This means that from the beginning, those creating the items should be cognizant of the comprehensive nature of the assessment. This understanding should also be shared by instructors and curriculum designers, who would then modify their strategies accordingly. This approach ensures that our assessment tool is accurately evaluating the intended subject matter. For instance, if we are assessing apples, we want to ensure our tool is not inadvertently measuring oranges.
This shift can pose a significant challenge for many who are accustomed to a different focus, such as those who have been dealing with oranges but now want to transition to apples. However, we are here to provide the necessary support and guidance in this transition.
Chicken and Egg
In the journey towards adopting a comparative judgement approach in assessment, it's understandable that users may feel hesitant to make significant changes that could impact curriculum and pedagogy without first having a solid understanding and confidence in the method. This situation can seem like a 'chicken and egg' problem - it's challenging to alter the curriculum and pedagogical strategies without evidence of the method's effectiveness, yet it's difficult to demonstrate its effectiveness without implementing the changes.However, there are numerous strategies to navigate this challenge. We have collaborated with various organizations in different contexts to successfully implement comparative judgement. It's important to remember that the comparative judgement approach is expected to reduce inconsistencies in assessment outcomes and can be undertaken by a group of assessors, not just a single one.
This method has been shown to be more accurate than traditional marking, and each student's work is judged holistically, avoiding a single decisive judgement. Moreover, the use of comparative judgement can lead to significant gains in learning, especially when students are activated as learning resources for one another
It's also worth noting that comparative judgement is not a new method of assessment. It was first proposed in the 1920s and has been used to quantify the quality of things that are difficult to measure, such as writing or drawing.
In conclusion, while the transition to a comparative judgement approach may seem daunting, it's a journey that we can navigate together. With our experience and your commitment, we can make this transition smooth and beneficial for all involved.
Rock and a hard place
RM Compare is an innovative product. Adopting it as an assessment solution can create an 'innovators dilemma' for some organisations which can feel a bit like being between a rock and hard place.
Our expertise goes beyond features and functionality to think about wider strategy and tactics of any proposed implementation. We can help.
Lemons and Lemonade
RM is a global assessment provider serving customers at all stages of their digital journey. We recognise that getting from where you are to where you might want to be will have challenges.
For those thinking about using a comparative approach as part of their service or solution we are here to help. Let's make lemonade together!
From a stool to Lemonade
I told you it would make sense (well, sort of!). The message hopefully is that we understand your challenges and blockers to bringing Comparative Judgement into your assessment service or solution. We have been there, seen it and done it many times. We are here to help.