Making the judgements
RM Compare allows judges to compare two pieces of work side-by-side and decide which is better. This is done multiple times by multiple assessors (we call them Judges) to ensure a high level of confidence.
As an Admin
Once you’ve finished setting up your session, you’ve uploaded your items (or allowed the judges to upload their own) and you’re ready to let your judges begin with the comparisons, you can find the Run session button on the top right corner of the session dashboard.
RM Compare will email your judges to let them know the session has begun.
As a Judge
When you are invited to be a judge in an RM Compare session you will receive an email invitation and will be able to join the session right away. You will also receive a second email notification when the judging session begins.
To get started with your judgements log into your account and find your sessions listed in your judge dashboard. If the Start Judging button is showing grey it means the session has not started yet. When the Admin starts the session the button will turn blue and you're good to go.
Learn how to submit your own items to be assessed during the session.
Making the judgements
You will be shown two pieces of work (items) and all you have to do is read the holistic statement at the top left of your screen and decide which item is better. Once you’ve taken a look at each item and decided, simply click the A or B button to make your judgement.
Once you’ve finished your judgements you’ll be returned to your judging dashboard. You’ll notice that My % column is now at 100%. This means you’ve completed all of your judgements. Global % shows the total amount of judgements completed by all judges in the session.
How many judgements do I need to complete?
RM Compare works by pairing items in different combinations so you’ll need to complete multiple pairs in a session. RM Compare automatically calculates the number of judgements all judges have to make and allocates the same number to every judge.
You can track how many judgements you have left from your total allocation. This is shown in the top of the page in brackets. For example (3/12) means that you have completed 3 judgements out of your allocation of 12 judgements in total.
Do I have to make all my judgements in one go?
RM Compare does NOT require you to complete all of your judgements at once unless you want to. If you do decide to take a break then simply log out. When you log back in RM Compare will take you back to the last judgement you were about to make.
Can I see the items one-by-one?
You can choose to see items one-by-one or side-by-side (default view) by switching the view options at the top right corner (e.g., A&B, A or B). The way you view items makes no difference to the judgements that you make - you must still select the winning item by clicking on the A or B circles.
How can I leave feedback or comments?
Session admins can enable judges to leave feedback on individual pieces of work, making them optional or compulsory.
If it has been enabled by your admin, you can use the speech bubble button on the top of the item to add your feedback.
Admins can also ask judges to justify their decisions after making a choice. If that’s the case you’ll see a box pop-up after the judgement where you can leave your comment.
How can I see the final results?
If your admin session has enabled judges to see the results you’ll be able to access them from the session dashboard.
RM Compare will create one report for each completed round of judgements. Generally speaking later rounds have higher confidence levels because more judgements have taken place. See How to interpret your reports
If the session admin has enabled feedback and comments, you will be able to see what other Judges have said about your work.
Can't find what you’re looking for?
Get in touch, we are here to help.