- Product
The comparison process
A step by step guide to complete your judgements
RM Compare allows judges to compare two pieces of work side-by-side and decide which is better. This is done multiple times by multiple assessors (we call them Judges) to ensure a high level of confidence.
When you are invited to be a judge in an RM Compare session you will receive an email invitation and will be able to join the session right away. You will also receive a second email notification when the judging session begins.
To get started with your judgements log into your account and find your sessions listed in your judge dashboard. If the Start Judging button is showing grey it means the session has not started yet. When the Admin starts the session the button will turn blue and you're good to go.
If you have been invited to the Session as a Contributing Judge you will be able to upload you own Item.
Making the judgements
You will be shown two pieces of work (items) and all you have to do is read the holistic statement at the top left of your screen and decide which item is better. Once you’ve taken a look at each item and decided, simply click the A or B button to make your judgement.
Personalised and accessible viewing controls
It is important that Judges can consider Items in a way that meets their personal preference and needs. RM Compare works well on all major browsers and devices. It also offers a number of user controls in the interface.
You can read more about the available viewing controls here.
Once you’ve finished your judgements you’ll be returned to your judging dashboard. You’ll notice that My % column is now at 100%. This means you’ve completed all of your judgements. Global % shows the total amount of judgements completed by all judges in the session.
How many judgements do I need to complete?
RM Compare works by pairing items in different combinations so you’ll need to complete multiple pairs in a session. RM Compare automatically calculates the number of judgements all judges have to make and allocates the same number to every judge.
You can track how many judgements you have left from your total allocation. This is shown in the top of the page in brackets. For example (3/12) means that you have completed 3 judgements out of your allocation of 12 judgements in total.
Do I have to make all my judgements in one go?
RM Compare does NOT require you to complete all of your judgements at once unless you want to. If you do decide to take a break then simply log out. When you log back in RM Compare will take you back to the last judgement you were about to make.
Can I see the items one-by-one?
You can choose to see items one-by-one or side-by-side (default view) by switching the view options at the top right corner (e.g., A&B, A or B). The way you view items makes no difference to the judgements that you make - you must still select the winning item by clicking on the A or B circles.
How can I leave feedback or comments?
Session admins can enable judges to leave feedback on individual pieces of work, making them optional or compulsory.
If it has been enabled by your admin, you can use the speech bubble button on the top of the item to add your feedback.
Admins can also ask Judges to provide feedback about their decisions and the Items they are seeing.
Will I see all of the Items? Will I see the Item I uploaded?
Not necessarily. It might be that there are many more items than the judgements you have been asked to complete. It might also be the case that the adaptive algorithm presents you with a limited number. In this case you may see the same item more than once.
You may see an Item that you uploaded. This should not have an adverse effect on the rank because each item is still seen numerous times by other judges. There are also checks and balances, including misfit reports, to identify any judging that is not in concensus with the general sentiment of the judging pool.
Test your knowledge
Five quick questions to test your understanding