Ask anything
RM Compare has been optimised for use with your LLM.
See our Training Centre for more information.
- Reporting
- Product
Premium Reporting
Premium Reporting allows you to dig even deeper into your data with a range of interactive analytic tools.
Parameter evolution
An items parameter value will change during the comparative process. In the example below we can see how the adaptive algorithm generated stability in the parameter value over 16 rounds of judging.
Parameter value frequency
We can see how at either end of the range of items we have a small number of items. The highest ranked item can be seen on the right and side, and the lowest on the left. Most items are bunched towards to the centre of the distribution.
Reliability vs round
This view can help us to understand how the overall reliability of the session developed over time. We can see that the algorithm began to take effect from round 4 and was quickly able to impact on the session reliability. High levels of reliability stabalised as session progressed.
Decision time distributions
In this chart we can see how long our judges were spending on their decisions. Time judgement tails off as time elapses with the bulk of judgements taking place in 60 seconds or less.
Standardised residuals
A Standardised residual is a measure of the strength of the difference between observed and expected values. In simple terms the RM Compare algorithm describes how likely one Item is to be chosen over another (expected value). We can then calculate the SR against the actual judgement made (observed value).
Decision Time vs Standardised Residuals
We are able to get an understanding of the relationship between Decision Time and and the Standardised Residuals. This chart (below) indicates that the SR does not appear to be influenced by Decision Time.
Judge Participation
We may want to compare how individual judges interacted with the session. In the chart below we can see how the first 200 judgements we completed by 6 judges, each one completing their set after the previous one had finished.
Test your knowledge
Five quick questions to test your understanding