Product

Simplified Pairs Accelerator

Comparative Judgement without the adaptive algorithm.

The Simplified Pairs method can be used to calibrate the scores (that is, the marks) from two assessments on the same scale.

How is the Simplified Pairs method used in RM Compare?

Standard RM Compare sessions employ an adaptivity algorithm to intelligently surface pairs of items for judgement – this is called Adaptive Comparative Judgement (ACJ). This approach has a number of benefits including a dramatic improvement in efficiency. However, a Simplified Pairs session needs to remove the adaptivity and instead take control of both the pairing process and the judge allocation.

For Example

Let’s say we want to calibrate Test A (30 Items / Scripts) with Test B (30 Items / Scripts). In a standard ACJ session, all 60 scripts would be uploaded into RM Compare for judging. Pairs of items would be surfaced by the algorithm – in some cases, the pairs would consist of both items from the same Test, in others, there would be one item from each Test. This approach does not meet our calibration requirement. Instead, we want to force a comparison between two tests. We may also want to control which judges see which pairings and in which order they see them.

Group	Name	Domain	Expiration	Security	Purpose
necessary	csrftoken	compare.rm.com	365 days, 0:00:00	HTTP	Helps prevent CSRF attacks
necessary	_cf_bm	vimeo.com	1 day, 0:00:00	HTTP	Used to distinguish between humans and bots
preferences	wtm	compare.rm.com	365 days, 0:00:00	HTTP	Used to store users cookie preference choices
statistics	_ga	rm.com	365 days, 0:00:00	HTTP	Registers a unique ID used to generate statistical data on how visitor used the website
statistics	_ga_#	rm.com	365 days, 0:00:00	HTTP	Used by Google Analytics to collect data on user visits to the website
statistics	_hp2_#	rm.com	1 day, 0:00:00	HTTP	Collects data on the user's navigation and behaviour on the website
statistics	_hp2_id.#	rm.com	365 days, 0:00:00	HTTP	Collects data on the user's navigation and behaviour on the website
statistics	_hp2_ses_props.#	rm.com	1 day, 0:00:00	HTTP	Collects data on the user's navigation and behaviour on the website
statistics	vuid	vimeo.com	365 days, 0:00:00	HTTP	Collects data on the user's visits to the website
marketing	td	googletagmanager.com	0:00:00	HTTP	Used by Google Tag Manager to collect data on the user behaviour and interaction with the website
marketing	h	heapanalytics.com	0:00:00	HTTP	Collects data on the user behaviour and interaction with the website

Name	Domain	Purpose	Expiration	Security
csrftoken	compare.rm.com	Helps prevent CSRF attacks	365 days, 0:00:00	HTTP
_cf_bm	vimeo.com	Used to distinguish between humans and bots	1 day, 0:00:00	HTTP

Name	Domain	Purpose	Expiration	Security
_ga	rm.com	Registers a unique ID used to generate statistical data on how visitor used the website	365 days, 0:00:00	HTTP
_ga_#	rm.com	Used by Google Analytics to collect data on user visits to the website	365 days, 0:00:00	HTTP
_hp2_#	rm.com	Collects data on the user's navigation and behaviour on the website	1 day, 0:00:00	HTTP
_hp2_id.#	rm.com	Collects data on the user's navigation and behaviour on the website	365 days, 0:00:00	HTTP
_hp2_ses_props.#	rm.com	Collects data on the user's navigation and behaviour on the website	1 day, 0:00:00	HTTP
vuid	vimeo.com	Collects data on the user's visits to the website	365 days, 0:00:00	HTTP

Name	Domain	Purpose	Expiration	Security
td	googletagmanager.com	Used by Google Tag Manager to collect data on the user behaviour and interaction with the website	0:00:00	HTTP
h	heapanalytics.com	Collects data on the user behaviour and interaction with the website	0:00:00	HTTP

Simplified Pairs Accelerator

Comparative Judgement without the adaptive algorithm.

How is the Simplified Pairs method used in RM Compare?

Further reading

Related Articles

Cookies