From Item Repositories to Standards Repositories: Making Shared Quality Reusable (Part 1/3)

Awarding organisations create a remarkable amount of evidence about quality.

Every assessment series generates candidate work, marks, annotations, examiner judgements, standardisation materials, moderation outcomes and awarding decisions. Each of these tells part of the story of what good performance looks like. Together, they represent years of subject expertise, professional judgement and operational effort.

Yet much of that value remains surprisingly hard to reuse.

A senior examiner may know instinctively why one response is stronger than another. A standardisation meeting may produce a rich, nuanced shared understanding of how a mark scheme should be applied. A moderation exercise may uncover the exact characteristics that separate a secure performance from an exceptional one.

But when the assessment window closes, that shared understanding is often dispersed. It may live in PDFs, slide decks, spreadsheet tabs, marking platforms, shared drives and, most importantly, the accumulated experience of the people involved.

The evidence still exists. The shared standard is much harder to find.

The standards we keep recreating

This creates a familiar cycle across assessment.

An awarding organisation creates an assessment and gathers evidence. Examiners mark, standardise and moderate. Through that work, the organisation develops a shared understanding of quality. Materials are distributed for the series, the assessment is completed and the evidence is archived.

Then the next series begins.

New examiners need to be brought into line. Existing examiners need to reconnect with the standard. Materials are updated and redistributed. Examples are found, reformatted and discussed again. Much of the work is necessary—but much of it is also repeated because the organisation has no easy way to preserve and actively deploy what it has already learned.

What if that shared understanding did not have to be rebuilt from the ground up each time?

What if the work, the judgement and the context behind it could become a durable organisational asset?

That is the opportunity behind a standards repository.

A library is not yet a standard

The idea of collecting examples of assessed work is not new. Across education, there is increasing interest in repositories that allow teachers, assessors and organisations to see real examples of performance rather than relying solely on abstract descriptors.

Northern Ireland’s Independent Review of Statutory Assessment, for example, recommended a Writing Repository: a lower-workload approach intended to gather examples of pupil writing from schools, enable anonymous comparison and help teachers understand quality across the system.

This is an important development. Seeing authentic work can make an abstract standard tangible. A piece of writing, a portfolio, a design project, a recording or a practical performance often communicates more about quality than a page of descriptors ever could.

For awarding organisations, an item repository can bring together candidate work with the information that makes it meaningful: the relevant task, specification, mark scheme, original mark, annotation, assessment series and approval status.

That is a valuable foundation. It makes evidence easier to find, inspect and share.

But a library of examples is not automatically a shared standard.

A response might have been awarded 28 marks. It might have been approved as a useful exemplar. It might even have been discussed in a senior examiner meeting. But unless someone can see the context, provenance and evidence behind its status, it is still just a file with a label.

A standards repository goes further. It preserves not only the work, but the explanation of why the work matters.

From examples to evidence of quality

The difference is subtle, but important.

An item repository can tell you that a candidate produced a particular response in a particular assessment series. A standards repository can tell you what that response represents.

It can retain the mark scheme and original mark, but also the examiner annotation, the relevant performance criteria, the task conditions, the specification version and the approval history. It can show whether the work was selected as a definitive script, a standardisation example, a moderated sample or a useful illustration of a particular level of performance.

This matters because standards are not inherent in an item. They are established through evidence and professional agreement.

A strong candidate response is not automatically a strong reference point. It becomes one when an organisation can explain its relationship to the construct being assessed, the assessment task, the criteria, the approved outcome and the purpose for which it may be reused.

That is especially important for awarding organisations. A standard is rarely universal. A response may be an excellent reference point for one qualification, component, task type and specification version, while being inappropriate for another. A piece of work may remain valuable as historical evidence but no longer be suitable for current examiner training. A set of examples may be useful for discussion but not robust enough to support live decisions.

A standards repository makes these boundaries clear.

It records who owns the standard, who can access it, what it is approved for and when it should next be reviewed. It makes the standard inspectable rather than implicit.

Where comparative judgement fits

Adaptive Comparative Judgement has an important role in this picture, but it is not the only source of standards evidence.

Awarding organisations already have authoritative evidence created through rubric-based marking, mark schemes, examiner standardisation, moderation and awarding decisions. That evidence should not be discarded or treated as inferior. It provides the context and authority that make assessment outcomes defensible.

Comparative judgement can add something different.

For suitable forms of complex work—where quality is integrated, holistic and difficult to capture through a long sequence of isolated marking decisions—ACJ can create a stable relative rank from repeated expert comparisons. It can help make the relative positions of reference items visible and reusable.

This is particularly valuable when an organisation wants to move from a set of approved examples to a ruler: a reference scale against which new work can be compared.

But the distinction matters.

A collection of scripts ordered by their original marks may be an authoritative and useful reference set. It does not automatically become an ACJ ruler simply because it has been placed in mark order. The original marks reflect the construct, weighting and rules of the mark scheme. A comparative ruler reflects the agreed construct used by judges in the comparison process.

Where those constructs align, comparative judgement can strengthen the evidence around a set of reference items. Where they do not, the organisation should preserve the value of the original marked evidence rather than force an artificial equivalence.

The role of a standards repository is to hold these forms of evidence together without confusing them.

It can preserve the authority of a marked exemplar, while also recording comparative-calibration evidence when that evidence has been created. It can make clear whether an item is part of an approved reference set, an ACJ-calibrated ruler, or both.

A living infrastructure for standards

This is the shift from storing assessment artefacts to maintaining assessment infrastructure.

A repository of items is useful because it makes examples available. A standards repository is more powerful because it turns those examples into active organisational assets.

It enables an awarding organisation to retain and reuse the knowledge created through marking and moderation. It helps new examiners see authentic work in context. It allows senior examiners to curate and update reference sets rather than reassemble materials from scratch. It creates a clearer audit trail of how standards were established, reviewed and deployed.

And, where appropriate, it allows selected collections of work to be calibrated through comparative judgement and converted into reusable rulers.

Those rulers can then support standardisation, moderation, quality assurance and controlled comparison of new work. They can give assessors a shared point of reference, not as a replacement for professional judgement, but as a stronger foundation for it.

The goal is not to freeze standards in time. Standards need maintenance. Specifications change, tasks evolve, candidate responses shift and new evidence emerges.

A living standards repository recognises this. It allows organisations to review, version, retire and replace reference material deliberately. It turns change from an excuse to start again into a managed process of maintaining a shared standard.

From one-off effort to reusable value

For awarding organisations, this is ultimately an operating-model question.

Do standards live mainly in individual people, individual assessment series and individual documents? Or do they become governed organisational assets that can be inspected, improved and deployed whenever consistent judgement is needed?

RM Assessment already helps organisations create, deliver and manage assessment. RM Compare can add a complementary standards and calibration layer: a place where assessment evidence can be retained in context, governed for appropriate use and, where suitable, strengthened through comparative judgement.

The result is not a bigger archive.

It is a more durable way of working.

Instead of repeatedly reconstructing shared understanding of quality, organisations can maintain it. Instead of treating candidate work and expert judgement as temporary outputs of a series, they can turn them into reusable evidence. Instead of relying on static documents alone, they can place trusted standards into the workflows where assessors, examiners and moderators need them.

That is the journey from item repositories to standards repositories.

The future of assessment is not simply storing more examples of work. It is making evidence of quality visible, governed and reusable—so shared standards can endure beyond the assessment event that created them.

In Part 2, we will explore the practical question that follows: how can awarding organisations use existing rubric-marked exemplars as the foundation for reusable standards, and when can those standards be strengthened through Adaptive Comparative Judgement?