Why 'Guild Knowledge' is your organisation's most valuable, and most overlooked asset

In the previous posts, we explored what Guild Knowledge is (the tacit, experience‑based ability to recognise quality) and showed what the research says about how it can be developed through Learning by Evaluating. This post makes the organisational case: what happens when Guild Knowledge is absent, why conventional responses make it worse, and how to build it deliberately and at scale.

The problem no‑one names

Every organisation has people whose judgement it depends on. The experienced assessor who knows immediately what a "distinction" looks like. The senior manager who can read a project proposal and spot its weaknesses in minutes. The compliance officer who recognises a policy breach from a brief description. The recruiter who can review a portfolio and tell within seconds whether someone has the skill being assessed.

These people are not just more experienced. They have internalised a standard. They carry inside their heads a calibrated, working model of quality that allows them to make fast, reliable judgements in complex, ambiguous situations.

When they leave, that standard leaves with them. When they retire, are promoted, or move on, the organisation does not just lose a body: it loses accumulated Guild Knowledge that may have taken decades to build. And this loss is typically invisible until it surfaces as inconsistency, error, complaint, or at its worst a high‑profile failure of judgement.

The problem is not new. But three forces are making it more acute.

Why the problem is getting worse

1. Knowledge loss is accelerating

Research by Gartner found that 58% of the workforce will need new skill sets to perform their current roles in the coming years, driven by technological change. Meanwhile, studies consistently show that up to 70% of critical organisational knowledge is tacit by way of being embedded in people's heads rather than written down in any system, process or manual.

When experienced practitioners leave, most organisations have no systematic way to transfer that tacit knowledge. APQC research has found that organisations lose an average of $47 million in productivity per year to inadequate knowledge transfer. In high‑dependency roles such as assessment, clinical supervision, quality assurance, and compliance, the costs are often higher, because the judgements being made carry real‑world consequences.

2. VUCA conditions have raised the stakes for human judgement

The environments most organisations operate in today are characterised by volatility, uncertainty, complexity and ambiguity (VUCA). Standardised processes and documented procedures are useful in stable, predictable environments, but they break down when conditions are ambiguous or novel.

In VUCA conditions, the only thing that adapts quickly enough is calibrated human judgement. Rules cannot cover every case. AI cannot reliably navigate genuine ambiguity without well‑calibrated human oversight. The practitioners who cope well in complex, high‑stakes situations are not the ones with the best procedures: they are the ones with the best‑developed Guild Knowledge.

3. AI is making tacit judgement more critical, not less

There is an understandable temptation to assume that AI will eventually replace the need for human expert judgement i.e that if systems get good enough, organisations will no longer need to invest in developing the intuitive, holistic sense of quality that experts carry.

The evidence points in the opposite direction. Large language models and automated scoring systems can approximate expert judgement on well‑defined, stable tasks, but they are brittle in novel situations, require calibrated human oversight to catch systematic errors, and cannot substitute for the evaluative reasoning that comes from deep domain experience.

More fundamentally: the people most valuable in an AI‑assisted world are those who can tell when the AI is wrong. That requires high‑quality Guild Knowledge. An organisation that has allowed its Guild Knowledge to atrophy is not protected by AI, it is exposed by it.

Why conventional responses don't fix this

When organisations recognise that judgement quality is inconsistent, the typical responses are:

• Write better rubrics and criteria documents - make the standards more explicit, more granular, more detailed.
• Add more governance - introduce panels, approval layers, inter‑rater reliability checks, audit trails.
• Mandate more training - run workshops on the criteria, show exemplars in a presentation, add a module to the induction programme.

None of these are bad ideas. But on their own, they address the symptom, not the cause.

Sadler identified this clearly in 1989: the problem is not that novices have failed to read the rubric carefully enough. The problem is that rubrics, however detailed, cannot fully articulate tacit standards. Standards are multi‑dimensional, context‑dependent, and ultimately learnt through exposure to real examples and guided practice in judgement, not by reading about them. Adding more words to a document does not produce Guild Knowledge.

Governance and approval layers go further. They can actually disguise the absence of Guild Knowledge by routing decisions to the few people who have it, without ever developing it in the many. And one‑off training sessions with exemplar slides have a half‑life: without repeated calibration, judgement drift sets in within weeks.

What builds Guild Knowledge is repeated, structured, feedback‑rich practice in making judgements. This is exactly what Learning by Evaluating research has consistently shown to work.

The cost of operating without it

Unreliable judgement is not just a quality issue. It carries direct, measurable financial consequences.

• Cost of Poor Quality (COPQ) in knowledge‑intensive organisations typically runs at 15–25% of operating costs, largely driven by the hidden costs of rework, inconsistent decision‑making and downstream error.
• In assessment and awarding, grade appeal rates and moderation failures carry significant direct and reputational costs. The UK's experience of algorithmic grade controversies in 2020 was, in part, a story of what happens when tacit standards that underpin centre‑assessed grades cannot be defended under scrutiny.
• In recruitment, poor hiring decisions cost an average of 1–3x the annual salary of the role in question, and much of this is attributable to uncalibrated judgement at the portfolio review or interview stage.
• In regulated professions, inconsistent competency judgements create compliance risk, reputational exposure, and - in clinical and safety‑critical contexts - real harm.

These costs are real, but they are often invisible because they are distributed across the organisation: in rework costs, in appeals handling, in management time spent re‑adjudicating decisions, in the quiet reputational erosion that follows when quality varies by who happens to be doing the assessment.

What RM Compare makes possible

RM Compare is built around the proposition that Guild Knowledge can be developed deliberately, efficiently and at scale. It is not just left to accumulate slowly through years of experience.

The evidence from organisations that have made the shift is concrete.

At NCFE, RM Compare was used for the moderation of centre‑assessed grades at Level 1 and Level 2. The results were a 58% reduction in time spent on moderation compared to the previous process, alongside a 43% reduction in the number of decisions required. Assessors reported that the process was clearer and more consistent, and the organisation had an auditable, defensible record of every judgement.

At Purdue University, a single 20‑minute Learning by Evaluating session before an assignment produced a significant improvement in student performance across a cohort of 550 students, with seven of the top ten performers coming from the LbE group. The cost: 20 minutes and no additional teaching resource.

At Cain College, an ipsative LbE programme spanning four assignments produced a 40.75% improvement in scores for the lowest‑achieving quartile of students, alongside high ACJ reliability across all sessions. Again, without changing the curriculum, the teaching team, or the assessment criteria - just by giving students structured practice in making calibrated judgements.

These are not outliers. They represent what happens when an organisation stops assuming Guild Knowledge will develop on its own and starts building it deliberately.

Three ways to start

1. Let individuals test their own judgement - right now

⏱️RM Compare | NOW gives any practitioner a short, login‑free session where they can capture or upload an item, make a few paired comparisons against a trusted standard, and immediately see how accurate their judgement is.

This is not a training module or a compliance exercise. It is a direct, personal experience of Guild Knowledge development: estimate, compare, reveal, adjust. Used as part of regular practice - before moderation panels, before assessment windows, during induction - it begins to close the gap between novice and expert judgement faster than any rubric workshop can.

2. Use Live sessions to build shared standards across a team

📳RM Compare | Live enables real‑time ACJ sessions where teams evaluate work together, building a shared, statistically grounded understanding of quality. Unlike a presentation of exemplars, a Live session produces an explicit, ranked consensus that the whole group has contributed to. That consensus becomes a form of Guild Knowledge, held not just by one expert but by everyone who took part.

3. Build, govern and scale standards with Studio and Hub

💻RM Compare | Studio creates durable, statistically validated quality "rulers" from ACJ sessions, standards that represent real expert consensus, not averaged rubric scores. 🧭RM Compare | Hub makes those standards shareable, governable and connectable across programmes, centres and organisations.

Together, 💻Studio and 🧭Hub turn what was previously locked in experts' heads into an organisational asset. A standard that can be used consistently by assessors, made available in Live sessions, and experienced by individuals through ⏱️NOW.

Guild Knowledge is a strategic choice

For most organisations, the current state is not a deliberate policy. Nobody decided that Guild Knowledge would be left to accumulate by accident, or that judgement calibration would be treated as optional. It is simply that no practical tool existed to do anything different.

That is no longer true. The research base is robust. The case studies show measurable impact. And the tools are available today, five‑minute session in ⏱️RM Compare | NOW to a full organisational standard‑setting and governance infrastructure in 💻Studio and 🧭Hub.

The question is no longer whether Guild Knowledge can be built deliberately. It is whether your organisation chooses to build it.

In the next post in this series, we will bring the ecosystem together showing how 🧭NOW, 📳Live, 💻Studio and 🧭Hub map to different assessment needs and how organisations move from a first encounter with ACJ all the way to a fully governed, shared standards infrastructure.

Ready to start developing Guild Knowledge today?