Why EdTech's obsession with daily engagement is breaking learning

Most digital products in education, awarding, and professional development are hooked on the exact same commercial metric: Attention. Daily Active Users (DAU), session length, streaks, push notifications, and time-in-app sit at the absolute center of traditional EdTech strategy.

In many consumer business models it can be argued that it not only makes total sense, it's the 'only' way to may money. In education and high-stakes assessment, however, this obsession with engagement depth hides a quiet, troubling trade-off: cognitive offloading. When a digital tool is designed to keep a user "on rails," constantly dictating the next step or generating automated answers, the software gets smarter while the human user becomes permanently dependent on a digital crutch. They perform, but they don't deeply learn or develop true assessment literacy.

What if that entire metric is fundamentally broken? What if the true measure of a product's success is that individual usage falls over time?

The Research Imperative: Quality Over Quantity

This tension between screen time and genuine capability is precisely what the International Baccalaureate (IB) examined in their research summary,Understanding today's learners: Cognitive impacts of technology use on adolescent learning (Arztmann & Gallagher, 2026).

Reframing the debate away from simplistic "more tech vs. less tech" arguments, the IB's synthesis of cognitive research delivers a clear verdict: the quality of technology integration explains far more variance in learning outcomes than frequency of use alone.

Drawing on extensive international studies and OECD datasets, the report highlights that unstructured, passive, or heavy screen time taxes working memory and executive function, leading to task fatigue and attention fragmentation. Lower and average performers are particularly vulnerable to disengagement when digital environments are cluttered or open-ended.

Digital tools yield protective, highly positive outcomes only when they demand active, structured, high-cognition interaction within a low-friction setting.

Core Takeaways from the IB Research

• Active Evaluative Cognition: Quality of engagement, requiring users to actively process, compare, and reason, drives learning outcomes and marker calibration rather than passive content delivery.
• Protecting Working Memory: Streamlined user experiences minimise UI-induced cognitive load, preserving executive function for deep processing.
• Preventing Disengagement & Fatigue: Digital interactions must be bounded, structured, and purpose-driven micro-tasks rather than open-ended screen sessions.
• Closing Attainment & Calibration Gaps: Adaptive comparative feedback supports assessors and learners across distributed networks provided it augments rather than replaces human judgment.

From Theory to System Practice: Evidence from Swedish Municipalities

The principles outlined in the IB research aren't just theoretical ideals, they are being actively proven across entire educational jurisdictions in Europe. Our newly published Swedish Case Study and Blog Overview showcase how Adaptive Comparative Judgement delivers concrete calibration at scale.

At the upcoming researchED National Conference 2026 in London (Saturday 5th September), Dr. Eva Hartell will be presenting full findings from her research in Sweden. Her work demonstrates how comparative judgment transforms assessment from a burdensome administrative task into an active, formative calibration event across an entire umbrella network.

By engaging in rapid, pairwise comparative decisions, Swedish educators demonstrated that:

• Evaluator Alignment & Consistency Improved: Judge-misfit values clustered tightly, showing assessors made significantly more consistent decisions after a short comparative session.
• Internalised Standards ("Guild Knowledge"): Assessors made faster, steadier decisions with less cognitive friction, moving away from checklist marking toward holistic evaluation of reasoning and conceptual depth.
• Stable Scales with Reduced Volume: Once a panel calibrates to a shared standard, RM Compare constructs reliable assessment scales with significantly reduced judging volume.

The "Tuning Fork" Model & Assessment AS Learning

The IB's research directly validates the core design philosophy behind RM Compare. We believe that true personalisation isn't about an algorithm profiling a user from afar, rather it is about giving school networks and awarding bodies an agile, on-demand utility to calibrate judgement on their own terms.

We don't want your attention every day. We are designing for the Tuning Fork Model: a rapid, on-demand way to estimate quality, compare against a trusted standard, calibrate judgment, and then get back to the real work with a sharper eye.

When an assessor, examiner, or student opens RM Compare, they step off the automated conveyor belt and engage in Assessment as Learning:

1. Estimate & Compare: The user views or captures work and makes rapid, side-by-side comparative decisions ("Which item better demonstrates quality?").
2. Calibrate: The system reveals how accurately their initial quality estimates aligned with a trusted benchmark standard.
3. Internalise 'Guild Knowledge': As pioneered by education researcher D. Royce Sadler, experienced practitioners carry standards as tacit "Guild Knowledge" inside their heads. By actively judging pairs, learners, teachers, and markers rapidly internalize what quality looks like.
4. Return to Work: The user closes the device and applies that newly sharpened judgment directly to their own creating, marking, or moderating, without needing the software as a permanent crutch.

As capability transfers into the human mind, individual usage naturally changes. A novice assessor might run 15 comparative judgements to build their initial "eye" for quality, while a veteran moderator might run 5 in few minutes purely as a sanity check. Individual frequency drops because the human has become smarter.

Experience "Assessment as Learning": The RM Compare | NOW

As education systems and awarding bodies prepare for the upcoming academic year, we are moving RM Compare | NOW from Alpha into open BETA before the end of August.

Designed as a login-free, mobile-first micro-utility, Live | NOW allows educators, examiners, and students to experience comparative judgment in seconds, benchmarking work against trusted standards, testing their own Guild Knowledge, and seeing Assessment as Learning in action.

Ready to calibrate your own department, school group, or exam team? Watch this space.