- Research
Multi-School Art Assessment Project Report
E-ACT Art assessment project Phase 2
Some things are "harder" to assess than others - they just are. This project is investigating the potential to use Adaptive Comparative Judgement to assess and standardise Art artefacts in a Mult-Academy Trust across multiple constituent schools. The Case Study from the first phase of the project is available here.
For the second phase E-Act art teachers joined assessment expert Vicky Merrick and the RM Compare Team at the TTS Inspiration Centre in Nottingham for a day of discovery and research. The day culminated with the completion of a Multi-Cohort Session assessing digital images of clay tiles.
Multi-Cohort Session
The pool of 17 judges worked together to assess 23 Items (Clay tiles). To simulate the MAT experience the Judges were allocated to one of 5 schools.
In total there were 18 rounds of Judging with each item being seen 14 times.
The Rank Order
The Judging team produced a rank order with extremely high levels of reliability. There were concerns that the digitisation process of the 3D artefacts would reduce reliability but this was not evident in the results.
The Judge Misfit
Two Judges (Circled red) showed signs of 'mis-fitting'. Both judges were non-domain experts (i.e. not Art Teachers). In this regard the mis-fit could be seen as encouraging and expected. The overall reliability of the session was not impacted by the mis-fitting judges.
Time to Judgement
The Median time to judgement was around 12 seconds. We can see from the bar chart that most Judgements were able to be completed quickly. However, each Item was seen for a total of over 3 minutes on account of it being seen multiple times.
Rank to Ruler
It is possible to convert the RM rank orders into 'linear' rulers. The potential benefits of rulers were covered in a recent report into ongoing research.
The interactive demo below provides a demonstration of how this might look. The data used here came from the session data extract. The report allows us to compare the relative performance of students by school. Visit the Advanced Reporting area for other examples.
Next steps
The intention is to extend this project in the Autumn by way of a much larger field based session which will involve all E-Act schools, in addition to looking at other curriculum areas.
Get involved
We were pleased to invite representatives from a number of other Multi-Academy Trusts to take part in the day. If you are interested in taking part in our next event (Autumn 2024) please get in touch.