A Framework for Certification Content
Exploring whether AI can help write exam questions without losing what makes them fair.
A proof-of-concept for a multi-agent framework that supports drafting and review of certification exam items, while preserving the psychometric rigour and technical accuracy that make an exam meaningful. The work is deliberately careful.
The problem
Language models produce fluent draft content quickly. Certification-quality items need more than fluency — item-writing discipline, statistical soundness, and validation against real behaviour. The question is where AI helps and where humans must stay in the loop.
The approach
Modelled the item lifecycle as a series of small, specialised jobs — generation, review, quality checks — each handled by an appropriate agent, all under expert human oversight. Built structured evaluations rather than trusting first impressions.
The outcome
A working POC and, more importantly, a clearer view of what a responsible AI-assisted content pipeline looks like in a domain where correctness is non-negotiable.