Results

Q21: A contested item that highlights benchmark limitations. "Environments that are rich in stimulus improve the brains of pre-school children" is classified as a neuromyth in the Dekker et al. (2012) instrument. The neuroscience behind this classification is that the original enrichment findings came from rats raised in deprived laboratory conditions (OECD, 2002, Understanding the Brain: Towards a New Learning Science), and the broad generalisation to human children is not well-supported by the evidence. However, this classification is debatable. The ambiguity in both "enrichment" and the assumed baseline makes a well-informed "True" response reasonably defensible.

Every model tested answered this item incorrectly (endorsing it as true) with high confidence. We retain this item because it is part of the established instrument and because it illustrates an important point: some items in neuromyth instruments have real ambiguity that benchmark designers should acknowledge.

Q15 vs Q27: The learning styles trap. Q15 ("Individuals learn better when they receive information in their preferred learning style") is false. It is the most widely-believed myth in education. But Q27 ("Individual learners show preferences for the mode in which they receive information") is true. People do have preferences; those preferences just do not improve learning outcomes. Many models confuse these two statements, getting Q15 right but Q27 wrong, or vice versa.

Near-universal overconfidence. Eight high-prevalence myths receive a follow-up confidence probe after the model answers. Across all confidence probes administered (8 items per model), no model selected "Uncertain" on any item, even when answering incorrectly. Every response was "Very Confident" or "Somewhat Confident." The sample is small (8 items), but the pattern is consistent across all models tested.

Example: S01 (Retrieval Practice). A Year 8 science teacher quizzes students on new material before teaching it and provides no feedback after revealing answers. The teacher concludes "retrieval practice doesn't work for science." The correct diagnosis: testing before encoding violates the retrieval-must-follow-encoding sequence, and skipping feedback eliminates the error correction mechanism. Frontier models identify both issues. Smaller models tend to give generic advice ("try different strategies") without pinpointing the specific implementation errors.

Why such low agreement? Raw accuracy of 50% sounds like a coin flip, but Cohen's kappa tells the more precise story: at κ = 0.252 average, models are only marginally better than what chance would produce given the class distribution. The ACARA achievement standards used as grading criteria may lack sufficient specificity for absolute classification. They describe what students should know and do at each level, but may not provide enough detail about what distinguishes Above Satisfactory from Satisfactory work. Two hypotheses warrant further investigation. First, the ACARA achievement standards may lack sufficient specificity for absolute classification, and models might perform better with richer criteria that explicitly describe the characteristics of each grade level. Second, the task may require a kind of grounded assessment expertise that current models lack. A structured reasoning pilot (requiring models to reason step-by-step before classifying) made performance worse, not better. Models were prompted to analyse the evidence in student work against the standard before committing to a classification. Rather than improving calibration, this appeared to make models more critical and deficit-focused, pushing more responses toward Below Satisfactory rather than producing more nuanced distinctions across all three categories.

These results should be interpreted alongside the documented limitations. Scores may be influenced by training data overlap, LLM judge bias, and cultural specificity of items.

Training data overlap is a particular concern for benchmarks that draw on publicly available items. The neuromyth statements from Dekker et al. (2012) and the teacher certification items from the Pedagogy Benchmark are both accessible online, meaning some or all items may appear in model training data. High accuracy on these benchmarks may partly reflect memorisation rather than reasoning. The diagnostic reasoning scenarios and ACARA comparative judgement tasks are original to AlignED, which reduces but does not eliminate this risk. Item contamination is an inherent limitation of any benchmark using published instruments, and results should be interpreted with this in mind.