Since Black and Wiliam published ‘Inside the Black Box’ in 1998, AfL strategies have dominated a great deal of professional development time, had significant influence on national education policy and has become an unquestioned feature of ‘good practice’.
However, to what extent does empirical evidence support the effectiveness of AfL strategies as implemented in schools? It turns out that this is a complex question to answer – but a broad answer might be ‘less than you think’.
One problem is that even teaching strategies rated as very effective are highly sensitive to context. For example, in Marzano’s review of effective teaching strategies he says:
“Feedback was one of the highest ranked instructional strategies. … But, an examination of the research on feedback indicates that even this “very high-yield” strategy doesn’t always work to enhance student achievement. For example, when Kluger and DeNisi (1996) examined the results of 607 experimental/ control studies on the effect of feedback, they found that over 30% of them showed a negative effect on student achievement.
Where positive evaluations of AfL appear in the literature, there are serious questions about the quality of the evidence these assessments are based on. For example, in a new review of the effects and impact of AfL the authors suggest there is a great deal of indirect evidence supporting AfL, for example:
“There is consistent evidence that participants in AfL initiatives attribute an increase in students’ achievement to the implementation of AfL or of one of the strategies related to it.”
Source: CfBT (2013) Assessment for learning: effects and impact
However, many of the conclusions about the efficacy of AfL appear to be based on rather subjective measures of outcome. Expectation effects and confirmation bias could easily distort the perceived effectiveness of AfL strategies – especially when judged by the teachers implementing them! Additionally, there are usually so many initiatives going on in a school at any one time, perceived positive effects may be due to a completely different innovations!
So – are there any well-focused studies which show clear evidence of the effectiveness of AfL using quantitative measures? From the same review:
“There is only one quantitative study that has been conducted which was clearly and completely centred on studying the effect of AfL on student outcomes. This produced a significant, but modest, mean effect size of 0.32 in favour of AfL as being responsible for improving students’ results in externally mandated examinations.”
Ok, not a huge effect size (Hattie suggests using an effect size of 0.4 as a benchmark of high value effectiveness – but it’s about the same as 0.29 for homework), but a positive effect in a well-focused quantitative study . However, they go on to say …
“It must be mentioned, however, that this study has some methodological problems, explicitly recognised by their authors. These are related to the diversity of control groups they considered and the variety of tests included for measuring students’ achievement. All this affects the robustness of comparisons within the study.”
So, the answer appears to be that there is no genuinely robust evidence to support the effectiveness of AfL strategies.
One reason for the mixed results on the effectiveness of AfL might be the way that it has been implemented. There are reasons to believe that the national strategy and many of the methods taught to teachers were not in line with the original strategy (e.g. it was poorly implemented, it has become gimmicky, highly bureaucratised).
Another possibility might be that AfL is implemented in a way that makes teacher compliance easy to measure for SLT, but actually undermines the effectiveness of the strategy. For example, requiring teachers to report ‘progress’ through (non-existent) sub-levels and generating targets by which a student will reach the next (non-existent) sub-level.
Lastly, there is a genuine question as to whether the ‘core elements’ of AfL strategies as originally presented by Wiliam and Black – placing the student at the centre of learning through self-assessment and independent learning – may contradict what cognitive scientists are saying about the way children learn. Some of the psychological evidence emerging from Europe and the US appears to directly contradict some of the claims and aims of AfL.
So where next? Well, perhaps as a profession, we should stop, reflect and start questioning received wisdom – a good example is set by the Learning Spy in these recent blog articles:
AfL (in some form or another) may well have utility in education and no one wants to throw out the baby with the bath water. However, we should be sceptical of claims about the unbridled benefits of AfL until we can answer some of Marzano’s questions:
“Are some instructional strategies more effective in certain subject areas? Are some instructional strategies more effective at certain grade levels? Are some instructional strategies more effective with students from different backgrounds? Are some instructional strategies more effective with students of different aptitudes?” … “Until we find the answers to the preceding questions, teachers should rely on their knowledge of their students, their subject matter, and their situations to identify the most appropriate instructional strategies”