Reasons to be cheerful about research

The hard problems of education

Education is faced with many hard problems. Perhaps two of the biggest at the moment are: Which teaching methods lead to the greatest gains for our students? How do we overcome the link between low socio-economic status (of students and school communities) and poor educational outcomes?

Over my career as a teacher I’ve witnessed numerous ‘expert’ authorities claim to have solved some of these hard problems. Most of the suggested approaches were pseudoscience and wishful thinking: Brain Gym is the classic case; VAK, NLP and other neuro-myths are still highly prevalent examples.

We’ve had national strategies which took promising ideas like formative assessment and implemented it whole-scale with little attention paid to how bureaucratic or ineffective it was (e.g. APP).

We’ve had the schools regulator creating unhelpful orthodoxies regarding ‘good teaching’ (e.g. ‘talk-less teaching’), leading to school leaders implementing policies with a focus on mechanistic compliance amongst teachers rather than genuine efficacy.

Many of these interventions are based on nothing more than superstitious thinking by teachers and school leaders who desperately need to show they’re doing ‘something’ to resolve the hard problems of education within the context of their school. David Didau relates a perfect example of this mindset in a recent blog post.

We need a more rational approach to improving education. Surely, using education research to guide policy and professional development would be a better approach?

One of the obstacles has been the impoverished state of much education research. In a recent blog, Richard Farrow identifies the range of problems; poor research design, small samples, uncritical reporting of effect sizes, using inappropriate measures of teacher quality, poor assumptions about the validity of self-report, ignorance of expectancy effects and participant reactivity, potential bias due to financial interests – to list some examples!

To sum it up simply, as Ben Goldacre put it:

“Teachers. If a non-randomised study of 30 kids in one class counts as significant evidence, your sector is broken.”

It’s clear that an uncritical approach to research has been a major problem in education. Some of the stuff out there might be typified as ‘opinion with numbers’ rather than an attempt at a genuinely scientific approach. As Farrow points out, the problem is not always the researchers, who often detail the significant limitations of the work, but the fact that flawed research is used without such caveat to guide policy.

Reasons to be cheerful

The problems and difficulties in researching education shouldn’t put us off the attempt. The alternative is that our evidence base remains at a broadly journalistic level and the profession remains essentially vulnerable to the well-meaning but misguided or the outright unscrupulous.

When it comes to tackling subjects of such enormous complexity, science can achieve important insights. Some claim that the hard problems of education are too complex and open-ended for a scientific, evidence-based approach – but you only have to look at fields like climate change or genetics to see how much can be accomplished in complex fields.

Modern cognitive science provides a fairly robust starting point for understanding how children actually learn. The differences between children is often cited as a major obstacle for applying this research, but these objections rather ignore the fact that all humans learn in remarkably similar ways – snowflakes, for all their uniqueness, are all made of water after all. By focusing on the common ways in which we learn, we actually make it easier to understand the differences. Psychological models of learning can usefully guide professional judgement in the classroom when it comes to dealing with so many individual needs. As our evidence-base improves, ways to approach those differences will be easier to develop because we understand why and how they depart from the typical model.

Teachers work extraordinarily long hours and there’s no sign that workload reduction is a serious priority for any of the political parties. By and large, teachers are interested in research that might help their teaching, but have little to no time to read and synthesise numerous journal articles or books. Teachers come from a wide-variety of disciplines; most of which do not have any scientific training. Not all teachers can reasonably be expected to train up to a post-graduate understanding of research and statistics, thus it is useful to have access to a specialist who can find high-quality research articles and summaries on specific questions. Research leaders could also make a positive contribution to helping teaching professionals access and understand research, whether working with beginning teachers, newly or recently qualified, or as part of a whole-school CPD.

Undertaking small-scale qualitative or quantitative research in schools won’t provide high quality evidence that will impact beyond the specific classroom or school context. However, we should encourage the attempt to move beyond opinion as the basis of evaluating our teaching. Lesson study (for example) is more-or-less a case report or a case series study of the impact of lessons upon a small number of students. Schools could reasonably attempt research up to the level of cohort studies to try and evaluate their whole-school initiatives.

Beyond the scale of an individual school, randomised controlled trials – especially with blinding – can help test the effectiveness of innovations within education. RCTs can never entirely tell us ‘what works’, but well-constructed studies with appropriate samples can at least help weed out what doesn’t typically work.

For all the problems with education research, I agree with John Tomsett’s assessment:

“We should not turn away from research due to skepticism; rather, we should set the agenda as a profession. A rich knowledge of good quality research, and an active engagement in that research, empowers teachers and school leaders. It gives us the tools to make better informed decisions about pedagogy and any small gain in decision-making which improves students’ learning experiences should be seized.”

Schools risk being a ‘cash cow’ for psychologists and educationalists looking to make their academic careers or their personal fortunes selling ‘solutions’ to the hard problems. A greater understanding of research methods and statistics will help teachers develop professional scepticism – ‘immunising’ teachers and future school leaders against the most egregious snake-oil.

If we want to find better answers to the hard problems, we need education research to step up and raise its game. It’ll only really do that if the profession can engage critically with research and start demanding more from it. If teachers can learn to better scrutinise and challenge evidence, perhaps we’ll even start to make headway with arguably the hardest problem in education:

Can we make teachers and schools accountable in a way that doesn’t make teaching and school leadership unattractive and unsustainable?