What Came Back From the Data Raid

We are all collecting immense amounts of data on student wellbeing these days. The proliferation is well-intentioned, and it has been spurred by a significant global industry, but it has also generated more data than many of us know what to do with.

I currently have data sitting across multiple platforms, sheets and tabs. I have internal and external student, teacher and parent surveys; comparative data from our sister school, like schools and regional schools; warm data and street data; and access to the ever-growing body of often contradictory research that seems, somehow, relevant to wellbeing strategy.

The challenge, everyone knows, is that there is no single measure of wellbeing. Each school therefore chooses the proxies it considers most appropriate to its own context. And I have yet to hear of a school that is not trying to pull all this data into one place to use it more holistically, save time, and identify insights that might, ideally, stave off the impending apocalypse.

But collecting more data does not necessarily make things better. That is not how it works. Sometimes I wonder whether we are overcomplicating things, or simply busying ourselves in search of the Holy Grail. Again and again, my own data seems to reduce to one simple conclusion: relationships matter most to children, and the quality of those relationships shapes everything else.

Schools with low attendance rates often also have low student wellbeing metrics. So it would stand to reason that some bean-counting tsar might conclude that the best way to improve student wellbeing is to stamp out low attendance, introducing, or doubling down on, punitive sanctions to enforce school attendance. These schools may well improve attendance, but they rarely make children happier. Whisper it quietly, but perhaps the students are not convinced school is worth attending, or the relationships are not there for them. Schools can work backwards from that reality, or they can fine parents.¹

Perhaps I am missing something, but I do not think you can genuinely improve wellbeing through punitive action.

Survivorship Bias

I think data can mislead us in ways that go beyond misuse. Sometimes it can only show us the thing it was built to see. Which is where a statistician named Abraham Wald comes into my ramble.

In 1943, Wald was presented with data on bullet damage to American bombers returning from raids over Germany.² The holes clustered in the wings and fuselage, so the military decided to reinforce those areas. But Wald pointed out a small problem in this plan: they were only studying the planes that came back. The bombers hit in the engines or the cockpit never returned. The data was not showing where planes were most vulnerable. It was showing where a plane could be hit and still survive.

So the real lesson was not where the holes were, but where they were not.

It seems obvious once it has been pointed out. It rarely feels obvious beforehand. Wald described survivorship bias as the tendency to draw conclusions only from what survives long enough to be counted.

Schools can read their data in similar ways. Strong exam results, university destinations, positive feedback after graduation: these are the returning planes. They constitute an evidence base from which schools draw conclusions about the health of their systems. And those conclusions are often reassuring.

Returning Planes

Michael Sandel has a name for this kind of returning plane: a wounded winner. These are outwardly successful young people carrying costs that do not show up in the data. They have learned to perform and achieve, but not necessarily how to fail, or how to hold their self-worth apart from their credentials, or the borrowed prestige of an esteemed university.

They land. They look intact. But the damage is internal.

This is where the bomber analogy becomes more complicated.

Wald’s insight was about passive oversight. Analysts were studying an incomplete sample and drawing the wrong conclusions. The planes were not damaged by the analysts. They were damaged by German anti-aircraft fire. The error was in the reading, not in the system.

But schools do not have that excuse.

The pressure that produces wounded winners is not entirely external. It can come from the culture of a school, in what gets celebrated in assembly, what gets displayed in cabinets, or how teachers and students talk about grades. Schools can be genuinely committed to wellbeing and still be generating, through their reward structures and rhetoric, the very anxiety they are trying to address. The issue is not just whether we see the damage. It is whether we are also, in part, responsible for causing it.

An Obvious Flaw

I was sharing the bomber analogy with a colleague earlier this week, and I was pleased that he liked it. He also immediately pointed out the obvious flaw:

Aircraft are standardised. Children are not.

Damn. It is always obvious once someone else says it.

I am often unable to tell whether a child is showing signs of withdrawal or perfectionism, unless I know them really, really well, or whether what I am seeing is determination, anxiety, or something else entirely. So it makes sense that good schools have moved toward multidisciplinary approaches. No single teacher, counsellor or head of grade can ever see the whole child. Understanding can only be assembled across many perspectives.

What Came Back

But there is something my own student data keeps returning to, even when I try to look elsewhere.

When we ask students what is best about school, and what is the one thing they would change, the answers do not cluster around curriculum, assessment or facilities. They cluster around two things: their friendships and their teachers. The same two variables appear on both sides of the question, as the source of what makes school good and the source of what makes it hard. We suspected relationships were what mattered most. The students are simply confirming it, and telling us which relationships matter most to them.

Which means the wellbeing conversation cannot remain at the level of systems and indicators. It has to reach the conditions under which teachers can actually know their students, connect with them, and build warmth and trust. So a conversation about student wellbeing has to be prepared to extend to workload, for teachers and students, time, class sizes, and the culture of inclusion, access and belonging.

Schools are shaped by what they choose to measure. But they are experienced by students through things that resist measurement entirely: the conversation after the lesson, a teacher who noticed them, having some choice in what courses they take, knowing they will be listened to if they speak up.

The planes we should worry about are the ones that never came back. Likewise, the students we should worry about are the ones nobody noticed leaving.

---

¹ [Author’s footnote — placeholder]

² Wald was a Jewish mathematician from Transylvania who had fled Nazi persecution when Germany annexed Austria in 1938. By 1943 he was working in a classified unit at Columbia University called the Statistical Research Group, alongside Milton Friedman, Frederick Mosteller and George Stigler. There was a bureaucratic complication: Wald was officially classified as an enemy alien and barred from access to restricted material. A federal court granted him an emergency naturalisation hearing to resolve the impasse — so that a Jewish refugee from Nazi Europe could work on the mathematics of bombing Nazi Europe. The military took his advice. The engines got the armour.

---

---