Reliability and perfection

I

reliability, n. the extent to which an experiment, test, or measuring procedure yields the same results on repeated trials.

perfection, n. the action or process of improving something until it is faultless.

II

“There’s a paradox here. Ask most research physicians how a profession can advance, and they will talk about the model of ‘evidence-based medicine’. …[I]n a 1978 ranking of medical specialities according to their use of hard evidence from randomised clinical trials, obstetrics came in last. Yet almost nothing else in medicine has saved lives on the scale that obstetrics has.”

– from The Score: How childbirth went industrial.

III

I have found myself in a state of slowly trying to procure the most perfect lesson. To fix things up so that it runs minute-by-minute according to a plan, in which I am asking just the right questions, have tasks that are carefully scaffolded at the precise level for each of my students, with every stage of the lesson considered and noted down. The problem with this is that the complexity of the details that I’ve built up in my mind across the 75 minutes of a lesson don’t play out as planned. This, of course, is because I’m dealing with actual people – and young, inquisitive, social teenagers at that.

Instead of needing perfectly tailored lessons they need reliability.

IV

The Apgar Score is a measurement tool that is used in obstetrics. It is a very, very simple tool to administer, consisting of five questions worth two points each. As a result of having this standardised measure used for diagnostic purposes and for comparing the health of newborns, this tool has saved the lives of many babies and their mothers who would otherwise have been at risk of death. And part of the beauty of this tool is that it concentrates on the notion of overall reliability as opposed to the perfect measurement of many characteristics for each and every child.

For an individual doctor, it may seem more useful to work with a detailed tool that goes into the particulars of the baby they are delivering. But consider this one child in the context of all of the babies that are born and how the doctor can learn from the vast numbers of babies and apply that learning to the one individual baby.

If you can gather data on a general population, then as a consequence you will be able to have greater insight into what goes on within that population, including broader patterns or trends, in contrast to the perspective that would be gained by looking at a smaller case study in a greater amount of detail.

V

Given the simplicity of the Apgar score, it can be delivered by anyone, no matter the level of expertise and no matter the location. The same sort of thinking needs to be applied in schools.

In the complex environment of a school, we are consistently guilty of trying to find the most perfect way of measuring where a student is at or planning a lesson or analysing the performance and progress of a teacher year by year. Now, don’t get me wrong, all of this is important and I don’t deny the utility in understanding each of these insights. However, if we are to better consider the progress of a student and the progress of a number of students overall, then we need to be looking at them as a conglomerate. We need to be thinking, what is a more reliable way in which we can assess improvement and be able to do this in an efficient way?