The certainty of chance
Ben Goldacre reminds readers how associations may simply reflect the play of chance, and describes Deming’s illustration of this.Key Concepts addressed:
Ben Goldacre, The Guardian, Saturday September 6 2008
Britain’s happiest places have been mapped by scientists, according to the BBC: Edinburgh is the most miserable place in the country, and they were overbrimming with technical details on exactly how miserable we are in each area of Britain. The story struck a chord, and was lifted by journalists throughout the nation, as we cheerfully castigated ourselves. “Misera-Poole?” asked the Dorset Echo. “No smiles in Donny,” said Doncaster Today.
From the Bromley Times, through Bexley, Dartford and Gravesham, to the Hampshire Chronicle, everyone was keen to analyse and explain their ranking. “Basingstoke lacks any sense of community or heart,” said Reverend Dr Derek Overfield, industrial chaplain for the area. And so on.
Exactly what kind of data is the good reverend explaining there? The Times had some methodological information. “Researchers at Sheffield and Manchester universities based their findings on more than 5,000 responses from the annual British Household Panel Survey.” According to the BBC it was presented in a lecture at some geographical society. “However,” they said quietly, “the researchers stress that the variations between different places in Britain are not statistically significant.”
Here, nestled away, halfway through their gushing barrage of data and facts, was an unmarked confession: this entire news story was based on nothing more than random variation.