Taking science on faith

There's a news story this week in the journal, Nature, entitled "Classical behavioural studies flawed".

"One of the most famous experiments in biology isn't the solid piece of work it's usually portrayed as, say Dutch researchers who have replicated the study. Instead, it's more like an anecdote that became slightly more legendary each time its author retold the story.

The work in question was done in 1947 by the Dutch researcher Niko Tinbergen on the begging behaviour of herring-gull chicks. At the time, the dominant idea in animal behaviour was that learning was all-important. Tinbergen argued that animals co into the world with instincts already adapted to their environments." (The photo is taken from the Wikipedia article on herring gulls.)

Tinbergen's idea became a classical textbook study that guided much of the conceptual thinking in the field of 'ethology' as it was then called, and led to much Darwinian and hence genetic determinism and what has become evolutionary psychology. The finding was that chicks hatch with the instinct to peck at red, the color of a spot on the adult herring gull's bill, to coax their parents into feeding them. His experiments, however, showed that chicks were more likely to peck at any color other than red. Over the course of writing up the results for 10 different publications, he slightly altered the way he described his results so that he explained away those findings -- he said, for example, that he had showed them red more often than other colors and he presumed that this caused them to become acclimated to it -- but he always concluded that they prefer red. He wasn't making up data -- he was explaining away results he didn't like. The Nature authors in fact conclude that he was essentially correct (arguments about such studies will be left for another day), but of course he didn't know that at the time.

The thrust of the Nature story is that we shouldn't call this sloppy science because science was less rigorous then. Even Darwin's work would be pummeled if it were to be peer reviewed today. While that's probably true, it's only partly true. Modern criteria for inference and study design were not invented last year, but have been around for some time. Darwin did not have many of the tools, such as statistical methods, that we use but (as Mendel, Morgan, and many others, including epidemiologists showed) systematic, well-controlled inferential study designs were certainly known.

Good science or not, this is an example of how scientists, who are supposed to always be open to having their results falsified, indeed who (it is often taught in class) go out of their way to falsify their 'hypotheses', in fact tend to cling tightly to their conclusions in spite of contradictory results. Even in the era of 'more rigorous' science, if not significant, results are often described as 'nearly significant' or 'suggestive'. Outliers are discarded. Forced, post hoc reasoning is often offered to explain away weak results or exceptions, and of course the problem always warrants further study. But science is a human endeavor -- more often than scientists like to admit, results are taken on faith.