Tuesday, October 19, 2010

Awww, I take it all back! Retraction in science

Scientists, like other people, don't like to be wrong. We make our reputations, satisfy our egos, please our children, and make our careers by being right--by seeing farther than has been seen before.  Being wrong gets old in a hurry, and it may be hard to recover from that if it happens too often.

No one likes getting old either, so, when the idea was published that one could reverse some aspects of aging transgenically,  it was great news.

Until.  Until it turned out that maybe it wasn't so true after all.
Harvard researchers have retracted a far-reaching claim they made in January that the aging of stem cells might be reversible.
A set of papers on this have been retracted, or warning given that the reported findings may not be all that trustworthy. This story was in the news, but there's a site called Retraction Watch, that tries to catalog and report these instances, so other investigators or interested parties will know about the retractions. These are more than the Errata notices at the back of many journal issues (and there are routinely more of these than heretofore because of the haste-to-publish rat-race we're all in).

No matter whether it's complete or not, this is a fine service if people know about it.  But while retractions are often reported in the news as splashy scandals, and they may involve miscreant or sloppy scientists or their lab staff, retraction should be encouraged rather than discouraged.  And the quicker the better!

Care should be exercised before submitting something for publication, though it is probably regularly undermined by haste.  But if mistakes are caught, they need to be made widely know. We build our careers on the backs of others' published work.  Cost, time, effort, and even policy or medical advice can be based on what is published.

Haste and other pressures lead to dissembling and mistakes in science, but there is (we think) very little outright fraud.  There are cases, however, and the offenders need to be sanctioned by ostracization from the profession.  But, again, it's rare and should not pollute attitudes towards science.  These mistakes are not as problematic as science going off in poor-payoff directions, which is systematic and can be common and costly.

We're all human and saying we've been wrong is important, and should be encouraged.  More is often learned that way, or even by 'negative' results, than by incremental or trivial positive results.  We were glad to learn of Retraction Watch.  Repeat retractors' operations need to be put under scrutiny.  But retraction for honorable reasons should be praised, not turned into scandal.

2 comments:

n8craig said...

I'd like to see this applied to archaeology as well. I've never found an errata section in an archaeology journal, and I can think of several famous papers that contain major errors. Worse yet, excavations can't be replicated. Pile on that the fact that archaeologists are typically very protective of their data (despite NSF data sharing policies). I can think of at least two NSF funded archaeologists who outright refused to share minor pieces of data that are now over five years old. Civil discourse and debate should be part of the process of science. Though most archaeologists borrow from Merton's middle range theory (as a bridging argument), few archaeologists actually practice CUDOS (a tradition that incorporates both systematic skepticism and openness about procedures, data, etc).

Ken Weiss said...

I don't know what middle range theory is, but data sharing is a difficult issue. Fortunately, in genetics things are pretty good. Much of the credit goes to PLoS publication, and to the NIH and Nat. Center for Biotech Information, which require release of data and, after a moratorium of some 6mo or a year (I don't remember) public access to papers published under NIH support.

Of course games can be played. Raw data may be useless to those without the lab or computational resources to analyze it, etc.

Still, while people do feel protective of hard-won data, sharing is rather ubiquitous, a very good state of affairs.

The Errata issue doesn't take care of intentional fraud, but that seems to be very rare. It reflects the haste in publication and sometimes even in editing. Most Errata, I need to point out, are trivial (mislabeled figures, author names omitted by mistake, etc.). Few are fundamental to the point of a paper.

Replication is sometimes, but not always possible in genetics and life science. But usually, if something is important, some forms of checking can--and will--be done. That doesn't mean the system is perfect, of course. Some major journals will not publish negative results even though they can be the most informative, or they won't publish rebuttals to papers (or bury them in unseen, online postings). A negative results site for drug testing has been set up, by NIH I think (we don't do that kind of research), but it's voluntary and I've heard that it isn't getting much contribution.

Not publishing negative findings biases understanding and leads to lots of waste in doing things that have little chance of success. But that's another, more political, subject about current genetics.