There was a recent discussion on our favorite medium, the BBC (in this case, the World Service on radio) about the problem of scientific fraud. How common is it? How harmful? How does it get through the peer review process?
Science depends more than perhaps any other area of human societal life, on honesty. That's because we each can only do our own experiments, and can't possible do all that was done before that set the stage for what we attempt to do. Fortunately, real fraud seems to be very rare in science. It occurs, but is usually punished when caught, by strong sanctions. It is very serious indeed, because investigators can spend precious time, effort, and resources pursuing ideas suggested by published reports of important findings or methods. The findings are widely cited and used in support of the new ideas in, for example, grant applications. In anthropology, the Piltdown fossil fraud was accepted by many scientists for decades, and built into texts and other frameworks for analyzing human evolution. The recent Korean cloning fraud misdirected many labs into wasteful cloning experiments.
Major fraud is caught in various ways, perhaps most often by others trying, and eventually failing, to replicate an important result. Minor fraud may be more common--we know about the major cases because of the problems they cause, but few care about minor results. And dissembling and exaggeration and self-promotion are rife. The public may not be aware of it, but scientists usually are (and often justify doing it themselves because 'everybody does it' and 'you have to, to get funding or to be published in Science.')
Puffery has consequences similar to fraud in that skillful hyperbole establishes fads, and most of us, desperate for attention and funding and so on, eagerly jump on band-wagons (especially when a new toy, like fMRI, gene expression arrays, and the like are available). Overall, it is likely that puffery in fact causes more damage by diverting funds and energies in areas less promising than is believed.
Peer review is the system that supposedly keeps up the scientific standard. An article is submitted to a journal, and if deemed relevant, is sent to 2 or 3 experts to judge. They are to judge quality and appropriateness, importance or novelty, and find factual errors and so on that may be in the reported work. But, of course, these peers may also be rivals, or (almost always) too busy to pay close attention to the paper they're sent. Nowadays, with the huge proliferation in the numbers of frenetically competing investigators, and new journals, and tons of online 'supplemental' information allowed, it is nearly impossible to maintain a standard.
When reviewers find unclear statements, things not well explained, or actual errors, then they do the original authors a huge positive service by improving the paper before it is published.
Peer reviewers were never asked to find fraud, however. They are hardly ever in a position to do that anyway. They, like all of us, must in practice assume the honesty of the authors. So peer review doesn't generally find fraud.
One thing the system of peer review does that we find increasingly irritating--and try to avoid ourselves--is empower the reviewer to insist that the author(s) modify the paper in ways that, when you get right down to it, would make the paper more like what the reviewer would write if s/he were the author. This leads to often extensive modification in ways that are forced rather than natural, and not what the authors actually wanted to say. The authors, anxious to get their work published and move on, will then litter their revision with every kind of cumbersome citation, caveat, and implicit rhetorical bow to the reviewers--so they can tell the Editor they responded to all of the comments.
More and more, we feel, the reviewer should find mistakes and unclear aspects of papers, but just let the authors have their own conclusions and interpretations. If the authors do a bad job, then readers of the paper will do better work, ignore the paper, or whatever. But if the authors do a good job--including a good job of presenting their case in a style they wish to use--then the paper they wrote is better off unadulterated.
Peer review is valuable, but could be improved if reviewers were instructed to stick to the important issues. Let the poets in science out of the lab!