Tuesday, January 29, 2013

Scientific fraud and why it happens--or how could it NOT?

A piece by Jennifer Couzin-Frankel, "Shaking Up Science," in last week's Science hits hard. Couzin-Frankel writes of two journal editors and their quest to understand cheating in science, and the pressures that have made it so widespread.  From our point of view, they've gotten it just about right.

Immunologist and microbiologist, Ferric Fang and Arturo Casadevall are editor and past editor of the journal Infection and Immunity. They were just doing their jobs until, 5 years ago or so, it occurred to them to wonder what was going wrong with science that people weren't loving it like they once had.
Discovery for its own sake was being sidelined by a push to publish in high-impact journals. Funding was scarcer than ever. Scientists focused on narrow fields and often couldn't communicate their professional passions at a cocktail party.
They began to wonder what they could change, to bring back the passion.  So, they started writing opinion pieces on the subject, people started reading them, and it took off.  To date, they've written 14 and are still going.

They first wrote about the pressure to get grants, and problems with peer review and so on, but one day they, as editors, received a letter telling them that one of the papers they'd published was fraudulent.  They broadened their interest, then, to include how grant pressures might lead to fraud, and they've written a lot about that since.

They wondered whether the prestige of a journal might encourage more fraud, or at least cutting of corners, because high-impact journals are better for careers and so scientists might be more willing to compromise their ethics to publish in high-impact places.  They in fact discovered a robust correlation between prestige and number of papers retracted for fraud.  This didn't surprise them.  As they said in their paper about this in Infection and Immunity in 2011:
Articles may be retracted when their findings are no longer considered trustworthy due to scientific misconduct or error, they plagiarize previously published work, or are found to violate ethical guidelines. Using a novel measure that we call the “retraction index,” we found that the frequency of retraction varies among journals and shows a strong correlation with the journal impact factor. Although retractions are relatively rare, the retraction process is essential for correcting the literature and maintaining trust in the scientific process.
They published a paper in PNAS last October about retractions*, and why papers are pulled. Still rare, about 1 in every 10,000 papers is retracted, they are becoming more frequent but whether this is because fraud is increasing or detection is more efficient is not something they can answer, although they suspect the former.  They reported that 65% of retractions are due to misconduct -- plagiarism, falsification, duplicate publication and the like -- and only 21% due to error.

Fang and Casadevall have thought a lot about why misconduct happens, and why it seems to be increasing, and they largely put it down to flaws in the system, not bad people.  Many departments, for example, will only interview candidates who have been first authors on Science or Nature papers,  meaning, as they put it, that the editors of these journals are making all manner of hiring and career decisions.  And of course the pressure to get grants is widespread and pernicious, largely because universities depend so heavily on overhead from grants.

FIG 1 
Gender distribution of scientists committing misconduct. The percentage of scientists sanctioned by the U.S. Office of Research Integrity who are male, stratified by rank, is compared with the percentage of males in the overall United States scientific workforce (error bars show standard deviations) (blue and green bars are from NSF data, 1999–2006).Source: Fang, Bennett and Casadevall, mBio, 2013
Fang and Casadevall's most recent publication is online in the journal that Dr Casadevall now edits, mBio.  Here, they document, among other things, the gender difference in the proportion of scientists who commit fraud.  Two thirds are male, and they do it at all ages and stages of their careers, which means, as they say, that it's not only when scientists are in training that ethical issues need to be addressed, but throughout scientific careers.  And, it's important, as they say, because:
As many of humanity’s greatest problems require scientific solutions, it is critical for the scientific enterprise to function optimally. Misconduct threatens the scientific enterprise by undermining trust in the validity of scientific findings.
It also slows down progress and costs a lot of money.  Fraudulent results, because they aren't replicable by other labs, are unlikely to form the foundation of future work, but it takes time and money for others to pursue a fraudulent lead and fail.  And it's usually taxpayer money, not to mention valuable career-building time.

How could we not?
Given the System that we, especially senior investigators like ourselves, have acquiesced to or even encouraged, step by step, these findings are no wonder at all.  In a middle-class environment, where ordinary people not just the idle rich can do science, and where we have to do it in universities rather than our basements, and when it is costly, and when we have to persuade the public to pay for it (and our salaries), it would be astounding if we didn't see what we see!

Further, given that all of this puts institutions into the position of needing to insist on imposing the rat-race on its employees, while trying to be 'objective' in evaluating faculty performance by using score-counting rather than judging real creativity (hard to do once organizations get too big, and always hard to avoid insider power structures), and when scaling up brings in more money than having fewer people working at a slower pace, on more focused problems, again we get what we have agreed to over the last few decades.  We agreed to it because when it started a half-century and more ago, there was far less demand on the available supply of resources, and we believed as a nation that government-supported science would help us beat the USSR in the cold war, solve epidemiological problems, and so on.  At the time, something like 50% of grant applications were funded.  The system that developed served all our interests.  We remember this directly from our own experience.

But as the system grew into a System, and put Malthusian pressures on everyone, flooding the market with more people than it could support, to force them into relentless competition with each other, such as even forcing us to have too many graduate students and post docs to boost our status and do our work for us, to enable us to get more grants and as a mark of our prestige whether or not there were ample jobs for them, naturally imposing intense pressure to work 24/7 and so on---that it lead to shaving the truth is only to be expected.

But more important to us than all of the issues raised above, is that the truth is shaved in much more pernicious ways than outright fraud.  The latter may be much more common than widely realized, but is still relative rare compared to investigators knowingly, and as policy, exaggerating their findings.  Negative or less-than-positive results tend to be understated or omitted, or journals won't publish them.  After all, a negative result can't be used to generate a grant to keep doing the same sort of study so we can keep our careers going.  Complicated issues are buried in 'supplemental information' that, at least sometimes, shows that the main paper's claims are over-stated. Marginal positive findings blown all out of proportion by various kind of obfuscating rhetoric and statistical blizzardry.

And of course, the media and science journalists (too many of whom have scant understanding of science, or whose own jobs and sense of prestige pressures them to breathless overstatement) are hungry for the Big Headline story, again 24/7.  It's a mutual reinforcement game of truth-shading. It's not just restricted to science, either.  In an individualistic, profit-based, advertising-driven society like ours, science is just doing what's the going thing for everybody else.  We're all shopping in the same mall.

Investigators, even the most senior, absolutely know that they're playing the game.  Students are trained in 'grantsmanship' from shortly after they arrive.  One regularly hears the smirking comment that to get funded one proposes to do work s/he's basically already done.  And on and on.  None of this dishonor is at all surprising nor, within the profession, is it at all a secret.

Outright fraud is only the tip of the proverbial problem. The rest, the dissembling and so on, and the play of vested interests because of the need to keep the funds flowing, is more subtle and harder to find or document.  The System can wear down anyone who really wanted to try to document its behavior.  But it is widespread if not nearly universal.  Only some, the super-human or like those of us too senior to play the game and who feel liberated to speak about it, can resist.

In this sense, the explanation of science's behavior provides no excuse for it, and yet explains why the problem is not our fault!  It is a System, as we've noted, into which people are brought as junior investigators.  Their forebears (like us) built it, but newcomers inherit it as part of their professional environment.  They must play the game from Day One to survive in their careers for which they've trained so long.  Only by grass-roots demands to scale back, slow down, be more responsible in our statements, reduce the size of the over-stuffed research establishment, and so on, will we return to a more honorable, or more clearly honest, way of doing business.  But right now, the grass roots are quivering in the winds of our times, and there is no rebellion yet in sight.

*A list of retractions is maintained by two science journalists at Retraction Watch.  They welcome tips and comments.


Josh Nicholson said...

Refreshing honest and sad at the same time. I am a 4th year graduate student and I debate daily what I should do next... Is it worth it to play the game?

Anne Buchanan said...

Well, of course whether or not it's worth it to play the game is entirely up to you, but fortunately, at least, the rules of the game are clear and open. In part thanks to Drs Fang and Casadevall.

Ken Weiss said...

A very tough question, Josh. You are a capable, curious, energetic guy and probably can find a place in the System. Whether that likely leads to a good life any more is an open question.

Probably, those of us who have been in the game a long time, have seen its better days (as we feel about it, at least) and who were lucky enough to do well in it, but are rather soured at what it's become, are not the right people to ask.

For most--by far most--people, the greatest impact will be by teaching. If you like that kind of thing, and I think your enthusiasm would make you very good at it, it can be very satisfying.

There are down sides--grumbly students, grading exams--but if you teach 100 students a year, say, and they live on average 60 more years, that's 6,000 person-years of potential influence you can have _every year_! Compare that to the modal number of citations for most scientific papers (i.e., zero or one) and the almost immeasurably short shelf-life of papers. Is the grueling struggle in today's environment worth it?

Teaching at any level, even (or especially?) high school, one can be satisfying, build community, have a personal life, and feel useful.

Science writing of any sort, working in social media, doing an accurate and knowledgeable job of science journalism (a rare thing!), can be an important and also satisfying way of life.

Of course, it's in the interest of graduate programs to make you feel that would be a kind of failure, since their job is to magnify their importance through their students. That's fine as far as it goes, but it's your life, after all.

There are, of course, other completely different ways to live that can be as satisfying. I rather fell into what I've been doing (having been other things before), whether for good or not, but I was not one who marched through the system the way so many do today.

Now you have prominent contacts and at least one relative who made it in the big time, and that may make it harder to consider other avenues. It's up to you, of course.

How the current atmosphere will change, and/or what would happen if funding largely dried up, or what universities will do when codgers like me retire, are all unknowns.

The bottom line is: look in a mirror and decide who you see there--being sure not to care who others might see there. That way, at least, to the extent one can know ones-self (never an easy job), the decision will at least be yours alone.

Josh Nicholson said...

Dear Ken,

Thanks for the advice. I am sure I would never receive it from any of my advisers! We are too busy talking of grants and pubs. It is a pity that academics shame students if they do not become their clones following in their footsteps. A shame because only 10% will really have the option. Thus 90% of students ultimately feel like failures. My girlfriend has written a topic about this to be published in the Chronicle of higher education. Specifically, the need for graduate education to graduate. (it is not published yet)

I am exploring a few different paths that hopefully I can share with you when and if they come to fruition. One is directly related to this post on reproducibility-a practical solution.

Again thanks for your honesty and insight. Stranger to Stranger.

Anonymous said...

I'm always kind of amused when scientists fret about too many researchers, wasting money, etc. Look around at what goes on in the non-science world - bogus litigation for billions. Crooked politicians buying votes by wasting public money. Subsidies paid to the politically connected. Half a billion dollars passed to Solyndra through a crooked connection to the US president that... vanishes, everyone knows it, and no one does anything. By the standards of the world we live in, scientists are pristine. They don't get paid much and occasionally generate things of great value. The real question is not whether they are deserving of public funds - it's why they continue being honest while getting treated like dirt by the crooks in Congress that sell the country out every day for campaign money.

Ken Weiss said...

I agree with all but that we are pristine by comparison. I think--and I believe we have regularly or routinely said this--the anthropological fact that we are part of society, we're human and fallible, we mirror our times today as our forebears did theirs.

But acquiescence is the first step towards tyranny, and there must always be resistance to our foibles, lest they get farther out of hand.

Naturally, those who do resist are almost by definition the minority within. Because science, like any system of its sort, is a hierarchy and has masses who are for one reason or another unaware of the issues, or too vulnerable to try to resist them. Systems keep their masses under control.