As in many departments, our graduate students and post-docs here in the Penn State Anthropology Department hold weekly 'journal clubs' where recent interesting papers are discussed. Last week, the students discussed the nature, value and importance of tabulations of journal impact factors (IF), basically the citation rate per published paper. There have been many papers and commentaries on this subject in recent years, but this session focused on a paper by Brembs, Button and Munafo entitled "Deep impact: unintended consequences of journal rank," published in 2013 in Frontiers in Neuroscience.
The
IF scandal
This article assesses the assigned IF of journals relative to their retraction, error or fraud, and reliability or replicability rates. The objective picture of the IFs is not encouraging. Statistical analysis shows that the 'major' journals--the expensive, exclusive, snobbish high-status ones are, in terms of the quality and accuracy of their content no better, and arguably worse than the less prestigious journals. We won't go into the details but basically there is a rush to publish dramatic results in those status journals, the journals are in business to attract and generate attention, and that leads them to receive, and publish, splashier claims. In this self-reinforcing pattern they garner the honors, so it is worrisome that they appear to do this without actually publishing the best research or, worse, systematically publishing unreliable work.Our student seminar discussed this problem and the effect it may have on their careers, if their work is going to be judged by a somewhat rigged, and inaccurate, scoring system. If they can't get into the high-IF elite publishing club, which is somewhat self-reinforcing, how can they compete for jobs and grants and get their work known?
We have several reactions to this. For students and others who may have a stake in (if not in the heart of!) the system, here are a few thoughts on alternatives to the high IF journals. The picture is grim, but in some surprising ways, not at all hopeless.
Some thoughts for students:
First, TheWinnower, founded by Josh Nicholson, a graduate student at Virginia Tech, is a new online site where one can send papers but also where blogs and other such new-media communications can be published, and these publications given a DOI (formal document identifier) and hence be more regularly citable and permanently archived. It's but one of many new communication venues. A lot of what is on these media is of course superficial fluff, but (Sssh!! don't tell anyone!) so is a lot of any sort of publication, even (believe it or not!) in the 'major' journals and so has it always been, even in the old-time printed journals of yore.
Secondly, there are allies in any movement towards change, not just from the grass roots where pressure for social change usually arises. There are thoughtful and progressive administrators, and serious scholars and scientists, who are resisting the pressure to use IF score-counting, in career evaluations, purportedly to make them more 'objective'.
And, there are many people making their way largely and in various ways on blogs, open-access publishing, online teaching, communicating with people via Twitter and other outlets (most of which we, being quite senior, probably don't even know of!). Writing for public media of all sorts has always been a mainline, legitimate way to build careers in anthropology, especially its sociocultural sides. But generally, critiques of the system at all levels, such as repeated revelations about score-counting bureaucracies and IF biases, as well as objections to closed access publishing, will have their impact if they are repeated often and loudly enough.
Thirdly, ironically and reassuringly perhaps, the tightening of the grant prospects and the well-documented concentration of funding in the hands of senior investigators, means more people will have to rely less on grants, and their university employers will simply have to recognize that. Teaching and other forms of service, scholarship, and outreach will simply have to be reinvigorated. Universities aren't just going to close shop because their grant funds shrink. They're not even going to be able to keep shifting towards hiring poorly paid Instructors. So the field is open for innovation and creativity.
Fourthly, also ironically, the greater the rush to the Big Journals, the better it may be for the job prospects of current grad schools? Why? Well, fewer people in the running for each job will have such publications on their CVs than perhaps was the case in the past. As long as applicants realize that others will want the same jobs they do, and they develop their skills and depth of thought accordingly, they'll compete well. After all, colleges and universities will simply not be able to hold out for those few with BigName publications, even if they wanted to. They'll be 'stuck' having to evaluate people on their actual merits. And, not so trivial as you might think, most of their faculty haven't got BigName papers either, and might not want to be outshone by adding a junior hyper-achiever to their midst. Indeed, many less research-intensive but wholly academically serious places feel, correctly, that applicants for faculty positions who have BigName publications don't really want to work there and will move on as soon as they can get a ‘better’ job, and/or in the meantime won't be dedicated to teaching, students and the local institution. So things aren't always as dire or as one-sided as they seem--even if times are relatively difficult right now.First, TheWinnower, founded by Josh Nicholson, a graduate student at Virginia Tech, is a new online site where one can send papers but also where blogs and other such new-media communications can be published, and these publications given a DOI (formal document identifier) and hence be more regularly citable and permanently archived. It's but one of many new communication venues. A lot of what is on these media is of course superficial fluff, but (Sssh!! don't tell anyone!) so is a lot of any sort of publication, even (believe it or not!) in the 'major' journals and so has it always been, even in the old-time printed journals of yore.
Secondly, there are allies in any movement towards change, not just from the grass roots where pressure for social change usually arises. There are thoughtful and progressive administrators, and serious scholars and scientists, who are resisting the pressure to use IF score-counting, in career evaluations, purportedly to make them more 'objective'.
And, there are many people making their way largely and in various ways on blogs, open-access publishing, online teaching, communicating with people via Twitter and other outlets (most of which we, being quite senior, probably don't even know of!). Writing for public media of all sorts has always been a mainline, legitimate way to build careers in anthropology, especially its sociocultural sides. But generally, critiques of the system at all levels, such as repeated revelations about score-counting bureaucracies and IF biases, as well as objections to closed access publishing, will have their impact if they are repeated often and loudly enough.
Thirdly, ironically and reassuringly perhaps, the tightening of the grant prospects and the well-documented concentration of funding in the hands of senior investigators, means more people will have to rely less on grants, and their university employers will simply have to recognize that. Teaching and other forms of service, scholarship, and outreach will simply have to be reinvigorated. Universities aren't just going to close shop because their grant funds shrink. They're not even going to be able to keep shifting towards hiring poorly paid Instructors. So the field is open for innovation and creativity.
Fifth, if the intense rigors of the research-intensive Fast Lane appeal to you, well, you know the gig and its competitive nature, and if you get your advanced degree from a fine and well-regarded program that will give you a chance at getting the brass ring. Those avenues are of course open, even if highly competitive.
 |
| "Painted Pony Bean" by Liveon001 © Travis K. Witt - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons - |
But why does anyone even tally such things as impact factors?
An obvious question one should be why anybody would tally impact factors in the first place? Who has what to gain? The answer has to be that it is in someone's interest and someone will gain by it. After all, when some of us started our careers, there was no such thing (or, the earlier version Science Citation Index, was remote, in the library, laborious to look through and then usually only for legitimate resource searching). Scholarship itself was on average at least as good as now, careers were made without bean-counting but more on merit and substance, and bean-counting expectations were lower (and respect for teaching higher), the grant game much, much less intense.
IF scores are computed by a commercial company, Thompson-Reuters, as---what? As a favor to the publishing industry, and for what we would call a kind of academic bourgeois market for baubles and vanity. Journals self-promote by gaming their IFs, universities self-promote by gaming their faculty's IF ratings. They have money to make by promoting and, yes, manipulating their IFs (see the above article for just some of the ways). One can ask whether there is even a single reason for such score-keeping to be done other than for reasons of artificially constructed status hierarchies.
One motivation for this bean-counting is the heavy proliferation of online journals. Some of these are very highly respected, and deservedly, while others are chaff or, worse, scams for making money playing on fears and insecurities of faculty members needing advancement. IFs will at least be some assistance to an administrator or grant reviewer who wants to have an idea of a faculty candidate's record. But if the IFs are systematically unreliable, or manipulated, or even reverse indicators of actual work quality as some articles like the one above have suggested, that is a rather lame rationale for using IFs. Administrators evaluating their faculty members' careers should look at the actual work, not just some computer-tallied score about it. That may not be easy, but administrators are well-paid and accepted their jobs, after all.
There was in the past an insider Old Boy network in academe, that discriminated more arbitrarily in terms of funding, over-powerful Editors who controlled who published and what they published, and less opportunity for women and cultural minorities (based on ethnic as well as university status hierarchies). To increase fairness, but also to avoid discrimination lawsuits, and to play the self-promotion PR spin game, universities and their administrative officials learned the value of being 'objective' and hiding behind Excel spreadsheets rather than judgment. More objectivity did in many ways dislodge the older elite insider networks, but a system of elites has clearly re-established itself, and manipulable IF factors and their associated commercial incentives have helped reestablish some dominance in the academic system. It may still be wide open in many ways, but is heavily burdened by the game because the corporate university has become so money-oriented. This is very well documented. Things like IFs serve those interests.
The academic world will experience change, and is changing, and the new ways of communication are better and faster and more open-structured than ever. They make life more frenetic, but that will probably calm down because it's exhausting everyone. There will of course always be an elite, and for some that's a happy community to be part of. But it's not to everyone's taste. How long it will take coup-counting administrators to accept these other venues such as online communications, is unclear, but it's happening.
Social change requires resistance to the status quo, usually organized resistance (or else money-based leverage). Bureaucracies do need to be pressured, by faculty, graduate students and post-docs, and people like Department Heads and Chairs. But, it has to happen, and it will.
An obvious question one should be why anybody would tally impact factors in the first place? Who has what to gain? The answer has to be that it is in someone's interest and someone will gain by it. After all, when some of us started our careers, there was no such thing (or, the earlier version Science Citation Index, was remote, in the library, laborious to look through and then usually only for legitimate resource searching). Scholarship itself was on average at least as good as now, careers were made without bean-counting but more on merit and substance, and bean-counting expectations were lower (and respect for teaching higher), the grant game much, much less intense.
IF scores are computed by a commercial company, Thompson-Reuters, as---what? As a favor to the publishing industry, and for what we would call a kind of academic bourgeois market for baubles and vanity. Journals self-promote by gaming their IFs, universities self-promote by gaming their faculty's IF ratings. They have money to make by promoting and, yes, manipulating their IFs (see the above article for just some of the ways). One can ask whether there is even a single reason for such score-keeping to be done other than for reasons of artificially constructed status hierarchies.
One motivation for this bean-counting is the heavy proliferation of online journals. Some of these are very highly respected, and deservedly, while others are chaff or, worse, scams for making money playing on fears and insecurities of faculty members needing advancement. IFs will at least be some assistance to an administrator or grant reviewer who wants to have an idea of a faculty candidate's record. But if the IFs are systematically unreliable, or manipulated, or even reverse indicators of actual work quality as some articles like the one above have suggested, that is a rather lame rationale for using IFs. Administrators evaluating their faculty members' careers should look at the actual work, not just some computer-tallied score about it. That may not be easy, but administrators are well-paid and accepted their jobs, after all.
There was in the past an insider Old Boy network in academe, that discriminated more arbitrarily in terms of funding, over-powerful Editors who controlled who published and what they published, and less opportunity for women and cultural minorities (based on ethnic as well as university status hierarchies). To increase fairness, but also to avoid discrimination lawsuits, and to play the self-promotion PR spin game, universities and their administrative officials learned the value of being 'objective' and hiding behind Excel spreadsheets rather than judgment. More objectivity did in many ways dislodge the older elite insider networks, but a system of elites has clearly re-established itself, and manipulable IF factors and their associated commercial incentives have helped reestablish some dominance in the academic system. It may still be wide open in many ways, but is heavily burdened by the game because the corporate university has become so money-oriented. This is very well documented. Things like IFs serve those interests.
The academic world will experience change, and is changing, and the new ways of communication are better and faster and more open-structured than ever. They make life more frenetic, but that will probably calm down because it's exhausting everyone. There will of course always be an elite, and for some that's a happy community to be part of. But it's not to everyone's taste. How long it will take coup-counting administrators to accept these other venues such as online communications, is unclear, but it's happening.
Social change requires resistance to the status quo, usually organized resistance (or else money-based leverage). Bureaucracies do need to be pressured, by faculty, graduate students and post-docs, and people like Department Heads and Chairs. But, it has to happen, and it will.
