The Case for a Peer Review Market (guest post)
“The academic peer review system as it currently stands is frustrating and dysfunctional for many of those who participate in it.”
So writes David Thunder, Research Fellow in Political Philosophy at the University of Navarra.
In the following guest post, he briefly identifies some of the problems with the current system of peer review, and suggests they can be remedied with a kind of referee marketplace in which editors can shop for—and purchase—referee services.
[detail of an artwork by Yau Hoong Tang]
The Case for a Peer Review Market
by David Thunder
The academic peer review system as it currently stands is frustrating and dysfunctional for many of those who participate in it. Below, I detail some of its most salient limitations, and afterwards propose an innovation that could mitigate these issues.
Limitations of the Current Peer Review System
1. Because peer review services are often pro bono or done for a nominal fee, they rely on the goodwill and sense of personal responsibility of each reviewer, not on an enforceable contractual obligation. Because of the pro bono nature of many reviews, the motivation for doing them is something like “duty to the profession,” unless a reviewer has a strong personal interest in a specifical manuscript. There is no enforceable contractual obligation to speak of. This has two negative consequences:
- First, reviews may be either half-hearted or submitted very late, which has negative knock-on effects for publishers and authors alike.
- Second, due to the limits of moralistic motives that are not remunerated, editors may have to spend months trying to secure a scholar willing to conduct a time-consuming review.
2. Because there is almost no form of public accountability for reviewers, and they know this, the quality of reviews is mixed. Some are excellent, others acceptable, and others based on personal prejudices or superficial readings of a text. In any case, the effect of a pro bono system, combined with the fact that reviewers’ work is not publicly evaluated or held accountable, is to create a class of gatekeepers who only answer privately to editors for the quality and timeliness of their work, and cannot realistically be pressured too much given that their work is pro bono.
Effects of the Current System for Authors and Publishers
1. Publishers are put in a difficult situation in which they subject authors to lengthy, career-damaging delays over which they have little control.
2. Publishers may find that their ability to bring work to the market efficiently is undermined by needlessly drawn out review times.
3. Authors may find articles and book manuscripts sitting under review far beyond the estimated review times. In the case of book manuscripts, these delays may be especially long (anywhere from 6 months up to 18 months), and have negative repercussions for a scholar’s career.
A Proposal for a Peer Review Market
I propose to create a virtual peer review market, through which both authors and publishers/editors can search for available peer reviewers and solicit their services on a competitive and professional basis.
The basic idea is that each peer reviewer compiles an online profile which may be independently vetted before it goes live. The profile would include name, educational background, publications, number of peer reviews conducted, average rating for their reviews, and a negotiable offer price and guaranteed turnaround time for reviews.
Editors and authors would also have an online profile in the system, and they could solicit peer reviewers’ services and indicate their required turnaround time. They could also use the system to rate the work done under different variables (e.g. quality, comprehensiveness, punctuality), and make payment for services rendered.
Benefits of a Peer Review Market
People who are paid for the work they do tend to give it a higher priority and feel a strong sense of duty to do it properly and on time. In addition, if reviewers know they will be rated publicly for their work, they are more likely to take is seriously. The end result of a marketized system is more peer reviews of a higher quality, and more peer reviews turned in on time.
A second advantage of a public peer review market is that editors and authors may search for eligible reviewers from a much larger database that includes important data about past experience and reliability as well as areas of specialization. When people know they can make money from reviewing manuscripts, they will have a greater incentive to up their game and develop an attractive portfolio of review experiences. This is a win-win for authors, editors/publishers and reviewers alike.
Challenges and Questions
A professional peer review market may pose certain challenges, especially at the start:
- Universities, journals and book editors would have to expand their budget for academic reviews if pro bono reviews are phased out.
- Some unscrupulous reviewers may try to “game” the system by getting paid for low quality reviews. But this should be discovered fairly quickly when their work is rated by editors and/or authors.
What do you think are the strengths and weakness of this proposal? Do you have any practical suggestions for the implementation of this or other reforms of our peer review system?
The post The Case for a Peer Review Market (guest post) first appeared on Daily Nous.