I’m a sucker for good satire. In a recent post I referenced Dorothea Salo’s delightfully satirical article, “How to Scuttle a Scholarly Communication Initiative” where she lays out a detailed agenda for dissuading academic libraries from effective participation in scholarly communication activities on their campuses. This week, while trying to find the best hook for posting about the ‘sting operation’ conducted on a selection of open access journals recently reported in the journal Science, I landed on Mike Taylor’s October 7, 2013 blog post, “Anti-tutorial: how to design and execute a really bad study.”
The blog-o and Twitter-spheres have over the last four days offered extensive reporting and analysis of the article that appeared in the October 4, 2013 issue of Science. If you are one of a handful of persons who by now has not heard about this story the gist is this: The author of the article, John Bohannon, a biologist and science journalist, designed a fake research paper with obvious methodological and scientific errors and sent out permutations of it using the names of fictitious authors from fictitious African institutions to 304 open access journals over an eight month period from January to August 2013. Bohannon reports his results thusly: “By the time Science went to press, 157 of the journals had accepted the paper and 98 had rejected it.” Based upon these incredible acceptance rates of an obviously faked paper, the article appears to level a clear and damning indictment upon the peer review processes and/or ethical practices of open access journals, especially those that levy article processing charges (APCs)—which Bohannon calls the “standard open-access ‘gold’ model”—as a condition of publication. Near the top of the article, Bohannon characterizes his findings as “reveal[ing] the contours of an emerging Wild West in academic publishing.”
As persons began to dig into Bohannon’s data it became apparent that many of the journals to which he sent his paper were known predatory journals, predisposing the outcome. Too, objections were raised about Bohannon’s methodology including the fact that he did not also send his fake paper to traditional toll-access journals, which might have offered a basis of comparison on acceptance/rejection rates. Further, is it really true that the levying of APCs constitutes the “standard open-access ‘gold’ model”? Despite some interesting findings—such as the fact that many open access journals not only rejected the paper but also raised serious ethical concerns (e.g., PLOS ONE), while several journals owned by large commercial publishers such as Elsevier, SAGE, and Wolters Klewer accepted the paper—speculations are beginning to surface about the real purpose of this article, including why Science (a well-respected toll-access journal) agreed to publish it in the first place, given its data and methodological problems.
No one is disputing the problem of predatory open access journals, and all agree that publishers of these journals should be put out of business. But at the end of the day, there isn’t really much new offered in the article. Indeed, apart from the research element, it has a similar tone as a piece run last April in The New York Times, down to the use of the phrase ‘Wild West,’ and the fact that traditional toll-access journals were not scrutinized. (I wrote about that article here.) Is this just another attempt to discredit, or at least cast doubt on the credibility and viability of open access as a publishing model?
“Truthiness” and “sciencey”
Assuming the speculation is correct—that there is still a need and a strong desire to discredit open access—how might this be accomplished? Enter Mike Taylor’s “anti-tutorial.” I commend it to your reading. (Please remember that it’s satire.) [Apologies: It occurred to me after originally posting that I neglected to identify Dr. Michael Taylor as a paleontologist in the Department of Earth Sciences at the University of Bristol, UK.]
Oh, I can’t finish this post without mentioning this most delightful excerpt from Taylor’s piece:
[As a molecular biologist with a Ph.D., John Bohannon] does know what science looks like, and he’s made the “sting” operation look like it. It has that sciencey quality. It discusses methods. It has supplementary information. It talks a lot about peer-review, that staple of science. But none of that makes it science.
“It has that sciencey quality.” I love that word sciencey! When I read it, it reminded me immediately of comedian Stephen Colbert’s use of the word “truthiness” in a segment of his political satire television show The Colbert Report back in October 2005. The Wikipedia article on “truthiness” cites a quote from Colbert about his use of the word: “We’re not talking about truth, we’re talking about something that seems like truth—the truth we want to exist.” Indeed, the Wikipedia article cites a definition of truthiness “as a quality characterizing a ‘truth’ that a person making an argument or assertion claims to know intuitively ‘from the gut’ or because it ‘feels’ right without regard to evidence, logic, intellectual examination, or facts” (emphasis added).
Maybe when we use “sciencey” we’re talking about something that seems like science. But I definitely hope it’s not the science we want to exist. In the end, John Bohannon’s article doesn’t really help us to evaluate the status of open access because it is flawed science. Publishing it in Science doesn’t make it any better. Thanks Mike, for an insightful and entertaining tutorial.
P.S. I discovered that although obsolete or rare, both “truthiness” and “sciencey” are real words, with entries in The Oxford English Dictionary (I consulted the 1933 edition).