“Truthiness” isn’t quite truth, and “sciencey” isn’t quite science, even if published in Science: Mike Taylor’s “Anti-tutorial: how to design and execute a really bad study”

I’m a sucker for good satire. In a recent post I referenced Dorothea Salo’s delightfully satirical article, “How to Scuttle a Scholarly Communication Initiative” where she lays out a detailed agenda for dissuading academic libraries from effective participation in scholarly communication activities on their campuses. This week, while trying to find the best hook for posting about the ‘sting operation’ conducted on a selection of open access journals recently reported in the journal Science, I landed on Mike Taylor’s October 7, 2013 blog post, “Anti-tutorial: how to design and execute a really bad study.”

The blog-o and Twitter-spheres have over the last four days offered extensive reporting and analysis of the article that appeared in the October 4, 2013 issue of Science. If you are one of a handful of persons who by now has not heard about this story the gist is this: The author of the article, John Bohannon, a biologist and science journalist, designed a fake research paper with obvious methodological and scientific errors and sent out permutations of it using the names of fictitious authors from fictitious African institutions to 304 open access journals over an eight month period from January to August 2013. Bohannon reports his results thusly: “By the time Science went to press, 157 of the journals had accepted the paper and 98 had rejected it.” Based upon these incredible acceptance rates of an obviously faked paper, the article appears to level a clear and damning indictment upon the peer review processes and/or ethical practices of open access journals, especially those that levy article processing charges (APCs)—which Bohannon calls the “standard open-access ‘gold’ model”—as a condition of publication. Near the top of the article, Bohannon characterizes his findings as “reveal[ing] the contours of an emerging Wild West in academic publishing.”

As persons began to dig into Bohannon’s data it became apparent that many of the journals to which he sent his paper were known predatory journals, predisposing the outcome. Too, objections were raised about Bohannon’s methodology including the fact that he did not also send his fake paper to traditional toll-access journals, which might have offered a basis of comparison on acceptance/rejection rates. Further, is it really true that the levying of APCs constitutes the “standard open-access ‘gold’ model”? Despite some interesting findings—such as the fact that many open access journals not only rejected the paper but also raised serious ethical concerns (e.g., PLOS ONE), while several journals owned by large commercial publishers such as Elsevier, SAGE, and Wolters Klewer accepted the paper—speculations are beginning to surface about the real purpose of this article, including why Science (a well-respected toll-access journal) agreed to publish it in the first place, given its data and methodological problems.

No one is disputing the problem of predatory open access journals, and all agree that publishers of these journals should be put out of business. But at the end of the day, there isn’t really much new offered in the article. Indeed, apart from the research element, it has a similar tone as a piece run last April in The New York Times, down to the use of the phrase ‘Wild West,’ and the fact that traditional toll-access journals were not scrutinized. (I wrote about that article here.) Is this just another attempt to discredit, or at least cast doubt on the credibility and viability of open access as a publishing model?

“Truthiness” and “sciencey”

Assuming the speculation is correct—that there is still a need and a strong desire to discredit open access—how might this be accomplished? Enter Mike Taylor’s “anti-tutorial.” I commend it to your reading. (Please remember that it’s satire.) [Apologies: It occurred to me after originally posting that I neglected to identify Dr. Michael Taylor as a paleontologist in the Department of Earth Sciences at the University of Bristol, UK.]

Oh, I can’t finish this post without mentioning this most delightful excerpt from Taylor’s piece:

[As a molecular biologist with a Ph.D., John Bohannon] does know what science looks like, and he’s made the “sting” operation look like it. It has that sciencey quality. It discusses methods. It has supplementary information. It talks a lot about peer-review, that staple of science. But none of that makes it science.

“It has that sciencey quality.” I love that word sciencey! When I read it, it reminded me immediately of comedian Stephen Colbert’s use of the word “truthiness” in a segment of his political satire television show The Colbert Report back in October 2005. The Wikipedia article on “truthiness” cites a quote from Colbert about his use of the word: “We’re not talking about truth, we’re talking about something that seems like truth—the truth we want to exist.” Indeed, the Wikipedia article cites a definition of truthiness “as a quality characterizing a ‘truth’ that a person making an argument or assertion claims to know intuitively ‘from the gut’ or because it ‘feels’ right without regard to evidence, logic, intellectual examination, or facts” (emphasis added).

Maybe when we use “sciencey” we’re talking about something that seems like science. But I definitely hope it’s not the science we want to exist. In the end, John Bohannon’s article doesn’t really help us to evaluate the status of open access because it is flawed science. Publishing it in Science doesn’t make it any better. Thanks Mike, for an insightful and entertaining tutorial.

P.S. I discovered that although obsolete or rare, both “truthiness” and “sciencey” are real words, with entries in The Oxford English Dictionary (I consulted the 1933 edition).

About these ads
Posted in "The Hat Tip", Book/Article Review, Commercial Publishing, Economics & Business Models, Open Access, Peer Review, Scholarly Journals
9 comments on ““Truthiness” isn’t quite truth, and “sciencey” isn’t quite science, even if published in Science: Mike Taylor’s “Anti-tutorial: how to design and execute a really bad study”
  1. Maria Hrynkiewicz / Versita says:

    Dear Gary,

    I am pleased to report, that the infamous article was also submitted to our (Versita) journal ”Natural Products Against Cancer”…and was promptly rejected. Thought, you might be interested. I was pleased to have it found out.

    Best regards,


  2. Mike Taylor says:

    I actually had Colbert’s “truthiness” in mind when I coined “sciencey”.

  3. Kevin Smith says:

    About a year ago there was a discussion on the ATLA’s [American Theological Library Association] ATLANTIS list about the journal Missiology, the price of which increased by over 300% overnight when it was bought up by Sage. To me, this is clearly abusive behavior, and it is a kind of abuse that directly threatens our core mission, since accepting the Missiology price increase means not subscribing to something else or buying fewer monographs. So if we want to punish abusers, our first step should be cancelling subscriptions to Missiology and journals that have had similar sudden and dramatic price increases (which, by their very scale and abruptness cannot represent real increases in the cost of publishing).

    Targeted cancellations aimed at improving our overall ability to spend the resources of our institutions responsibly seems like a more productive step than being sure that our catalogs do not link to the Journal of Natural Pharmaceuticals, which is the “bad” journal featured in the opening paragraphs of the Science article, for two reasons. First, most of those journals are unlikely to be linked in our catalogs anyway. At Duke, we use an API from the Directory of Open Access Journals to link to thousands of OA journals, but “Natural Pharmaceuticals” is not one of them. This must be a journal title that the article author got from Beall’s list, which means it was already a known problem and few libraries would have been treating it seriously even before Bohannon’s deeply flawed and slanted study was published.

    The other reason why it is more important to cancel journals like Missiology than it is remove links to OA journals is that the abusive behavior of the former is much more threatening to our long-term ability to do our jobs. By raising its price by 300%, Missiology directly compromised our ability to provide resources to our faculty and students, whereas a link to the Journal of Natural Pharmaceuticals is probably useless but does no active harm.

    NOTE: I was about to post this comment when I decided to verify the truth of what I had said above about Duke’s catalog. While it is true that the Journal of Natural Pharmaceuticals is not listed in the DOAJ, it turns out that we do link to it in our catalog. Why do we do so? Because it is listed and linked in both Academic OneFile and Academic Search Complete. That fact alone–the fact that prominent databases that all of us subscribe to index at least some of these allegedly “predatory” journals — should further complicate our notion of what constitutes a “known abuser.”

  4. […] Peter Suber | Björn Brems | Kausik Datta | Michael Eisen | Martin Paul Eve | The Economist | OASPA | Scholarly Kitchen | Retraction Watch | John Hawks | Guardian | NPR | Inside HigherED | LA Times | Curt Rice | Der Spiegel | Chronicle HE | Ernesto Priego | Gunther Eisenbach | Fabiana Kubke | Zen Faulkes | Mike Taylor | Libération | Rue89 | Le Monde | Nu Wetenschap | Nieuwsblad.be | Independent | Toronto Star | Lenny Teytelman | Tagespiegel | McBlawg | Jason Koebler | Salon.com | Bradley Fikes | Michael Schoon | SPARC | Paula Callan, ABC Science | Tracey Brown | Ernesto Priego, id. on LSEimpact blog | Steven Novella | Dan Vergano at Nat. Geographic | Stevan Harnad | MSN nl | The Telegraph India | Jacob Kastrenakes on The Verge | Michelle Meyer (Harvard Law) | Brian Wood (comparison with impact factors) | Bob O’Hara in The Guardian | Philip Moriarty at physics focus | Ulrich Herb | Gavia Libraria (on alternatives) | Gary Daught […]

  5. I was under the impression that Feynman already had a term for this: cargo-cult science, but sciencey is more punchy!

  6. […] fel och den försöker beskriva något som en sanning (truth) men blir istället ett exempel i “truthiness” som Gary F. Daught beskriver det. “Truthiness” definieras i Wikipedia som “as a quality characterizing a […]

  7. […] because of its flaws and that it is trying to describe something as the truth but instead gives  “truthiness”, as Gary F. Daught describes it, a face. “Truthiness” is defined in Wikipedia  as “a quality characterizing a […]

  8. […] the name Michael Taylor sounds familiar, it may be because, coincidentally, I previously posted about a piece he wrote on his blog about the John Bohannon/Science open access ‘sting’. […]

  9. […] however briefly, farther forward in the consciousness of the general public; the responses from Gary Daught andHeather Joseph are well-crafted and do not completely dismiss the concerns raised, while […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Enter your email address to follow this blog and receive notifications of new posts by email.

Blog Stats
  • 33,245 hits

Get every new post delivered to your Inbox.

Join 288 other followers

%d bloggers like this: