Stop this waste of people, animals and money

NATURE | COMMENT Sharing http://www.nature.com/news/stop-this-waste-of-people-animals-and-money-1.22554 David Moher, Larissa Shamseer, Kelly D. Cobey, Manoj M. Lalu, James Galipeau, Marc T. Avey, Nadera Ahmadzai, Mostafa Alabousi, Pauline Barbeau, Andrew Beck, Raymond Daniel, Robert Frank, Mona Ghannad, Candyce Hamel, Mona Hersi, Brian Hutton, Inga Isupov, Trevor A. McGrath, Matthew D. F. McInnes, Matthew J. Page, Misty Pratt, Kusala Pussegoda, Beverley Shea, Anubhav Srivastava, Adrienne Stevenset al. 06 September 2017 Predatory journals have shoddy reporting and include papers from wealthy nations, find David Moher, Larissa Shamseer, Kelly Cobey and colleagues. Illustration by David Parkins Predatory journals are easy to please. They seem to accept papers with little regard for quality, at a fraction of the cost charged by mainstream open-access journals. These supposedly scholarly publishing entities are murky operations, making money by collecting fees while failing to deliver on their claims of being open access and failing to provide services such as peer review and archiving. LISTEN Adam Levy takes a look inside predatory journals 00:00 Despite abundant evidence that the bar is low, not much is known about who publishes in this shady realm, and what the papers are like. Common wisdom assumes that the hazard of predatory publishing is restricted mainly to the developing world. In one famous sting, a journalist for Science sent a purposely flawed paper to 140 presumed predatory titles (and to a roughly equal number of other open-access titles), pretending to be a biologist based in African capital cities1. At least two earlier, smaller surveys found that most authors were in India or elsewhere in Asia2, 3. A campaign to warn scholars about predatory journals has concentrated its efforts in Africa, China, India, the Middle East and Russia. Frequent, aggressive solicitations from predatory publishers are generally considered merely a nuisance for scientists from rich countries, not a threat to scholarly integrity. Our evidence disputes this view. We spent 12 months rigorously characterizing nearly 2,000 biomedical articles from more than 200 journals thought likely to be predatory. More than half of the corresponding authors hailed from high- and upper-middle-income countries as defined by the World Bank. Of the 17% of sampled articles that reported a funding source, the most frequently named funder was the US National Institutes of Health (NIH). The United States produced more articles in our sample than all other countries save India. Harvard University (with 9 articles) in Cambridge, Massachusetts, and the University of Texas (with 11 articles across all campuses) were among the eight institutions with the most articles. It is easy to imagine other, similar institutions coming up in a different sample. The point is, the problem of predatory journals is more urgent than many realize. Related stories Illegitimate journals scam even senior scientists Predatory journals recruit fake editor Controversial website that lists ‘predatory’ publishers shuts down More related stories Articles in our sample consistently failed to report key information necessary for readers to assess, reproduce and build on the findings. Fewer than 10% of studies claiming to be randomized controlled trials described how patients were allocated to treatment groups; where blinding was possible, fewer than one-quarter noted whether patients and outcome assessors were blinded to group assignment. Whether authors are being duped or are overzealously seeking to lengthen their publication lists, this represents enormous waste. Just the subset of articles that we examined contained data from more than 2 million individuals and over 8,000 animals. By extrapolation, we estimate that at least 18,000 funded biomedical-research studies are tucked away in poorly indexed, scientifically questionable journals. Little of this work will advance science. It is too dodgily reported (and possibly badly conducted) and too hard to find. In our view, publishing in predatory journals is unethical. Individuals who agree to be studied expect that their participation could benefit future patients. Use of animals in biomedical research is rationalized on the assumption that experiments will contribute valuable information. Even assuming authors are publishing more than one paper from their study (and some are), they should be held to a higher standard of disclosure. Publishers, funders and research institutions must join together to prevent research from ending up in predatory journals. What we did We drew our sample from the journals and publishers whose status as predatory was deemed as “potential, possible, or probable” by librarian Jeffrey Beall of the University of Colorado, Denver. (These controversial compilations were taken offline early in 2017, but remain available in web archives and are one of the few tools that researchers have to investigate illegitimate journals.) “In our view, publishing in predatory journals is unethical.” We took a random subset of 185 publishers and obtained lists of their journals. At least two people, working independently, assessed whether each journal met Medline's selection criteria as having a biomedical scope. We randomly selected 200 journals from this list; we also included 45 biomedical standalone journals from a set that had been developed similarly for another study4. In February 2016, we visited each journal's website and downloaded copies of up to 25 articles, starting with the ones most recently published. The total number of articles we obtained came to 3,702, because many journals listed fewer than 25 articles. Of those, 1,907 reported primary biomedical research or systematic reviews and so were included in our analysis. This left us with 41 single-journal publishers and 179 titles from 51 multi-journal publishers (see Supplementary information). Sloppy work We examined each paper in light of reporting guidelines relevant for each type of study. For example, for randomized controlled trials, we cross-checked articles using a modified Consolidated Standards of Reporting Trials (CONSORT) checklist. Although adherence to and enforcement of guidelines is patchy even in mainstream publications, reporting quality in our sample was much worse. Articles were particularly deficient in descriptions of study methods, results and — for clinical trials and systematic reviews — study registration. Of the 94 randomized controlled trials that we examined, most items in CONSORT were reported 40% of the time. Fewer than 14% of trials gave a registration number or registry name. Yet, a study of mainstream journals in the Netherlands found registration information in at least 60% of trials5. Of the 21 systematic reviews in our sample, only two reported assessing the risk of bias. Yet 70% did so in an evaluation of 300 Medline-indexed systematic reviews6. Even for animal studies, for which reporting in mainstream journals is remarkably poor, we found that performance of predatory journals was much worse; for instance, just 3% of 201 relevant predatory articles reported blinding. A separate study found that blinding is recorded in 20% of articles in PLoS journals and in 21% of articles in Nature journals7. Of the articles in our sample that evaluated humans or whole animals, only 40% noted that they received approval from an ethics committee. Previous studies show that ethics-committee approval is mentioned in more than 90% of animal7 and 70% of human8 studies published in mainstream journals. Global problem Nearly three-quarters (1,397) of the publications we examined did not report information about funding; 10% stated that they were not funded. The remaining 323 articles named 345 different funders, mainly academic institutions (124) and government agencies (122). For 1,907 papers, corresponding authors came from 103 countries, including India (27%), the United States (15%), Nigeria (5%), Iran (4%) and Japan (4%) (see 'Global predation'). These figures should be interpreted in the context of total scientific output per nation. According to tallies in the academic databases Scopus and PubMed, the United States produced about 5 times as many biomedical articles as India last year, and 80 times as many as Nigeria. An analysis of general academic articles from 2013 to 2015 in Scopus found that 10% or more from India and Nigeria were in predatory journals, as compared to less than 1% from Japan and the United States9. Source: Study Reporting in Predatory Journals Group Researchers at universities in Indiana also looked at who has published in predatory pharmaceutical journals in 2013 (ref. 2). They compared authors who published in seven predatory titles on Beall's list with those in five open-access journals that rejected the journalist's bait in the sting operation1. Sixty-five per cent of authors of predatory-journal articles had never published before, as compared to 19% of those in vetted open-access journals. In that study, 75% of all authors in predatory journals were from South Asia (mainly India), 14% from Africa (mainly Nigeria) and only 3% from North America. By contrast, 57% of authors in our sample are from higher-income or upper-middle-income countries. This could be because our sampling looked mainly at corresponding authors rather than all authors, or because it considered a broader swath of articles, titles and years. It is possible that our sample included some journals particularly popular with high-income-country authors. Or, perhaps, predatory journals have stepped up their aggressive e-mails in richer countries. Corresponding authors in our survey named 1,291 institutions as primary affiliations; 15 did not name any. We contacted 16 vice-presidents (or the senior administrative person) of research at some of the top institutions whose researchers were publishing in predatory journals. Our e-mail to Bangalore Medical College and Research Institute bounced back. Three institutions provided feedback; one (Manipal University, India, 15 papers) detailed an intervention launched earlier this year, and provided data that the effort reduced the number of articles published in presumed predatory journals. Responses from the University of Benin in Nigeria (8 papers) and Menoufia University in Egypt (8 papers) said that they warned against or made lists of illegitimate publishers for researchers to consult. The Mayo Clinic in Rochester, Minnesota (7 papers), sent a note to Nature that articles in predatory journals are not considered for academic advancement. D. Y. Patil University in India, which, with 20 papers, had the most in our sample, did not reply. Nor did the University of Tehran, which, with 14 papers from 14 authors, tied with D. Y. Patil University for the most unique authors. We also attempted to contact corresponding authors at some of the leading institutions. Fifteen articles — including all 9 at Bangalore Medical College and Research Institute — did not include author e-mails. In total, we sent e-mails to 87 authors (who had collectively written 119 of the articles in our sample). Three bounced back. Of the 18 replies we received, 3 asked for educational material about predatory journals. Only two said that they were aware of the journal's categorization as potentially predatory, and only four were aware of Beall's list. Similarly, 90% of 1,088 Italian authors surveyed in 2016 stated that they were unaware of Beall's list (see go.nature.com/2ixzvqm). Only three people who responded to us had submitted their manuscript to other publications before its acceptance in the predatory journal. Seven indicated receiving guidance of some form on where to submit; an equal number said that their work had been cited. This suggests that authors are not using these titles as a last resort: the scientific community needs a better understanding of what precisely makes predatory journals attractive. Although, collectively, the articles we examined were atrocious in terms of reporting, we did not examine the performance of individual titles. We should note that several publishers have protested against their inclusion on Beall's list. Also, the term 'predator' in particular is questioned, because on occasion it is hard to tell whether a journal is simply inept or disdaining research integrity and scientific robustness to pursue profit. And rather than being prey, some authors may purposely seek out low-barrier ways to publish. Some may argue that authors in higher-income countries publish in the 'best' predatory journals. Our study was not set up to resolve this question; we did examine reporting of randomization and allocation across the 94 clinical trials, and found no statistical differences between rich-world corresponding authors and those from elsewhere. It is nearly impossible for prospective authors to differentiate predatory journals by metrics. For example, the Journal of Surgery from Avens Publishing Group — the title most favoured by US authors in our sample — does not have easily identifiable metrics that distinguish it from non-predatory journals. We contacted the editor-in-chief of the journal, who replied that he hadn't heard of predatory journals but that the journal sends manuscripts to peer reviewers and has rejected manuscripts on the basis of their merit. (Note that at least two other questionable journals carry the same title.) Our experience with these journals is that they provide both poor vetting and poor access. Their websites and archiving systems are unstable. Although some articles appear in PubMed (often after a delay), the titles are not indexed by Medline and are difficult to find. Indeed, some titles in our sample, such as the International Journal of Pharmaceutical and Medical Research, ask authors to assign copyright to the journal, which is against the mores of open access. Even without Beall's list, savvy authors should know when to suspect that a journal is predatory. Our research group has identified 13 characteristics of predatory journals; these include low article-processing fees (less than US$150); spelling and grammar errors on the website; an overly broad scope; language that targets authors rather than readers; promises of rapid publication; and a lack of information about retraction policies, manuscript handling or digital preservation. Manuscript submissions by e-mail and the inclusion of distorted images are also common4. However, predatory journals are becoming increasingly adept at appearing legitimate, and little is being done to warn authors away from them. Just one of the ten most common funders reported in our study, the University Grants Commission, India, provides guidance about journal selection on its website. Stop the rot We believe that publishers, research institutions and funders should issue explicit warnings against illegitimate publishers and develop cohesive recommendations on publication integrity together. Funders and research institutions should increase the funds that they make available towards open-access publication; prohibit the use of funds to support predatory journal publications; make sure that researchers are trained in how to select appropriate journals when submitting their work; and audit where grantees, faculty members and research staff publish. When seeking promotion or funding, researchers should include a declaration that their CV is free of predatory publications. Publication lists could be checked against lists such as the Directory of Open Access Journals (DOAJ) or the Journal Citation Reports. Developing automated tools to facilitate the proposed audits would also be valuable. Before approving a study, ethics committees should ask researchers to declare in writing their willingness to work with their institutional resources, such as librarians, to ensure they do not submit to any journals without reviewing evidence-based criteria for avoiding these titles. If not, predatory journals will continue to erode the integrity of scientific scholarship. Substandard publications have permeated authentic electronic databases. A problem largely unknown a decade ago, there are now a roughly estimated 8,000 predatory titles that collectively 'publish' more than 400,000 items a year3. We need to cut off the supply of manuscripts to these illegitimate outfits. Nature 549, 23–25 (07 September 2017) doi:10.1038/549023a References Bohannon, J. Science 342, 60–65 (2013). Show context Xia, J. et al. J. Assoc. Infor. Sci. Tech. 66, 1406–1417 (2015). Show context Shen, C. & Björk, B.-C. BMC Med. 13, 230 (2015). Show context Shamseer, L. et al. BMC Med. 15, 28 (2017). Show context van de Wetering, F. T., Scholten, R. J. P. M., Haring, T. & Hooft, L. PLoS ONE 7, e49599 (2012). Show context Page, M. J. et al. PLOS Med. 13, e1002028 (2016). Show context Baker, D., Lidster, K., Sottomayor, A. & Amor, S. PLoS Biol. 12, e1001756 (2014). Show context Taljaard, M. et al. Br. Med. J. 342, d2496 (2011). Show context Machacek, V. & Srholec, M. Predatory journals in Scopus (IDEA, 2017); available at http://go.nature.com/2wd3es7 Show context Related stories and links From nature.com Illegitimate journals scam even senior scientists 06 September 2017 Predatory journals recruit fake editor 22 March 2017 Controversial website that lists ‘predatory’ publishers shuts down 18 January 2017 Open-access index delists thousands of journals 09 May 2016 Author information Affiliations David Moher is a clinical epidemiologist at the Ottawa Hospital Research Institute, Ontario, Canada, and part of the Study Reporting in Predatory Journals Group. Larissa Shamseer is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Kelly D. Cobey is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; the School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada; and the School of Natural Sciences, Department of Psychology, University of Stirling, Stirling, Scotland. Manoj M. Lalu is in the Centre for Journalology, Clinical Epidemiology Program, and the Regenerative Medicine Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, and the Department of Anesthesiology and Pain Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. James Galipeau is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Marc T. Avey is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Nadera Ahmadzai is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Mostafa Alabousi is in Undergraduate Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Pauline Barbeau is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Andrew Beck is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Raymond Daniel is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Robert Frank is in Undergraduate Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Mona Ghannad is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; the Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Centre, University of Amsterdam, Amsterdam, the Netherlands; and the Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS-UMR1153) INSERM, Université Paris Descartes, Centre d'épidémiologie Clinique, Paris, France. Candyce Hamel is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Mona Hersi is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Brian Hutton is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Inga Isupov is in the Department of Radiology, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Trevor A. McGrath is in Undergraduate Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Matthew D. F. McInnes is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, and the Department of Radiology, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Matthew J. Page is in the School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia; and the School of Social and Community Medicine, University of Bristol, Bristol, UK. Misty Pratt is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Kusala Pussegoda is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Beverley Shea is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Anubhav Srivastava is in the David R. Cheriton School of Computer Science, Faculty of Mathematics, University of Waterloo, Waterloo, Canada. Adrienne Stevens is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the Translational Research in Biomedicine (TRIBE) Graduate Program, University of Split School of Medicine, Split, Croatia. Kednapa Thavorn is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada; and the School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Sasha van Katwyk is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Roxanne Ward is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Dianna Wolfe is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Fatemeh Yazdi is in the Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. Ashley M. Yu is in Undergraduate Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Hedyeh Ziai is in Undergraduate Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Canada. Corresponding author Correspondence to: David Moher Supplementary information PDF files Assessing the scientific content of predatory journals (1.5M)