Journal Article

Effect of revealing authors’ conflicts of interests in peer review: randomized controlled trial

John, Leslie K, Callaham, Michael L, Loewenstein, George, Marder, Andrew

2019

Overview

AbstractObjectiveTo assess the effect of disclosing authors’ conflict of interest declarations to peer reviewers at a medical journal.DesignRandomized controlled trial.SettingManuscript review process at the Annals of Emergency Medicine.ParticipantsReviewers (n=838) who reviewed manuscripts submitted between 2 June 2014 and 23 January 2018 inclusive (n=1480 manuscripts).InterventionReviewers were randomized to either receive (treatment) or not receive (control) authors’ full International Committee of Medical Journal Editors format conflict of interest disclosures before reviewing manuscripts. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain “counterfactual scores”—that is, the scores they believed they would have given had they been assigned to the opposite arm—as well as attitudes toward conflicts of interest.Main outcome measureOverall quality score that reviewers assigned to the manuscript on submitting their review (1 to 5 scale). Secondary outcomes were scores the reviewers submitted for the seven more specific quality ratings and counterfactual scores elicited in the follow-up survey.ResultsProviding authors’ conflict of interest disclosures did not affect reviewers’ mean ratings of manuscript quality (Mcontrol=2.70 (SD 1.11) out of 5; Mtreatment=2.74 (1.13) out of 5; mean difference 0.04, 95% confidence interval –0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol= 2.85 (1.12) out of 5; Mtreatment=2.96 (1.16) out of 5; mean difference 0.11, –0.05 to 0.26). Similarly, no effect of the treatment was seen on any of the other seven quality ratings that the reviewers assigned. Reviewers acknowledged conflicts of interest as an important matter and believed that they could correct for them when they were disclosed. However, their counterfactual scores did not differ from actual scores (Mactual=2.69; Mcounterfactual=2.67; difference in means 0.02, 0.01 to 0.02). When conflicts were reported, a comparison of different source types (for example, government, for-profit corporation) found no difference in effect.ConclusionsCurrent ethical standards require disclosure of conflicts of interest for all scientific reports. As currently implemented, this practice had no effect on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.

• Publisher: BMJ Publishing Group LTD,BMJ Publishing Group Ltd
• Subject: Adult / Bibliometrics / Biomedical research / Clinical trials / Conflict of Interest / Credibility / Disclosure / Disclosure - ethics / Disclosure - standards / Emergency medical care / Evidence-based medicine / Female / Humans / Male / Medical research / Medical Writing - standards / Middle Aged / Peer review / Peer Review, Research - ethics / Peer Review, Research - standards / Perceptions / Quality / Quality Control / Research Personnel - ethics / Research Personnel - standards / Researchers / Reviews