Asset Details
Do you wish to reserve the book?
Risk of bias in nonrandomized studies of interventions showed low inter-rater reliability and challenges in its application
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
Risk of bias in nonrandomized studies of interventions showed low inter-rater reliability and challenges in its application
Do you wish to request the book?
Risk of bias in nonrandomized studies of interventions showed low inter-rater reliability and challenges in its application
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
Risk of bias in nonrandomized studies of interventions showed low inter-rater reliability and challenges in its application
2019
Overview
To assess the inter-rater reliability (IRR) and usability of the risk of bias in nonrandomized studies of interventions tool (ROBINS-I).
We designed a cross-sectional study. Five raters independently applied ROBINS-I to the nonrandomized cohort studies in three systematic reviews on vaccines, opiate abuse, and rehabilitation. We calculated Fleiss' Kappa for multiple raters as a measure of IRR and discussed the application of ROBINS-I to identify difficulties and possible reasons for disagreement.
Thirty one studies were included (195 evaluations). IRRs were slight for overall judgment (IRR 0.06, 95% CI 0.001 to 0.12) and individual domains (from 0.04, 95% CI −0.04 to 0.12 for the domain “selection of reported results” to 0.18, 95% CI 0.10 to 0.26 for the domain “deviation from intended interventions”). Mean time to apply the tool was 27.8 minutes (SD 12.6) per study. The main difficulties were due to poor reporting of primary studies, misunderstanding of the question, translation of questions into a final judgment, and incomplete guidance.
We found ROBINS-I difficult and demanding, even for raters with substantial expertise in systematic reviews. Calibration exercises and intensive training before its application are needed to improve reliability.
Publisher
Elsevier Inc,Elsevier Limited
