Journal Article

Systematic review of current guideline appraisals performed with the Appraisal of Guidelines for Research & Evaluation II instrument—a third of AGREE II users apply a cut-off for guideline quality

Eikermann, Michaela, Hoffmann-Eßer, Wiebke, Lampert, Ulrike, Siering, Ulrich, Neugebauer, Edmund A.M.

2018

Overview

To investigate whether Appraisal of Guidelines for Research & Evaluation (AGREE) II users apply a cut-off based on standardized domain scores or overall guideline quality to distinguish between high- and low-quality guidelines, as well as to investigate which criteria they use to generate this cut-off and which type of cut-off they apply. We conducted a systematic search in MEDLINE, EMBASE, DARE, and the HTA-database for German- and English-language studies appraising guidelines with AGREE II. Information on cut-offs was extracted and analyzed descriptively. We identified 118 relevant publications. Thirty-nine (33%) used a cut-off, of which 24 (62%) used a 2-step and 13 (33%) used a 3-step approach. The cut-off for high quality lay between 50% and 70% (2-step) and 60% and 83% (3-step) of the highest possible rating. Twenty-four (62%) publications applied a cut-off based on standardized domain scores and 7 (18%) based on overall guideline quality. Eleven (28%) applied cut-offs to derive the recommendation for guideline use. A third of AGREE II users apply a cut-off to distinguish between high- and low-quality guidelines, often without clearly describing how the cut-off is generated. Many users might welcome a clear distinction between high- and low-quality guidelines; specifying a cut-off for this purpose might be useful. •In contrast to the specification of Appraisal of Guidelines for Research & Evaluation (AGREE)-II, a third of AGREE II users apply a cut-off to distinguish between high- and low-quality guidelines, often without clearly describing how the cut-off is generated.•Users in our analysis applied a 2- or 3-step cut-off.•Some of the AGREE II users who applied a cut-off mentioned the rationale for applying it.•In some cases, recommendation for use was derived from the cut-off based ratings of domains or overall guideline quality.•AGREE II users might welcome a clear distinction between high- and low-quality guidelines; specifying a cut-off for this purpose might be a useful approach.•Need of standardization of a procedure and its specification in AGREE II.

• Publisher: Elsevier Inc,Elsevier Limited
• Subject: AGREE instrument / Antidepressants / Appraisals / Clinical / Clinical decision making / Clinical medicine / Clinical practice guidelines / Databases, Factual / Decision making / Documents / Epidemiologic Methods / Epidemiology / Guidelines / Guidelines as Topic - standards / Humans / Liver cancer / Methodological guideline appraisal / Methodological quality / practice guidelines / practice guidelines as topic / Quality / Recommender systems / Studies / Systematic review / Systematic reviews