Explore the vast range of titles available.

BackgroundSubstantial amounts of public funds are invested in health research worldwide. Publicly funded randomised controlled trials (RCTs) often recruit participants at a slower than anticipated rate. Many trials fail to reach their planned sample size within the envisaged trial timescale and trial funding envelope.ObjectivesTo review the consent, recruitment and retention rates for single and multicentre randomised control trials funded and published by the UK's National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme.Data sources and study selectionHTA reports of individually randomised single or multicentre RCTs published from the start of 2004 to the end of April 2016 were reviewed.Data extractionInformation was extracted, relating to the trial characteristics, sample size, recruitment and retention by two independent reviewers.Main outcome measuresTarget sample size and whether it was achieved; recruitment rates (number of participants recruited per centre per month) and retention rates (randomised participants retained and assessed with valid primary outcome data).ResultsThis review identified 151 individually RCTs from 787 NIHR HTA reports. The final recruitment target sample size was achieved in 56% (85/151) of the RCTs and more than 80% of the final target sample size was achieved for 79% of the RCTs (119/151). The median recruitment rate (participants per centre per month) was found to be 0.92 (IQR 0.43–2.79) and the median retention rate (proportion of participants with valid primary outcome data at follow-up) was estimated at 89% (IQR 79–97%).ConclusionsThere is considerable variation in the consent, recruitment and retention rates in publicly funded RCTs. Investigators should bear this in mind at the planning stage of their study and not be overly optimistic about their recruitment projections.

GRADE and other evidence to decision (EtD) frameworks are widely used by guideline development groups (GDG) and other decision-makers. When GDGs judge the magnitude of desirable and undesirable health outcomes on EtDs, they typically categorize them as trivial, small, moderate, or large. However, generic judgment or decision thresholds (DTs) that could guide the user about such estimates of effect size or serve as references for interpretation of findings are not yet available. The objective of this study was to empirically derive DTs for EtD judgments about the magnitude of dichotomously assessed health benefits and harms. We conducted a methodological randomized controlled trial to derive empirical DTs across conditions and health outcomes. We invited stakeholders, including clinicians, epidemiologists, decision scientists, health research methodologists, experts in health technology assessment (HTA), members of GDGs, patient representatives, and the public to participate in the trial. We employed randomly assigned case scenarios to elicit ranges of absolute risk differences judged as small and moderate effects from study participants. We then used the collected data to derive empirical DTs. We also investigated the validity of our DTs by measuring the agreement between judgments that were made by GDGs in the past and the judgments that our DTs approach would suggest if applied to the same guideline data. A total of 445 stakeholders accessed the survey of which 409 were randomised and 288 rated at least one case scenario. Based on these participants, the study findings support our a priori hypothesis of a difference in the DTs for trivial, small, moderate, and large effects and are suggestive of a relation between raters' judgments and the joint measure of absolute effects and outcome values. The results permit the use and calculation of DTs for a variety of scenarios and we present three ways of how to use the results practically. In this trial we confirmed that empirically derived DTs discriminate between judgments on the EtDs. These DTs can be used for judgments about desirable and undesirable health effects in systematic reviews or to initiate and inform a discussion with a GDG. This ensures consistency in judgments across different guideline questions and promotes transparency in judgments. Decision thresholds (DTs) help with determining if effects of interventions should be considered absent, small, moderate or large. In this study we derived an overarching approach for these thresholds across conditions and outcomes. The results of this study, a randomized experiment, will help guideline developers and other decision-makers to make these judgments objectively. They will be particularly relevant for the use in Grading of Recommendations Assessment, Development, and Evaluation (GRADE) evidence to decision (EtD) frameworks.

Background When designing a randomised controlled trial (RCT), an important consideration is the sample size required. This is calculated from several components; one of which is the target difference. This study aims to review the currently reported methods of elicitation of the target difference as well as to quantify the target differences used in Health Technology Assessment (HTA)-funded trials. Methods Trials were identified from the National Institute of Health Research Health Technology Assessment journal. A total of 177 RCTs published between 2006 and 2016 were assessed for eligibility. Eligibility was established by the design of the trial and the quality of data available. The trial designs were parallel-group, superiority RCTs with a continuous primary endpoint. Data were extracted and the standardised anticipated and observed effect size estimates were calculated. Exclusion criteria was based on trials not providing enough detail in the sample size calculation and results, and trials not being of parallel-group, superiority design. Results A total of 107 RCTs were included in the study from 102 reports. The most commonly reported method for effect size derivation was a review of evidence and use of previous research (52.3%). This was common across all clinical areas. The median standardised target effect size was 0.30 (interquartile range: 0.20–0.38), with the median standardised observed effect size 0.11 (IQR 0.05–0.29). The maximum anticipated and observed effect sizes were 0.76 and 1.18, respectively. Only two trials had anticipated target values above 0.60. Conclusion The most commonly reported method of elicitation of the target effect size is previous published research. The average target effect size was 0.3. A clear distinction between the target difference and the minimum clinically important difference is recommended when designing a trial. Transparent explanation of target difference elicitation is advised, with multiple methods including a review of evidence and opinion-seeking advised as the more optimal methods for effect size quantification.

BackgroundIn the evolving healthcare landscape of the United Arab Emirates (UAE), establishing cost-effectiveness thresholds (CETs) is pivotal to informing decision-makers about the value of health technologies.ObjectiveThis study aimed to establish CETs for the UAE that harmonise with international standards while reflecting the nation’s unique healthcare needs and economic context.SettingUnited Arab Emirates.MethodsA multitiered methodology was employed, involving a literature review, a panel of national experts and workshops with key stakeholders, including healthcare providers, government health departments and healthcare payers. The panel and workshops were integral in assessing global CET practices and their applicability to the UAE providing a preliminary framework for CET in the UAE. Structured voting sessions were then conducted allowing voting on crucial aspects of CET to determine the baseline threshold, multipliers for severity, rarity and health gain, and methodologies for quantifying disease severity.ResultsCETs were linked to the economic status of the UAE, with a baseline threshold of 0.75 times the gross domestic product per capita for one quality-adjusted life year gained. A multiplier system was introduced to reflect societal views on disease severity, disease rarity and the relative health benefit of health technologies. Based on the voting results, disease rarity was deemed the most crucial factor, receiving a maximum multiplier of 3X, while severity and health gain were assigned a maximum of 2X. The multiplier values for both disease severity and relative health gain would be determined on a continuous scale. The proportional or relative shortfall method would be used to assess disease severity.ConclusionsThe proposed CET framework for the UAE will be dependent on local generation of cost-effectiveness evidence. Periodic review of CETs based on initial experiences ensures the responsiveness of policymakers to the changing healthcare and economic environment.

Background Treatment switching is common in randomised trials of oncology treatments, with control group patients switching onto the experimental treatment during follow-up. This distorts an intention-to-treat comparison of the treatments under investigation. Two-stage estimation (TSE) can be used to estimate counterfactual survival times for patients who switch treatments – that is, survival times that would have been observed in the absence of switching. However, when switchers do not die during the study, counterfactual censoring times are estimated, inducing informative censoring. Re-censoring is usually applied alongside TSE to resolve this problem, but results in lost longer-term information – a major concern if the objective is to estimate long-term treatment effects, as is usually the case in health technology assessment. Inverse probability of censoring weights (IPCW) represents an alternative technique for addressing informative censoring but has not before been combined with TSE. We aim to determine whether combining TSE with IPCW (TSEipcw) represents a valid alternative to re-censoring. Methods We conducted a simulation study to compare TSEipcw to TSE with and without re-censoring. We simulated 48 scenarios where control group patients could switch onto the experimental treatment, with switching affected by prognosis. We investigated various switching proportions, treatment effects, survival function shapes, disease severities and switcher prognoses. We assessed the alternative TSE applications according to their estimation of control group restricted mean survival (RMST) that would have been observed in the absence of switching up to the end of trial follow-up. Results TSEipcw performed well when its weights had a low coefficient of variation, but performed poorly when the coefficient of variation was high. Re-censored analyses usually under-estimated control group RMST, whereas non-re-censored analyses usually produced over-estimates, with bias more serious when the treatment effect was high. In scenarios where TSEipcw performed well, it produced low bias that was often between the two extremes associated with the re-censoring and non-recensoring options. Conclusions Treatment switching adjustment analyses using TSE should be conducted with re-censoring, without re-censoring, and with IPCW to explore the sensitivity in results to these application options. This should allow analysts and decision-makers to better interpret the results of adjustment analyses.

Periodic demographic health surveillance and surveys are the main sources of health information in developing countries. Conducting a survey requires extensive use of paper-pen and manual work and lengthy processes to generate the required information. Despite the rise of popularity in using electronic data collection systems to alleviate the problems, sufficient evidence is not available to support the use of electronic data capture (EDC) tools in interviewer-administered data collection processes. This study aimed to compare data quality parameters in the data collected using mobile electronic and standard paper-based data capture tools in one of the health and demographic surveillance sites in northwest Ethiopia. A randomized controlled crossover health care information technology evaluation was conducted from May 10, 2016, to June 3, 2016, in a demographic and surveillance site. A total of 12 interviewers, as 2 individuals (one of them with a tablet computer and the other with a paper-based questionnaire) in 6 groups were assigned in the 6 towns of the surveillance premises. Data collectors switched the data collection method based on computer-generated random order. Data were cleaned using a MySQL program and transferred to SPSS (IBM SPSS Statistics for Windows, Version 24.0) and R statistical software (R version 3.4.3, the R Foundation for Statistical Computing Platform) for analysis. Descriptive and mixed ordinal logistic analyses were employed. The qualitative interview audio record from the system users was transcribed, coded, categorized, and linked to the International Organization for Standardization 9241-part 10 dialogue principles for system usability. The usability of this open data kit-based system was assessed using quantitative System Usability Scale (SUS) and matching of qualitative data with the isometric dialogue principles. From the submitted 1246 complete records of questionnaires in each tool, 41.89% (522/1246) of the paper and pen data capture (PPDC) and 30.89% (385/1246) of the EDC tool questionnaires had one or more types of data quality errors. The overall error rates were 1.67% and 0.60% for PPDC and EDC, respectively. The chances of more errors on the PPDC tool were multiplied by 1.015 for each additional question in the interview compared with EDC. The SUS score of the data collectors was 85.6. In the qualitative data response mapping, EDC had more positive suitability of task responses with few error tolerance characteristics. EDC possessed significantly better data quality and efficiency compared with PPDC, explained with fewer errors, instant data submission, and easy handling. The EDC proved to be a usable data collection tool in the rural study setting. Implementation organization needs to consider consistent power source, decent internet connection, standby technical support, and security assurance for the mobile device users for planning full-fledged implementation and integration of the system in the surveillance site.

International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the importance of challenges and show that the lack of quality control has critical consequences. First, reproducibility and interpretation of the results is often hampered as only a fraction of relevant information is typically provided. Second, the rank of an algorithm is generally not robust to a number of variables such as the test data used for validation, the ranking scheme applied and the observers that make the reference annotations. To overcome these problems, we recommend best practice guidelines and define open research questions to be addressed in the future. Biomedical image analysis challenges have increased in the last ten years, but common practices have not been established yet. Here the authors analyze 150 recent challenges and demonstrate that outcome varies based on the metrics used and that limited information reporting hampers reproducibility.

Objectives: To analyze sources searched in Cochrane reviews, to determine the proportion of trials included in reviews that are indexed in major databases, and to compare the quality of these trials with those from other sources. Methods: All new systematic reviews in the Cochrane Library, Issue1 2001, that were restricted to randomized controlled trials (RCTs) or quasi-RCTs were selected. The sources searched in the reviews were recorded, and the trials included were checked to see whether they were indexed in four major databases. Trials not indexed were checked to determine how they could be identified. The quality of trials found in major databases was compared with those found from other sources. Results: The range in the number of databases searched per review ranged between one and twenty-seven. The proportion of the trials in the four databases were Cochrane Controlled Trials Register=78.5%, MEDLINE=68.8%, Embase=65.0%, and Science/Social Sciences Citation Index=60.7%. Searching another twenty-six databases after Cochrane Controlled Trials Register (CCTR), MEDLINE, and Embase only found 2.4% additional trials. There was no significant difference between trials found in the CCTR, MEDLINE, and Embase compared with other trials, with respect to adequate allocation concealment or sample size. Conclusions: There was a large variation between reviews in the exhaustiveness of the literature searches. CCTR was the single best source of RCTs. Additional database searching retrieved only a small percentage of extra trials. Contacting authors and manufacturers to find unpublished trials appeared to be a more effective method of obtaining the additional better quality trials.

Our objective was to assess the risks and benefits of carotid endarterectomy, primarily in terms of stroke prevention, in patients with recently symptomatic carotid stenosis. This multicentre, randomised controlled trial enrolled 3024 patients. We enrolled men and women of any age, with some degree of carotid stenosis, who within the previous 6 months had had at least one transient or mild symptomatic ischaemic vascular event in the distribution of one or both carotid arteries. Between 1981 and 1994, we allocated 1811 (60%) patients to surgery and 1213 (40%) to control (surgery to be avoided for as long as possible). Follow-up was until the end of 1995 (mean 6·1 years), and the main analyses were by intention to treat. The overall outcome (major stroke or death) occurred in 669 (37·0%) surgery-group patients and 442 (36·5%) control-group patients. The risk of major stroke or death complicating surgery (7·0%) did not vary substantially with severity of stenosis. On the other hand, the risk of major ischaemic stroke ipsilateral to the unoperated symptomatic carotid artery increased with severity of stenosis, particularly above about 70–80% of the original luminal diameter, but only for 2–3 years after randomisation. On average, the immediate risk of surgery was worth trading off against the long-term risk of stroke without surgery when the stenosis was greater than about 80% diameter; the Kaplan-Meier estimate of the frequency of a major stroke or death at 3 years was 26·5% for the control group and 14·9% for the surgery group, an absolute benefit from surgery of 11·6%. However, consideration of variations in risk with age and sex modified this simple rule based on stenosis severity. We present a graphical procedure that should improve the selection of patients for surgery. Carotid endarterectomy is indicated for most patients with a recent non-disabling carotid-territory ischaemic event when the symptomatic stenosis is greater than about 80%. Age and sex should also be taken into account in decisions on whether to operate.

The objective of the study is to present the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) conceptual approach to the assessment of certainty of evidence from modeling studies (i.e., certainty associated with model outputs). Expert consultations and an international multidisciplinary workshop informed development of a conceptual approach to assessing the certainty of evidence from models within the context of systematic reviews, health technology assessments, and health care decisions. The discussions also clarified selected concepts and terminology used in the GRADE approach and by the modeling community. Feedback from experts in a broad range of modeling and health care disciplines addressed the content validity of the approach. Workshop participants agreed that the domains determining the certainty of evidence previously identified in the GRADE approach (risk of bias, indirectness, inconsistency, imprecision, reporting bias, magnitude of an effect, dose–response relation, and the direction of residual confounding) also apply when assessing the certainty of evidence from models. The assessment depends on the nature of model inputs and the model itself and on whether one is evaluating evidence from a single model or multiple models. We propose a framework for selecting the best available evidence from models: 1) developing de novo, a model specific to the situation of interest, 2) identifying an existing model, the outputs of which provide the highest certainty evidence for the situation of interest, either “off-the-shelf” or after adaptation, and 3) using outputs from multiple models. We also present a summary of preferred terminology to facilitate communication among modeling and health care disciplines. This conceptual GRADE approach provides a framework for using evidence from models in health decision-making and the assessment of certainty of evidence from a model or models. The GRADE Working Group and the modeling community are currently developing the detailed methods and related guidance for assessing specific domains determining the certainty of evidence from models across health care–related disciplines (e.g., therapeutic decision-making, toxicology, environmental health, and health economics).