Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
125,391
result(s) for
"Program evaluation"
Sort by:
A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review
There is mixed evidence to support current ambitions for mobile health (mHealth) apps to improve chronic health and well-being. One proposed explanation for this variable effect is that users do not engage with apps as intended. The application of analytics, defined as the use of data to generate new insights, is an emerging approach to study and interpret engagement with mHealth interventions.
This study aimed to consolidate how analytic indicators of engagement have previously been applied across clinical and technological contexts, to inform how they might be optimally applied in future evaluations.
We conducted a scoping review to catalog the range of analytic indicators being used in evaluations of consumer mHealth apps for chronic conditions. We categorized studies according to app structure and application of engagement data and calculated descriptive data for each category. Chi-square and Fisher exact tests of independence were applied to calculate differences between coded variables.
A total of 41 studies met our inclusion criteria. The average mHealth evaluation included for review was a two-group pretest-posttest randomized controlled trial of a hybrid-structured app for mental health self-management, had 103 participants, lasted 5 months, did not provide access to health care provider services, measured 3 analytic indicators of engagement, segmented users based on engagement data, applied engagement data for descriptive analyses, and did not report on attrition. Across the reviewed studies, engagement was measured using the following 7 analytic indicators: the number of measures recorded (76%, 31/41), the frequency of interactions logged (73%, 30/41), the number of features accessed (49%, 20/41), the number of log-ins or sessions logged (46%, 19/41), the number of modules or lessons started or completed (29%, 12/41), time spent engaging with the app (27%, 11/41), and the number or content of pages accessed (17%, 7/41). Engagement with unstructured apps was mostly measured by the number of features accessed (8/10, P=.04), and engagement with hybrid apps was mostly measured by the number of measures recorded (21/24, P=.03). A total of 24 studies presented, described, or summarized the data generated from applying analytic indicators to measure engagement. The remaining 17 studies used or planned to use these data to infer a relationship between engagement patterns and intended outcomes.
Although researchers measured on average 3 indicators in a single study, the majority reported findings descriptively and did not further investigate how engagement with an app contributed to its impact on health and well-being. Researchers are gaining nuanced insights into engagement but are not yet characterizing effective engagement for improved outcomes. Raising the standard of mHealth app efficacy through measuring analytic indicators of engagement may enable greater confidence in the causal impact of apps on improved chronic health and well-being.
Journal Article
Youth Empowerment Solutions
We report on an effectiveness evaluation of the Youth Empowerment Solutions (YES) program. YES applies empowerment theory to an after-school program for middle school students. YES is an active learning curriculum designed to help youth gain confidence in themselves, think critically about their community, and work with adults to create positive community change. We employed a modified randomized control group design to test the hypothesis that the curriculum would enhance youth empowerment, increase positive developmental outcomes, and decrease problem behaviors. Our sample included 367 youth from 13 urban and suburban middle schools. Controlling for demographic characteristics and pretest outcome measures, we found that youth who received more components of the curriculum reported more psychological empowerment and prosocial outcomes and less antisocial outcomes than youth who received fewer of the intervention components. The results support both empowerment theory and program effectiveness.
Journal Article
Evaluation of the Use of Cancer Registry Data for Comparative Effectiveness Research
Researchers often analyze cancer registry data to assess for differences in survival among cancer treatments. However, the retrospective, nonrandomized design of these analyses raises questions about study validity.
To examine the extent to which comparative effectiveness analyses using observational cancer registry data produce results concordant with those of randomized clinical trials.
In this comparative effectiveness study, a total of 141 randomized clinical trials referenced in the National Comprehensive Cancer Network Clinical Practice Guidelines for 8 common solid tumor types were identified. Data on participants within the National Cancer Database (NCDB) diagnosed between 2004 and 2014, matching the eligibility criteria of the randomized clinical trial, were obtained. The present study was conducted from August 1, 2017, to September 10, 2019. The trials included 85 118 patients, and the corresponding NCDB analyses included 1 344 536 patients. Three Cox proportional hazards regression models were used to determine hazard ratios (HRs) for overall survival, including univariable, multivariable, and propensity score-adjusted models. Multivariable and propensity score analyses controlled for potential confounders, including demographic, comorbidity, clinical, treatment, and tumor-related variables.
The main outcome was concordance between the results of randomized clinical trials and observational cancer registry data. Hazard ratios with an NCDB analysis were considered concordant if the NDCB HR fell within the 95% CI of the randomized clinical trial HR. An NCDB analysis was considered concordant if both the NCDB and clinical trial P values for survival were nonsignificant (P ≥ .05) or if they were both significant (P < .05) with survival favoring the same treatment arm in the NCDB and in the randomized clinical trial.
Analyses using the NCDB-produced HRs for survival were concordant with those of 141 randomized clinical trials in 79 univariable analyses (56%), 98 multivariable analyses (70%), and 90 propensity score models (64%). The NCDB analyses produced P values concordant with randomized clinical trials in 58 univariable analyses (41%), 65 multivariable analyses (46%), and 63 propensity score models (45%). No clinical trial characteristics were associated with concordance between NCDB analyses and randomized clinical trials, including disease site, type of clinical intervention, or severity of cancer.
The findings of this study suggest that comparative effectiveness research using cancer registry data often produces survival outcomes discordant with those of randomized clinical trial data. These findings may help provide context for clinicians and policy makers interpreting observational comparative effectiveness research in oncology.
Journal Article
Discerning experts : the practices of scientific assessment for environmental policy
\"Discerning Experts assesses the assessments that many governments rely on to help guide environmental policy and action. Through their close look at environmental assessments involving acid rain, ozone depletion, and sea level rise, the authors explore how experts deliberate and decide on the scientific facts about problems like climate change. They also seek to understand how the scientists involved make the judgments they do, how the organization and management of assessment activities affects those judgments, and how expertise is identified and constructed.\"--cover
Book
Implementation fidelity in a complex intervention promoting psychosocial well-being following stroke: an explanatory sequential mixed methods study
Background
Evaluation of complex interventions should include a process evaluation to give evaluators, researchers, and policy makers greater confidence in the outcomes reported from RCTs. Implementation fidelity can be part of a process evaluation and refers to the degree to which an intervention is delivered according to protocol. The aim of this implementation fidelity study was to evaluate to what extent a dialogue-based psychosocial intervention was delivered according to protocol. A modified conceptual framework for implementation fidelity was used to guide the analysis.
Methods
This study has an explanatory, sequential two-phase mixed methods design. Quantitative process data were collected longitudinally along with data collection in the RCT. Qualitative process data were collected after the last data collection point of the RCT. Descriptive statistical analyses were conducted to describe the sample, the intervention trajectories, and the adherence measures. A scoring system to clarify quantitative measurement of the levels of implementation was constructed. The qualitative data sources were analyzed separately with a theory-driven content analysis using categories of adherence and potential moderating factors identified in the conceptual framework of implementation fidelity. The quantitative adherence results were extended with the results from the qualitative analysis to assess which potential moderators may have influenced implementation fidelity and in what way.
Results
The results show that the core components of the intervention were delivered although the intervention trajectories were individualized. Based on the composite score of adherence, results show that 80.1% of the interventions in the RCT were implemented with high fidelity. Although it is challenging to assess the importance of each of the moderating factors in relation to the other factors and to their influence on the adherence measures, participant responsiveness, comprehensiveness of policy description, context, and recruitment appeared to be the most prominent moderating factors of implementation fidelity in this study.
Conclusions
This evaluation of implementation fidelity and the discussion of what constitutes high fidelity implementation of this intervention are crucial in understanding the factors influencing the trial outcome. The study also highlights important methodological considerations for researchers planning process evaluations and studies of implementation fidelity.
Trial registration
ClinicalTrials.gov
, NCT02338869; registered 10/04/2014.
Journal Article
Applied analytics through case studies using SAS and R : implementing predictive models and machine learning techniques
Here readers can examine business problems and use a practical analytical approach to solve them by implementing predictive models and machine learning techniques using SAS and the R analytical language. This book is ideal for those who are well-versed in writing code and have a basic understanding of statistics, but have limited experience in implementing predictive models and machine learning techniques for analyzing real world data.
Book
The RE-AIM Framework: A Systematic Review of Use Over Time
We provided a synthesis of use, summarized key issues in applying, and highlighted exemplary applications in the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. We articulated key RE-AIM criteria by reviewing the published literature from 1999 to 2010 in several databases to describe the application and reporting on various RE-AIM dimensions. After excluding nonempirical articles, case studies, and commentaries, 71 articles were identified. The most frequent publications were on physical activity, obesity, and disease management. Four articles reported solely on 1 dimension compared with 44 articles that reported on all 5 dimensions of the framework. RE-AIM was broadly applied, but several criteria were not reported consistently.
Journal Article