Explore the vast range of titles available.

Understanding, categorizing, and using implementation science theories, models, and frameworks is a complex undertaking. The issues involved are even more challenging given the large number of frameworks and that some of them evolve significantly over time. As a consequence, researchers and practitioners may be unintentionally mischaracterizing frameworks or basing actions and conclusions on outdated versions of a framework. This paper addresses how the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework has been described, summarizes how the model has evolved over time, and identifies and corrects several misconceptions. We address 13 specific areas where misconceptions have been noted concerning the use of RE-AIM and summarize current guidance on these issues. We also discuss key changes to RE-AIM over the past 20 years, including the evolution to Pragmatic Robust Implementation and Sustainability Model, and provide resources for potential users to guide application of the framework. RE-AIM and many other theories and frameworks have evolved, been misunderstood, and sometimes been misapplied. To some degree, this is inevitable, but we conclude by suggesting some actions that reviewers, framework developers, and those selecting or applying frameworks can do to prevent or alleviate these problems.

OBJECTIVES/SPECIFIC AIMS: To review the multiple differences between traditional research design and on the ground pragmatic trials. To review two pragmatic projects, identify core assumptions and to contrast assumptions with the reality of conducting T3 and T4 research. METHODS/STUDY POPULATION: Observational mixed methods multi trial review of large multi site implementations. RESULTS/ANTICIPATED RESULTS: The complexities of implementation on the ground were consistently greater than anticipated and required changing assumptions and research design elements. DISCUSSION/SIGNIFICANCE OF IMPACT: Research findings are tremendously influenced by design and design implementation decisions. Anticipating the scope and breadth of the challenges will assist potential of successful implementation.

Valid measures of behavioral health integration have the potential to enable comparisons of various models of integration, contribute to the overall development of high-quality care, and evaluate outcomes that are strategically aligned with standard improvement efforts. The Practice Integration Profile has proven to discriminate among clinic types and integration efforts. We continued the validation of the measure’s internal consistency, intra-rater consistency, and inter-rater consistency with a separate and larger sample from a broader array of practices. We found that the Practice Integration Profile demonstrated a high level of internal consistency, suggesting empirically sound measurement of independent attributes of integration, and high reliability over time. The Practice Integration Profile provides internally consistent and interpretable results and can serve as both a quality improvement and health services research tool.

Responding to complex needs calls for integrating care across providers, settings and sectors. Among models to improve integrated care, case management demonstrates a good evidence base of facilitating the appropriate delivery of healthcare services. Since case management is a complex, multi component intervention, with its component parts interacting in a non-linear manner, effectiveness is largely influenced by the context in which the intervention is implemented. This paper discusses how to respond to implementation challenges to evaluating complex interventions for patients with complex needs. Building on the example of case management, we suggest that documenting innovation effectiveness remains important, but that evaluation needs to include theory-based and systems perspectives. We also suggest that implementation science needs to be part of intervention design while engaging stakeholders to define the most relevant research questions and implementation effectiveness, to optimize successful implementation and sustainability.

Health-related quality of life (HRQol) is a crucial dimension of care outcomes. Many HRQoL measures exist, but methodological and implementation challenges impede primary care (PC) use. We aim to develop and evaluate a novel machine learning (ML) algorithm that predicts binary risk levels among PC patients by combining validated elements from existing measures with demographic data from patient electronic health records (eHR) to increase predictive accuracy while reducing prospectively-collected data required to generate valid risk estimates. Self-report questions from previously validated QoL surveys were collected from PC patients and combined with their demographic and social determinant (SD) data to form a 53-question item bank from which ML chose the most predictive elements. For algorithm development, 375 observations were allocated to training (n = 301, 80%) or test partitions (n = 74, 20%). Questions that asked participants to rate how happy or satisfied they have been with their lives and how easy or hard their emotional health makes work/school showed a good ability to classify participants’ mental QoL (98% max balanced accuracy). Questions that asked participants to rate how easy or hard it is to do activities such as walking or climbing stairs and how much pain limits their everyday activities showed ability to classify physical QoL (94% max balanced accuracy). No demographic or SD factors were significantly predictive. Supervised machine learning can inform QoL measurements to reduce data collection, simplify scoring, and allow for meaningful use by clinicians. Results from the current study show that a reduced 4-question model may predict QoL almost as well as a full-length 40-question measure.

Background: Quality integrated care, which involves primary care and mental health clinicians working together, can help identify and treat adolescent depression early. We explored systemic barriers to quality integrated care at the provincial level in Ontario, Canada using a learning system approach. Methods: Two Ontario Health Teams (OHTs), regional networks designed to support integrated care, completed the Practice Integration Profile (PIP) and participated in focus groups. Results: The OHTs had a median PIP score of 69 out of 100. Among the PIP domains, the lowest median score was case identification (50), and the highest one was workspace (100). The focus groups generated 180 statements mapped to the PIP domains. Workflow had the highest number of coded statements (59, 32.8%). Discussion: While the primary care practices included mental health clinicians on-site, the findings highlighted systemic barriers with adhering to the integrated care pathway for adolescent depression. These include limited access to mental health expertise for assessment and diagnosis, long wait times for treatment, and shortages of clinicians trained in evidence-based behavioral therapies. These challenges contributed to the reliance on antidepressants as the first line of treatment due to their accessibility rather than evidence-based guidelines. Conclusion: Primary care practices, within regional networks such as OHTs, can form learning systems to continuously identify the strategies needed to support quality integrated care for adolescent depression based on real-world data.

Background Chronic diseases that drive morbidity, mortality, and health care costs are largely influenced by human behavior. Behavioral health conditions such as anxiety, depression, and substance use disorders can often be effectively managed. The majority of patients in need of behavioral health care are seen in primary care, which often has difficulty responding. Some primary care practices are providing integrated behavioral health care (IBH), where primary care and behavioral health providers work together, in one location, using a team-based approach. Research suggests there may be an association between IBH and improved patient outcomes. However, it is often difficult for practices to achieve high levels of integration. The Integrating Behavioral Health and Primary Care study responds to this need by testing the effectiveness of a comprehensive practice-level intervention designed to improve outcomes in patients with multiple chronic medical and behavioral health conditions by increasing the practice’s degree of behavioral health integration. Methods Forty-five primary care practices, with existing onsite behavioral health care, will be recruited for this study. Forty-three practices will be randomized to the intervention or usual care arm, while 2 practices will be considered “Vanguard” (pilot) practices for developing the intervention. The intervention is a 24-month supported practice change process including an online curriculum, a practice redesign and implementation workbook, remote quality improvement coaching services, and an online learning community. Each practice’s degree of behavioral health integration will be measured using the Practice Integration Profile. Approximately 75 patients with both chronic medical and behavioral health conditions from each practice will be asked to complete a series of surveys to measure patient-centered outcomes. Change in practice degree of behavioral health integration and patient-centered outcomes will be compared between the two groups. Practice-level case studies will be conducted to better understand the contextual factors influencing integration. Discussion As primary care practices are encouraged to provide IBH services, evidence-based interventions to increase practice integration will be needed. This study will demonstrate the effectiveness of one such intervention in a pragmatic, real-world setting. Trial registration ClinicalTrials.gov NCT02868983 . Registered on August 16, 2016.

ObjectiveWe developed and tested a measure to identify level of primary care behavioral health integration. We produced a thirty item, six domain electronically delivered measure, and a total score.MethodsWe generated a convenience sample of 137 survey responses, including 104 primary care practices. We provided each practice a summary of their own data, and generated a data base of all submissions. We calculated descriptive statistics.ResultsThe mean total score was 56/100. The Vermont Integration Profile (VIP) discriminated between types of practices in the direction hypothesized. Initial test retest reliability was good.ConclusionThe VIP demonstrated good feasibility and construct validity, initial reliability, low provider demand and good discrimination between types of practices.