Skip to main content
Download PDF

A mixed-methods observational study of strategies for success in implementation science: overcoming emergency departments hurdles

BMC Health Services Research volume 25, Article number: 147 (2025) Cite this article

Abstract

Background

Heart failure is a major public health concern, affecting 6.7 million Americans. An estimated 16% of emergency department (ED) patients with acute heart failure (AHF) are discharged home. Our Get with the Guidelines in Emergency Department Patients with Heart Failure (GUIDED-HF) toolkit aims to improve AHF self-care and facilitate safer transitions in care for these patients. We describe implementation barriers and facilitators, and the selection and refinement of implementation strategies, to facilitate future GUIDED-HF implementation.

Methods

A mixed-methods cross-sectional observational study was conducted in four United States EDs in two diverse healthcare systems in the Pacific West and Midwest. Data were collected using a survey and interviews with ED providers, nurses, and leaders. The survey assessed the ED context using the context scale of the Organizational Readiness to Change Assessment (ORCA). The Consolidated Framework for Implementation Research informed interviews. Quantitative data were summarized using medians (interquartile ranges) or percentages (frequencies). Wilcoxon rank-sum tests and Kruskal–Wallis tests were used to assess differences in the healthcare system and profession. Qualitative data were analyzed and summarized using rapid qualitative analysis. Convergence of quantitative and qualitative data was used to inform specific refining of implementation strategies to the local context (e.g., who should serve as champions, how best practice alerts should be implemented).

Results

Participants were predominately white (76%) with median (IQR) age 37.0 (32.0, 41.0). ED leaders/administrators, providers, and nurses comprised 15%, 55%, and 29% of participants, respectively. Sites reported an ORCA context scale score of 3.7 [3.4, 4.0] (scale of 1 = strongly disagree to 5 = strongly agree). Comparison of scores by profession showed a significant difference in the context score among providers (3.9 [3.5, 4.0]), leaders (3.7 [3.5, 4.0]), and nurses (3.6 [3.0, 3.9]) (p = 0.048). Qualitative data indicated implementation barriers (e.g., resource limitations, patient health literacy), facilitators (e.g., GUIDED-HF is patient-centric; site and intervention congruent values, norms, and goals), and site-specific needs due to contextual factors (e.g., education needs, feedback mechanisms, champions).

Conclusions

Specific determinants of implementation exist in ED settings and require the refining of implementation strategies to overcome site-specific barriers and enhance facilitators.

Trial registration

n/a.

Peer Review reports

Contributions to the literature

  • GUIDED-HF is a self-care intervention for patients with acute heart failure discharged home from the emergency department (ED), addressing the paucity of interventions to improve their transition from ED to home.

  • This study demonstrates the nuanced contextual differences between implementation sites and the need for a deliberately selected and refined implementation approach addressing the specific concerns and preferences of the local context.

  • The findings contribute to existing knowledge gaps of best practices for implementing complex healthcare interventions in the fast-paced ED setting.

Background

Effectively implementing interventions to improve outcomes in patients with heart failure (HF) remains an urgent need. Approximately 6.7 million US adults live with HF, and in the first 5 years following diagnosis, 83% are hospitalized at least once, and 54% are admitted three or more times [1]. These patients often experience worsening signs and symptoms, referred to as acute heart failure (AHF), requiring emergency care. Most ED patients are hospitalized, and over 18.2% and 31.2%, respectively, experience unplanned 30-day and 90-day HF readmission or mortality after hospital discharge [2]. Approximately 16% of patients with AHF are discharged home directly from the ED and of these 24% subsequently return to the ED or are admitted, necessitating the implementation of evidence-based interventions that improve ED-to-Home transitions [3, 4]. Self-management interventions in heart failure is a proven, acceptable method for enhancing ED-to-Home transitions, [3, 5, 67] including our GUIDED-HF self-care intervention, which improves ED-to-Home care transitions and reduce ED visits and hospital readmissions [6].

GUIDED-HF was previously developed and informed by the Care Transitions Intervention framework (Fig. 1), [8,9,10,11] which consists of three intervention stages: 1) hospital/ED visit, 2) home visit, and 3) follow-up coaching calls. These stages focus on self-care coaches presenting an overview of GUIDED-HF during the post-discharge ED visit and facilitating patient mastery of self-care items during the home visit and coaching calls (Fig. 1). ED providers refer eligible patients to the self-care coaches, who then contact the patients by phone to discuss GUIDED-HF and gauge patient interest in receiving the intervention. Over a 1-month period, a home visit occurs, followed by two coaching calls. Our prior multi-center randomized controlled trial (NCT02519283)(2015–08–10) (Clinicaltrials.gov) [6] comparing GUIDED-HF with usual care in patients with AHF discharged from the ED, showed statistically and clinically significant improvements in a 30-day global rank of cardiovascular death, ED revisits or hospitalization for AHF, overall symptom burden, and HF knowledge [6, 12, 13, 14].

Fig. 1
figure 1

GUIDED-HF multifactorial care transitions intervention

Full size image

Implementing interventions like GUIDED-HF in the fast-paced, complex ED setting with high patient turnover, various patient acuity levels, and complex care delivery can be challenging. Implementation barriers in ED settings include insufficient resources, lack of sufficient training and education, competing priorities, and insufficient time for implementation. However, using implementation science, the scientific study of methods to enhance the uptake and sustainability of evidence-based interventions, [15, 16] can bring clarity to implementation determinants and how best to achieve fit between the implementation context and strategies used to implement an intervention. [15,16,17,18] Developing implementation strategies is a necessary step towards generating organizational change [16, 19,20,21,22,23,24]. Studies have also highlighted the need to assess how different implementation contexts dictate unique barriers and facilitators to outcomes such as reach, adoption, effectiveness, fidelity, and sustainability [19, 25, 26]. Thus, identifying contextual determinants that might impede or enhance (i.e., barriers and facilitators) the implementation of GUIDED-HF and selecting implementation strategies to address known barriers and facilitators was a crucial first step towards its effective implementation and subsequent clinical effectiveness. Matching implementation strategies with the finer nuanced contextual differences in the ED setting was important for long-term sustainment of GUIDED-HF.

We, therefore planned for a Hybrid Type II implementation-effectiveness study to translate GUIDED-HF into real-world settings in two geographically diverse health systems representing four high-volume EDs in the United States and evaluate the implementation outcomes of reach, effectiveness, adoption, implementation and maintenance (RE-AIM). For an implementation approach that was practical and feasible to ED settings, and to prepare for the hybrid type II implementation formative evaluation, we first used a mixed-methods approach to identify barriers and facilitators, and select and refine strategies for GUIDED-HF implementation. We used a mixed-methods approach to quantitively measure the implementation context followed by a qualitatively exploration of implementation strategies to address contextual barriers and facilitators.

Our objective was to report the ED-specific barriers and facilitators we identified and the nuanced differences between sites informing the selection, addition and/or adjustment of our a priori implementation strategies to guide the subsequent Hybrid Type II implementation formative evaluation [27,28,29].

Methods

Overview

In preparing our implementation plan (Table 1), we proposed and used our prior approach of a similarly complex interdisciplinary intervention [30,31,32,33,34], implemented in the ED and hospital. This was informed by extant literature reporting effective implementation strategies for use in ED settings, [17, 18] the research team’s experiences with implementation, [34] and the ERIC (Expert Recommendations for Implementing Change) taxonomy of implementation strategies [20]. However, we anticipated and remained open to adding strategies as needed to address barriers and facilitators we identify in the course of our work. Our goal was to refine these strategies based on survey and interview results to address fine nuanced contextual differences between EDs. For example, based on anecdotal information from site PIs, we anticipated differences in ED workflow processes to facilitate communication with self-care coaches and eligible GUIDED-HF patients after ED discharge. We also anticipated education materials and strategies might differ to accommodate differences in ED staffing patterns between facilities. Our work aligns with recent recommendations for the development of implementation strategies that address local circumstances in ED settings, [35] and the use of qualitative methods [36] and well-known implementation frameworks and taxonomies in implementation studies [16, 37]. Our work will also enable clinicians and implementers to evaluate their contextual determinants against what we describe in this report and the usefulness of our selected and refined implementation strategies in their settings to effectively implement similarly complex interventions.

Table 1 A priori implementation strategies with definitions
Full size table

Study design and setting

A mixed-methods convergent QUANT→QUAL cross-sectional observational study design was used in four EDs in two large United States (US) healthcare systems located in the Pacific West (HC-PW) and the Midwest (HC-MW). The sites are geographically and socio-demographically diverse, with HC-PW serving a large community of privately insured, mostly white patients and HC-MW serving a large urban population of uninsured and underinsured Black patients. This diversity of healthcare settings included tertiary, community and freestanding EDs and was ideal for developing a refined implementation approach for future dissemination across equally diverse US EDs. The sites’ self-care coaching personnel were also diverse. HC-PW used staff who previously served as research coordinators in the ED to serve as study self-care coaches. HC-MW integrated the delivery of GUIDED-HF in their mobile integrated health unit, with community paramedics serving as self-care coaches, allowing for rapid in-home follow-up after ED discharge.

Participant characteristics

Participants were ED providers who were responsible for assessing HF patients, determining their diagnosis, and providing treatment (i.e., physicians, advanced practice nurses, and physician assistants) as well as ED nurses, and ED leaders/administrators.. We asked participants to self-identify to the group they affiliated with and if they identified with more than one group, to choose the group reflective of their primary role. Inclusion criteria were 1) current healthcare system employee, 2) aged ≥ 18 years, 3) one of the following: ED providers and nurses directly involved in patient care; ED managers, supervisors, educators, or clinical nurse specialists; or self-care coaches.

Data collection

Surveys

Surveys were administered using REDCap® (Research Electronic Data Capture), a secure web-based platform for online surveys [38, 39]. Participants were recruited using a study flier and email. The flyer was posted in all study ED sites and contained a link and QR code to the REDCap survey. The recruitment email was sent to all ED providers, nurses, and administrators/leaders and contained the REDCap survey link. Eligible individuals could view the study consent document and progress to the survey items. Consent was indicated by survey completion.

Measures

The context was assessed using the context scale of the Organizational Readiness to Change Assessment (ORCA) [40]. Prior research indicates the ORCA is appropriate for cross-sectional evaluation of organizational differences relevant to health intervention implementation [26, 41,42,43]. As our goal was to assess the ED context of the ORCA’s three scales, we used only the context scale, which consists of six subscales. Two subscales measure aspects of organizational culture (leadership culture and staff culture), one subscale assesses leadership practices, one assesses measurement, one assesses opinion leaders’ readiness to change, and one assesses resources to support practice changes in general [40]. The psychometric properties of the composite context scale (Cronbach alpha:0.85) and subscales (Cronbach alpha: 0.86–0.93) are strong.20 Demographic questions included items for sex, gender, race, and ethnicity. The final survey item asked about willingness to be contacted with questions and to participate in a future interview.

Interviews

Survey participants who indicated a willingness to be interviewed were contacted to schedule interviews. As only few survey respondents agreed to schedule the interview, a flier was also distributed to recruit additional interview participants. Interviews were conducted by a study team member (DS) using Zoom videoconferencing software and an interview guide informed by the Consolidated Framework for Implementation Research (CFIR) [44, 45]. Before the interview, participants were provided with an information sheet with details surrounding the GUIDED-HF intervention. Consent was indicated by participation in the interview and was also confirmed at the start of each interview.

Data analysis

Quantitative analysis

A composite context score was calculated as the mean score across all 23 sub-scale items. Scores for each context subscale were calculated similarly. Missing responses were imputed if the number of non-missing responses exceeded 85% of the number of questions in the survey. Determining whether to impute data was made based on the full scale and not the individual subscales. Data were imputed using the full scale and applied to calculations of the overall scale and the subscales. Missing responses were imputed as the mean of the non-missing responses within a given person. Descriptive statistics are reported by healthcare system and by profession (i.e., leader, provider, nurse). Wilcoxon rank-sum tests and Kruskal–Wallis tests were used to assess differences across healthcare systems and professions, respectively. Results were analyzed using R statistical software (version 4.4.0).

Qualitative data

Interview data were analyzed using inductive rapid qualitative analysis [46]. This approach was used for practical and efficiency reasons and allowed for timely results during the pre-implementation period that would inform GUIDED-HF implementation in the real-world ED setting. The validity of the qualitative data was ensured by reviewing and verifying transcripts for accuracy prior to coding. Two research team members (DPS & ABS) together reviewed the transcripts and summarized each participant’s words as statements in a rapid analysis template in Excel (Additional File A). Participant summary statements were then combined in an Excel sheet, summarizing all participants’ statements by CFIR construct and healthcare system. Finally, the research team members identified contextual similarities and differences between the two healthcare systems.

Convergence of data sources

Once both sources of data were analyzed, these data were combined using a joint display, offering a side-by-side comparison of the quantitative and qualitative data. As highlighted in (Table 6), similar to prior work, [47] the CFIR constructs and associated qualitative summary statements were organized by ORCA domains to gain an understanding of the organizational context at each site. The joint display allowed the study team to compare the qualitative and quantitative data elements, see patterns of contextual similarities and differences between the two healthcare systems, and subsequently use these insights to refine the implementation strategies for each site.

Results

Sample demographic characteristics

The final survey sample consisted of 78 participants, with 35 (45%) from HC-PW and 43 (55%) from HC-MW (Table 2). The sample was comprised of providers (55%) and nurses (29%), with leaders making up the remainder (15%) (Table 3). Participants were predominately white (76%) with median age 37.0 (32.0, 41.0). Comparing participants across healthcare systems, HC-PW had more female (51% vs. 47%) and fewer white (71% vs. 79%) participants, compared with HC-MW (Table 2). A total of 14 participants completed interviews with an equal number from HC-MW and HC-PW agreeing to be interviewed. Of the 78 who completed the Context Assessment, 24 participants indicated an interest in being interviewed – 16 providers, 6 nurses, and 2 leaders. –Upon reaching out to confirm their willingness and schedule the interview, 11 subsequently scheduled and completed interviews (nine providers, one nurse, and two leaders). Subsequent recruitment using a study flyer resulted in four additional responses (3 providers and one nurse) and of these, two providers and one nurse completed the interviews. Due to scheduling difficulties or participant preferences, both individual and group interviews were conducted for a total of three group sessions (ranging from 2 to 6 individuals per group) and three individual interviews. Individual interviews occurred when scheduling difficulties prevented group interviews. No new themes emerged at the end these interviews, resulting in data saturation, and thus recruitment of additional interview participants was discontinued.

Table 2 Descriptive statistics of demographic data for the overall sample
Full size table
Table 3 Descriptive statistics of demographic data of GUIDED-HF implementation by profession
Full size table

Convergence of findings: contextual assessment results and effects on implementation

Convergence of the quantitative and qualitative data informed the selection and refinement of implementation strategies for each site. The research team started with an initial list of strategies (Table 1), selected based on their prior experience with implementing interventions in ED and hospital settings. Context scale scores are reported by site (Table 4) and by profession (Table 5).

Table 4 Test of association for bsurvey results by ED location
Full size table
Table 5 Test of association for survey results by profession
Full size table

Convergence of the reported context scores (ORCA measures) and qualitative data (CFIR constructs) are reported in Table 6 and discussed next. CFIR constructs used in the interview guide and additional constructs identified were mapped to the mixed-methods data (see Additional File B).

Table 6 Convergence of quantitative and qualitative data
Full size table

The context scale assesses the quality of an organization’s context to support practice change. On a scale of 1 = strongly disagree to 5 = strongly agree, we observed a moderate composite context score (3.7 [3.4, 4.0]). Scores were similar across the two healthcare systems and aligned with interview findings demonstrating the participants’ overall optimism for incorporating GUIDED-HF as a practice change. Composite context scores differed across professions (p = 0.048), with providers reporting higher scores (3.9 [3.5, 4.0]) than leaders (3.7 [3.5, 4.0]) or nurses (3.6 [3.0, 3.9]). (See Tables 4 and 5). Post hoc pairwise tests using Wilcoxon Rank Sum tests showed that the significant difference was driven by the difference between Providers and Nurses (difference in median: 0.3; Bonferroni-corrected p-value: 0.04), and that there is no difference between Leader and Provider (difference in median: −0.2; Bonferroni-corrected p-value: 1.0), and between Leader and Nurse (difference in median: 0.1; Bonferroni-corrected p-value: 0.66).

Organizational culture: two dimensions

Leadership and staff culture subscale scores were similar (leadership culture: 4.0 [3.3, 4.0]; staff culture: 4.0 [3.5, 4.2]), indicating moderately high organizational culture for change. Leadership culture scores at HC-MW and HC-PW were also similar (4.0 [3.3, 4.0]), as were staff culture scores (HC-MW: 4.0 [3.6, 4.2]; HC-PW: 4.0 [3.5, 4.0]) (Table 3). However, salient differences arose within the qualitative interviews. HC-PW experienced significant staff turnover, especially among nursing staff, whereas HC-MW identified a stable “core group” of ED nurses known for leading practice change among other ED staff and embodying quality care.

Implementation based on organizational culture findings

Organizational culture differences were observed for educational session frequency, who should serve as site champions, and who should be part of consensus discussions. At HC-PW, the research team targeted dedicated ED faculty as champions, as they were more likely to be consistently present and employed by the unit throughout the study period. In contrast, HC-MW’s high-performing and respected “core group” of ED nurses were identified as pivotal to serving as site champions. High nursing staff turnover at HC-PW also signaled the need to repeat HC-PW’s educational sessions at least every 4–6 months to account for travel nursing contracts and staff turnover (Table 7).

Table 7 Implementation strategies and tailoring by site
Full size table

Leadership practice

The formal leadership subscale score (4.0 [3.5, 4.2]) demonstrated moderate leadership practice for implementing GUIDED-HF. Qualitative findings suggested leadership support for GUIDED-HF workflow development was essential for program success. In both healthcare systems, the involvement and support of the ED medical director and ED nurses were highlighted as important. Both sites indicated believing their respective leadership teams supported incorporating GUIDED-HF specifically and cultivated moderately strong leadership contexts (HC-MW: 4.0 [3.5, 4.2]; HC-PW: 4.0 [3.2, 4.1]). But, drawing on the qualitative findings, sites appeared uncertain regarding leadership’s follow through because of site-specific concerns. Whereas a mismatch between leadership and staff values (i.e., financial impact vs. clinical outcomes) contributed to hesitation among HC-PW clinicians, a sense of general lack of support for significant practice change defined HC-MW clinicians’ hesitation. HC-MW’s global distrust regarding leadership’s practice change support alerted the research team to the importance of addressing this distrust during implementation planning.

The score on the measurement context subscale, which assesses leadership setting goals, information sharing, and feedback, was moderately high (3.8 [3.1, 4.0]). The context score for measurement scored higher at HC-MW (4.0, [3.4, 4.0]) compared to HC-PW (3.8 [3.0, 4.0]). HC-PW clinicians expressed concern over the use of ED throughput metrics such as length of stay and time to be seen rather than metrics specifically related to clinical outcomes. They also perceived concern for throughput metrics occasionally being used punitively. At HC-MW, clinicians noted metrics were generally not used in compensation/performance decisions. These qualitative insights explain conflicting results in the quantitative data; specifically, HC-PW participants reported lower measurement context scores, though leadership is perceived as using “more” measurement overall. HC-PW clinicians preferred to receive performance feedback only in aggregate at the group level rather than individually. HC-MW clinicians appeared open to both individual- and group-level feedback. These insights were invaluable when considering how to tailor audit and feedback and adaptation of workflow processes.

Implementation based on leadership practice findings

The research team dedicated resources to show recognition and support for clinician and nurse participation at both sites. We worked with local leaders to provide site specific recognition. The research team also created a process to identify clinicians with low GUIDED-HF adoption rates and took steps to encourage future adoption and patient referral. Audit and feedback processes were planned to reflect the site preferences noted above. Survey and interview insights stressed the importance of designing clinically relevant proxies for clinician performance. For example, rather than providing clinicians with feedback regarding potential ED cost savings through GUIDED-HF referral, the study team focused subsequent updates on clinically oriented metrics such as changes in ED readmissions and ED follow-up visits. Feedback for both sites was structured using positive framing (Table 7).

Opinion leaders’ readiness to change

Although the ORCA readiness context subscale score (4.0 [3.8, 4.2]) suggested strong readiness for GUIDED-HF and practice change overall within the two healthcare systems (HC-MW: 4.0 [3.8, 4.0]; HC-PW: 4.0 [4.0, 4.2]), qualitative findings helped identify critical insights to further improve implementation readiness. Clinicians and staff at HC-PW reported a history of lackluster adoption of prior program initiatives due especially to provider resistance. HC-MW staff and clinicians suggested their ED already incorporates programs like GUIDED-HF for other clinical conditions and noted the system at large is currently engaged in an HF readmission reduction initiative.

Implementation based on readiness for change findings

These insights informed the deployment of GUIDED-HF and its marketing to clinicians. Pre-implementation marketing and education included informing providers of GUIDED-HF and the importance of their responding to the best practice alert (BPA) and referring patients as applicable. The unique role of the self-care coach in contacting patients and executing the intervention was emphasized, as was the transfer of workload from providers to self-care coaches. Educational materials and content were developed to emphasize the benefits of GUIDED-HF participation for patients. Identifying GUIDED-HF’s similarities to other positively viewed programs was key, and alignment with HC-MW’s larger HF readmission initiative was stressed in marketing materials and consensus discussions (Table 7).

Resources

The resources context subscale had the lowest overall score (3.0 [2.5, 3.5]). No differences were observed in scores between the two healthcare systems (Table 3). However, a statistically significant difference was observed among professions with a higher scored observed for providers for the availability of necessary resources to support practice change (3.2 [2.8, 3.8]) followed by nurses (2.8 [2.0, 3.1]) and leaders (3.0 [2.5, 3.6]) (p = 0.026). Qualitative interview findings showed that, at HC-PW, participants were concerned about the lack of resources, space, and bed shortages to support GUIDED-HF’s ongoing sustainability. HC-MW participants shared similar concerns and noted a need for infrastructure to support self-care coaches, with no significant infrastructure changes necessary to implement GUIDED-HF. Despite resource concerns, they displayed overwhelming optimism for GUIDED-HF and perceived GUIDED-HF as innovative and highly compatible with their existing practices in the ED. Participants remarked on the advantage of GUIDED-HF over other alternatives for discharging patients with AHF. They believed GUIDED-HF would reduce provider workloads and help overcome patient barriers to seeking care (particularly patient-related literacy challenges).

Interviewed participants reported multiple possible solutions to alleviate resource concerns. Participants reported using a BPA to trigger referral should be vetted and resource allocation should focus on equipping self-care coaches specifically to execute GUIDED-HF. Participants noted the need to train ED staff on GUIDED-HF using different training modalities and educational strategies. The message communicating the importance of implementing GUIDED-HF needed to be clear, simple, and standardized. Finally, feedback was seen as an essential component of engaging ED staff members. Participants mentioned specific needs when providing feedback, including longitudinal data on patient outcomes (e.g., patient follow-up visits and hospitalizations). Sites differed on the content, type, mode, and frequency of feedback they wanted.

Implementation based on resources findings

To address these concerns, we emphasized the GUIDED-HF intervention structure during education and marketing. Providers were reminded of the BPA trigger as the only GUIDED-HF component occurring in the ED setting, with self-care coach follow-up after ED discharge. This structure limited the use of ED resources, including provider time and space. GUIDED-HF’s role of ensuring follow-up care of patients with AHF after ED discharge was also emphasized during training events (Table 7).

Discussion

Despite a robust commitment to developing clinical practice guidelines and scientific reviews to keep pace with the rapid advancements in evidence-based emergency medicine, significant knowledge-to-practice gaps and care variation remain [48]. We utilized implementation science strategies to reduce this evidence-to-practice gap by presenting a novel, theory-informed example of organizing and understanding quantitative and qualitative contextual findings. Our goal was to identify implementation determinants in diverse ED settings to inform the development of implementation strategies. By developing, specifying, and reporting effective implementation strategies and mapping these strategies to the contextual settings where they will be most effective, our approach allows EDs to select the implementation strategies most suited to the determinants specific to their context. This work aligns with existing recommendations [35] and provides a starting point for clinicians and implementers as they develop implementation strategies that address their local ED setting.

Although the fast-paced environment of the ED, with high patient complexity and staff turnover is unquestionably unique, the current and prior studies’ results suggest ED settings experience similar barriers to practice change and needs for refined implementation strategies as other care specialties. In this study, ED sites identified insufficient resources and education for practice change as major barriers. This finding was consistent with a previous scoping review of determinants of prediction models in ED settings that found limited resources and education as significant determinants of successful implementation [49]. These needs also align with other care specialties and thus suggest that tailored implementation strategies developed in other care contexts (e.g., the ERIC taxonomy [20]) might be useful in emergency care settings.

This mixed-methods convergent analytic approach improves multiple aspects of implementation planning and provides opportunities for future implementation science research. Rather than using only quantitative or qualitative data to understand the implementation context, this study used complimentary methods, that is integrating quantitative and qualitative data, to gain a comprehensive understanding of contextual determinants. Minimal differences between sites were observed in the quantitative data, with the exception of the measurement sub-scale with questions that focused on the sharing of performance measures and feedback. The qualitative data elucidated reasons for the different ratings of this sub-scale between the two sites. Overall, the quantitative ORCA context scale allowed for a quick and high-level evaluation of site contextual factors as determinants of implementation But, the qualitative data brought to light the nuanced differences between the two healthcare systems and provided greater insight into specific site differences and implementation preferences that underscored the need for a tailored approach to address the specific concerns and preferences of the local context.. The convergence of the qualitative and quantitative findings was imperative for understanding how contextual challenges uniquely manifested in each system, allowing us to develop a cohesive implementation approach. We suggest prospective mixed-methods assessment is key to understanding ED contextual determinants of implementation.

The insights gained from this study can inform the development of tailored strategies to enhance the implementation of similarly complex interventions in ED settings. For instance, at sites like HC-PW, research teams in future may need to focus on addressing provider resistance and building trust, while at sites like HC-MW, they can build upon the existing experience and organizational focus on heart failure to facilitate a more seamless integration of the GUIDED-HF program. The findings also underscore the importance of assessing and addressing the multifaceted aspects of organizational context and culture when implementing practice changes in healthcare settings. For example, the moderate composite context score and moderately high organizational culture provide a generally supportive environment, but the observed differences across professions and healthcare systems highlight the need for a tailored approach that considers the unique characteristics and dynamics of each organization. The moderate leadership practice score and moderately high measurement context score suggest a generally supportive environment, but the site-specific differences identified through the qualitative insights further highlight the need for a tailored approach that addresses the unique characteristics and dynamics of each organization. Our work adds to prior recommendations for implementation in EDs, stressing the importance of understanding implementation barriers and facilitators, identifying strong local champions, having a study start-up meeting with local stakeholders, regularly communicating with each site, and using incentives to encourage compliance. Our work similarly demonstrates the need for consensus discussions with ED stakeholders and the use of champions and incentives to support staff implementing interventions in real-world settings. We also demonstrated that, with some site-specific adaptation, similar implementation strategies could be deployed across two diverse healthcare systems and their EDs. For example, champions were used at all sites, but the selection of who these would be was determined by site preference.

Some limitations of this study should be noted. As the GUIDED-HF implementation-effectiveness study is ongoing, this article details implementation determinants and the selection and refinement of implementation strategies but does not report on the outcomes of these implementation strategies in ED settings. It is unclear how our selected and refined strategies will impact implementation outcomes of interest (e.g., reach, adoption, and maintenance). But, our ongoing hybrid type II implementation formative evaluation will help determine the impact of these strategies. Additionally, stakeholder groups (e.g., staff, nurses, leaders, physicians) were not equally represented across sites. However, prior work suggests over- or under-representation of stakeholder groups is unlikely to impact the identification of relevant determinants or strategies for successful evidence-based practice implementation [50]. Also, because of the small sample of the different professional groups, comparing the qualitative findings across the various small groups was not possible and should be explored in future research.

Overall, our experience suggests converging quantitative and qualitative data is a low-burden, efficient, and beneficial approach for research teams desiring to prospectively refine implementation strategies. Our novel process can serve as a foundation for future implementation research work in the ED setting.

Conclusion

This study identified crucial contextual determinants to consider when selecting and refining implementation strategies in ED settings. Special consideration to the multifaceted aspects of the ED and broader organizational context and culture is crucial to facilitate practice change for effective implementation. This study provides ED settings with guidance for future implementation by mapping implementation strategies to contextual determinants, thereby allowing other ED settings to determine the strategies that might be most effective in their particular setting.

Data availability

The quantitative and qualitative data supporting the conclusions of this article will not be publicly available due to ongoing data analyses related to all study aims.

References

  1. Tsao CW, Aday AW, Almarzooq ZI, Anderson CAM, Arora P, Avery CL, et al. heart disease and stroke statistics-2023 update: a report from the american heart association. Circulation. 2023;147(8):e93–621.

    Article  PubMed  Google Scholar 

  2. Khan MS, Sreenivasan J, Lateef N, Abougergi MS, Greene SJ, Ahmad T, et al. Trends in 30- and 90-day readmission rates for heart failure. Circ Heart Fail. 2021;14(4):e008335.

    Article  PubMed  Google Scholar 

  3. Shah MN, Hollander MM, Jones CM, Caprio TV, Conwell Y, Cushman JT, et al. Improving the ED-to-home transition: the community paramedic-delivered care transitions intervention-preliminary findings. J Am Geriatr Soc. 2018;66(11):2213–20.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Naderi N, Chenaghlou M, Mirtajaddini M, Norouzi Z, Mohammadi N, Amin A, et al. Predictors of readmission in hospitalized heart failure patients. J Cardiovasc Thorac Res. 2022;14(1):11.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Collins S, Storrow AB, Albert NM, Butler J, Ezekowitz J, Felker GM, et al. Early management of patients with acute heart failure: state of the art and future directions A consensus document from the society for academic emergency medicine/heart failure society of America acute heart failure working group. J Cardiac Fail. 2015;21(1):27–43.

    Article  Google Scholar 

  6. Collins SP, Liu D, Jenkins CA, Storrow AB, Levy PD, Pang PS, et al. Effect of a self-care intervention on 90-day outcomes in patients with acute heart failure discharged from the emergency department: a randomized clinical trial. JAMA Cardiol. 2021;6(2):200–8.

    Article  PubMed  Google Scholar 

  7. Storrow AB, Jenkins CA, Self WH, Alexander PT, Barrett TW, Han JH, et al. The burden of acute heart failure on U.S. emergency departments. JACC Heart Fail. 2014;2(3):269–77.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Coleman EA, Parry C, Chalmers S, Min SJ. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med. 2006;166(17):1822–8.

    Article  PubMed  Google Scholar 

  9. Voss R, Gardner R, Baier R, Butterfield K, Lehrman S, Gravenstein S. The care transitions intervention: translating from efficacy to effectiveness. Arch Intern Med. 2011;171(14):1232–7.

    Article  PubMed  Google Scholar 

  10. Coleman EA, Smith JD, Frank JC, Min SJ, Parry C, Kramer AM. Preparing patients and caregivers to participate in care delivered across settings: the Care Transitions Intervention. J Am Geriatr Soc. 2004;52(11):1817–25.

    Article  PubMed  Google Scholar 

  11. Coleman EA, Rosenbek SA, Roman SP. Disseminating evidence-based care into practice. Popul Health Manag. 2013;16(4):227–34.

    Article  PubMed  Google Scholar 

  12. Butler J, Khan MS, Mori C, Filippatos GS, Ponikowski P, Comin-Colet J, et al. Minimal clinically important difference in quality of life scores for patients with heart failure and reduced ejection fraction. Eur J Heart Fail. 2020;22(6):999–1005.

    Article  PubMed  Google Scholar 

  13. Nassif ME, Windsor SL, Tang F, Khariton Y, Husain M, Inzucchi SE, et al. Dapagliflozin effects on biomarkers, symptoms, and functional status in patients with heart failure with reduced ejection fraction: The DEFINE-HF trial. Circulation. 2019;140(18):1463–76.

    Article  PubMed  CAS  Google Scholar 

  14. Stubblefield WB, Jenkins CA, Liu D, Storrow AB, Spertus JA, Pang PS, et al. Improvement in Kansas City cardiomyopathy questionnaire scores after a self-care intervention in patients with acute heart failure discharged from the emergency department. Circ Cardiovasc Qual Outcomes. 2021;14(10):e007956.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.

    Article  PubMed  Google Scholar 

  16. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24–34.

    Article  PubMed  Google Scholar 

  17. Tavender E, Babl FE, Middleton S. Review article: A primer for clinical researchers in the emergency department: Part VIII Implementation science: An introduction. Emerg Med Australas. 2019;31(3):332–8.

    Article  PubMed  Google Scholar 

  18. Tavender EJ, Bosch M, Fiander M, Knott JC, Gruen RL, O’Connor D. Implementation research in emergency medicine: a systematic scoping review. Emerg Med J. 2016;33(9):652–9.

    Article  PubMed  Google Scholar 

  19. Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57.

    Article  PubMed  Google Scholar 

  20. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: Recommendations for specifying and reporting. Implement Sci. 2013;8:139.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Balas EA, Boren SA. Managing Clinical Knowledge for Health Care Improvement. Yearb Med Inform. 2000;1:65–70.

    Google Scholar 

  23. Sinnema H, Majo MC, Volker D, Hoogendoorn A, Terluin B, Wensing M, et al. Effectiveness of a tailored implementation programme to improve recognition, diagnosis and treatment of anxiety and depression in general practice: a cluster randomised controlled trial. Implement Sci. 2015;10:33.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Jäger C, Freund T, Steinhäuser J, Stock C, Krisam J, Kaufmann-Kolle P, et al. Impact of a tailored program on the implementation of evidence-based recommendations for multimorbid patients with polypharmacy in primary care practices-results of a cluster-randomized controlled trial. Implement Sci. 2017;12(1):8.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital-based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(1):36.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Hagedorn HJ, Heideman PW. The relationship between baseline Organizational Readiness to Change Assessment subscale scores and implementation of hepatitis prevention services in substance use disorders treatment clinics: a case study. Implement Sci. 2010;5(1):46.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Sweet SN, Ginis KAM, Estabrooks PA, Latimer-Cheung AE. Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships. Implement Sci. 2014;9(1):74.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: A systematic review of use over time. Am J Public Health. 2013;103(6):e38-46.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Glasgow RE, Dickinson P, Fisher L, Christiansen S, Toobert DJ, Bender BG, et al. Use of RE-AIM to develop a multi-media facilitation tool for the patient-centered medical home. Implement Sci. 2011;6(1):118.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Stolldorf DP, Mixon AS, Auerbach AD, et al. Implementation and sustainability of a medication reconciliation toolkit: a mixed methods evaluation. Am J Health-Syst Pharm. 2020;77(14):1135–43. https://doiorg.publicaciones.saludcastillayleon.es/10.1093/ajhp/zxaa136.

  31. Stolldorf DP, Havens DS, Jones CB. Sustaining innovations in complex health care environments: a multiple-case study of rapid response teams. J Patient Saf. 2016. https://doiorg.publicaciones.saludcastillayleon.es/10.1097/PTS.0000000000000239. Epub ahead of print.

  32. Schnipper JL, Nieva HR, Mallouk M, et al. Effects of a refined evidence-based toolkit and mentored implementation on medication reconciliation at 18 hospitals: results of the MARQUIS2 study. BMJ Qual Saf. 2021.

  33. Schnipper JL, Reyes Nieva H, Yoon C, et al. What works in medication reconciliation: an on-treatment and site analysis of the MARQUIS2 study. BMJ Qual Saf. 2023:bmjqs-2022-014806. https://doiorg.publicaciones.saludcastillayleon.es/10.1136/bmjqs-2022-014806.

  34. Stolldorf DP, Ridner SH, Vogus TJ, Roumie CL, Schnipper JL, Dietrich MS, et al. Implementation strategies in the context of medication reconciliation: a qualitative study. Implementation Science Communications. 2021;2(1):63.

    Article  PubMed  PubMed Central  Google Scholar 

  35. de Wit K, Curran J, Thoma B, Dowling S, Lang E, Kuljic N, et al. Review of implementation strategies to change healthcare provider behaviour in the emergency department. Can J Emerg Med. 2018;20(3):453–60.

    Article  Google Scholar 

  36. Hamilton AB, Finley EP. Reprint of: Qualitative methods in implementation research: An introduction. Psychiatry Res. 2020;283:112629.

    Article  PubMed  Google Scholar 

  37. Tabak RG, Khoong EC, Chambers D, Brownson RC. Models in dissemination and implementation research: useful tools in public health services and systems research. Front Public Health Serv Syst Res. 2013;2(1):8.

    Google Scholar 

  38. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95:103208.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  PubMed  Google Scholar 

  40. Helfrich CD, Li Y-F, Sharp ND, Sales AE. Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implement Sci. 2009;4(1):38.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Hawk KF, D’Onofrio G, Chawarski MC, O’Connor PG, Cowan E, Lyons MS, et al. Barriers and facilitators to clinician readiness to provide emergency department-initiated buprenorphine. JAMA Netw Open. 2020;3(5):e204561.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Bowden CF, Worsley D, Esposito JM, Cutler GJ, Doupnik SK. Pediatric emergency departments’ readiness for change toward improving suicide prevention: A mixed-methods study with US leaders. J Am Coll Emerg Physicians Open. 2022;3(5):e12839.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Smelson DA, Yakovchenko V, Byrne T, McCullough MB, Smith JL, Bruzios KE, et al. Testing implementation facilitation for uptake of an evidence-based psychosocial intervention in VA homeless programs: A hybrid type III trial. PLoS ONE. 2022;17(3):e0265396.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  44. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11(1):72.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci. 2019;14(1):1–12.

    Article  Google Scholar 

  47. Kononowech J, Hagedorn H, Hall C, Helfrich CD, Lambert-Kerzner AC, Miller SC, et al. Mapping the organizational readiness to change assessment to the Consolidated Framework for Implementation Research. Implement Sci Commun. 2021;2:1–6.

    Article  Google Scholar 

  48. Van Parys J. Variation in physician practice styles within and across emergency departments. PLoS ONE. 2016;11(8):e0159882.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Chan SL, Lee JW, Ong MEH, Siddiqui FJ, Graves N, Ho AFW, et al. Implementation of prediction models in the emergency department from an implementation science perspective—determinants, outcomes, and real-world impact: a scoping review. Ann Emerg Med. 2023;82(1):22–36.

    Article  PubMed  Google Scholar 

  50. Huntink E, van Lieshout J, Aakhus E, Baker R, Flottorp S, Godycki-Cwirko M, et al. Stakeholders’ contributions to tailored implementation programs: an observational study of group interview methods. Implement Sci. 2014;9:185.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

None.

Disclaimer

The statements in this publication are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee or the official views of the National Institutes of Health. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health. Readers should not interpret any statement in this report as an official position of HHS. None of the authors has any affiliation or financial involvement that conflicts with the material presented in this report.

Funding

This publication was funded through a Patient-Centered Outcomes Research Institute (PCORI) Dissemination and Implementation Award (DI-2020C2-20363) and REDCap usage was supported by NCATS of the National Institutes of Health under award number: UL1 TR000445 from the National Center for Advancing Translational Sciences.

Author information

Authors and Affiliations

  1. Vanderbilt University School of Nursing, Nashville, TN, USA

    Deonni P. Stolldorf & Rachel A. Hilton

  2. Emergency Medicine, Vanderbilt University Medical Center, Nashville, TN, USA

    Alan B. Storrow

  3. Vanderbilt University Medical Center, Nashville, TN, USA

    Karen F. Miller

  4. Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA

    Dandan Liu & Cathy A. Jenkins

  5. Department of Emergency Medicine, Center for Policy and Research in Emergency Medicine, Oregon Health & Science University, Portland, OR, USA

    Joy Kim & Bory Kea

  6. Department of Public Health Sciences, Henry Ford Health, Detroit, MI, USA

    Deepika Boopathy

  7. Department of Emergency Medicine, Henry Ford Wyandotte Hospital, Wyandotte, MI, USA

    Satheesh Gunaga

  8. Emergency Medicine, Henry Ford Health and Michigan State University, Detroit, MI, USA

    Joseph Miller

  9. Emergency Medicine, Vanderbilt University Medical Center and, Veterans Affairs Tennessee Valley Healthcare System, Geriatric Research, Education and Clinical Center (GRECC), Nashville, TN, USA

    Sean P. Collins

Authors
  1. Deonni P. Stolldorf
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Alan B. Storrow
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Dandan Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Cathy A. Jenkins
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Rachel A. Hilton
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Karen F. Miller
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Joy Kim
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Deepika Boopathy
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Satheesh Gunaga
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Bory Kea
    View author publications

    You can also search for this author inPubMed Google Scholar

  11. Joseph Miller
    View author publications

    You can also search for this author inPubMed Google Scholar

  12. Sean P. Collins
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

DPS, SPC, and ABS led the conceptual development of the study and manuscript. DPS, SPC, ABS, DL, and CAJ developed the research design and methodology. DPS and ABS conducted the qualitative and DL and CAJ the quantitative data analysis. RAH and DPS conducted the mixed-methods analysis in collaboration with ABS. DPS and RAH generated the initial manuscript draft. DPS, SPC, ABS, DL, CAJ, KFM, JK, DB, SG, BK, and JM provided insight on the project and additional manuscript drafts. DPS finalized the manuscript. All authors have read and approved the final version of the manuscript.

Corresponding author

Correspondence to Deonni P. Stolldorf.

Ethics declarations

Ethics approval and consent to participate

This mixed-methods study (i.e., surveys and interviews) was approved by the Vanderbilt Institutional Review Board (IRB) (IRB# 211641) and by the local IRBs for each research site. Informed consent was obtained from all survey participants using an IRB-approved informed consent form in REDCap. GUIDED was delivered in the real-world setting as part of usual care for all patients with HF admitted to the emergency department. This study adheres to the EQUATOR network reporting guidelines and research checklist of “Mixed Methods Reporting in Rehabilitation & Health Sciences.”

Consent for publication

None required.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stolldorf, D.P., Storrow, A.B., Liu, D. et al. A mixed-methods observational study of strategies for success in implementation science: overcoming emergency departments hurdles. BMC Health Serv Res 25, 147 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12913-024-12102-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12913-024-12102-9

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us