Improving transitions in care for children and youth with mental health concerns: implementation and evaluation of an emergency department mental health clinical pathway

Abstract

Background

Emergency departments (EDs) are often the first access point for children and youth seeking mental health (MH) and addiction care. However, many EDs are unprepared to manage large volumes of pediatric MH patients. In addition, the fragmented Canadian MH system is challenged in connecting youth seen in the ED for follow-up community services. A provincial Emergency Department Mental Health Clinical Pathway (EDMHCP) for children and youth presenting to the ED with MH concerns was developed to address these challenges. The objective of the current study was to determine if EDMHCP implementation resulted in: (1) pathway use, (2) more patients discharged with MH recommendations, (3) MH service recommendations that aligned with patients’ risk assessments, and (4) changes in service outcomes, including ED length of stay (LOS), revisits, and admissions/transfers.

Methods

We implemented the pathway at four ED sites from 2018 to 2019 using the Theoretical Domains Framework to develop a tailored strategy at each site. We conducted chart reviews retrospectively in 2017–2018 (pre-implementation) and prospectively in 2019–2020 (post-implementation). Non-parametric tests examined differences in service outcomes between the implementation periods.

Results

Pathway use varied widely across sites, ranging from 3.1% at site 4 to 83.0% at the lead site (site 2). More referrals to community MH agencies (p <.001) were made at discharge during post-implementation at the lead site compared to pre-implementation, and mixed results were obtained regarding whether clinicians’ risk assessments aligned with MH service recommendations. LOS significantly increased at the lead site (p <.001) and non-lead sites (sites 1, 3, 4; p =.02) between pre- and post-implementation. Revisits and admissions/transfers did not change significantly at any site.

Conclusion

Implementation was partially successful at the lead site, showing high pathway use and greater referrals to community MH agencies. These findings emphasize the complexity of implementing pathways in various ED settings. Successful implementation requires integration into existing workflows.

Trial registration

ClinicalTrials.gov (NCT02590302). Registered on 29 October 2015.

Contributions to the literature

• We implemented a new clinical pathway in four hospital EDs and community MH agencies. EDs were paired with MH agencies to form coordinated community dyads. We measured pre- versus post-implementation pathway use, referrals, alignment of patient risk assessment with MH recommendations at discharge, and changes in ED utilization.

• Implementation was partially successful at the lead site, indicating that successful implementation of ED pathways requires integration into existing systems and workflows.

• Sustainability of ED pathways may be challenging at sites where specialized staff and infrastructure are not available.

Background

The emergency department (ED) is a common access point for children and youth seeking mental health (MH) and addiction care [1]. Pediatric MH visits to EDs have significantly increased since 2009 in Ontario, Canada [2,3,4]. Over 50% of children and youth use the ED as the first contact for MH concerns without previously seeking outpatient care, with many patients returning to the ED to seek additional care [5].

Although families increasingly rely on EDs for MH care, these settings are often unprepared to manage children and youth with urgent and emergent MH concerns [5]. EDs lack standardized screening tools, clinical resources, and clinicians trained to confidently manage pediatric MH patients [5,6,7]. Adding to this problem is the complex and fragmented MH system in Canada, which creates difficulty in providing a coordinated transition for patients between the ED and community services. The lack of streamlined referral processes to MH resources in the community results in ED clinicians discharging their patients without appropriate recommendations for follow-up care based on their needs [8].

Each of Ontario’s 33 geographical service areas have lead agencies that are responsible for ensuring that core child and youth MH services are available to their respective communities. Lead agencies can directly deliver these core services or work in partnership with other core service providers in their communities to develop and strengthen pathways to care. Pathways between lead MH agencies and EDs have been established in Ontario and are evolving, but many gaps in care remain.

To address these challenges, in 2011, the Provincial Council on Maternal Child Health in Ontario appointed a working group of Canadian hospital and community-based professionals to develop a clinical pathway for children and youth presenting to the ED with MH concerns. Clinical pathways are multidisciplinary care plans outlining the steps for providing standardized, high-quality care for a specific population [9, 10]. Known as the Emergency Department Mental Health Clinical Pathway (EDMHCP; Fig. 1), the EDMHCP is for pediatric patients aged 6–17 presenting with MH or addictions as their primary concern. It provides ED clinicians with a defined set of evidence-based screening tools, a pre-printed order set for chemical restraint use, a documentation tool to summarize the ED visit, and an algorithm to outline the care process from triage to discharge [11]. Development of the EDMHCP has been described elsewhere [11]. This pathway provides flexibility, enabling sites to modify the components that best fit their context. The EDMHCP provides a standardized approach to guide risk assessment and disposition decision-making. It also aims to improve care transitions between the ED, outpatient hospital services, and community mental health and addiction agencies (CMHAs) [7, 11].

Fig. 1

Emergency department mental health clinical pathway algorithm. Legend: ED: Emergency Department; MH/A: Mental Health/Addictions; Screening tools: C/YPS: Caregiver Perception Survey/Youth Perception Survey; ASQ 5: Ask Suicide-Screening Questions; C-SSRS: Columbia Suicide Severity Rating Scale; PSC-17: Pediatric Symptom Checklist-17; GAIN-SS: Global Appraisal of Individual Needs--Short Screener; PPO: Pre-Printed Order set; HEADS-ED: Clinician Screening Tool

Using the Theoretical Domains Framework, a tailored intervention strategy was developed to facilitate pathway implementation. The Theoretical Domains Framework provides an approach to understand behavioural determinants and inform intervention designs that address barriers and facilitators to implementation [12].

Study objectives

The primary objective of this study was to evaluate implementation of the EDMHCP in four ED-CMHA dyads in Ontario. Research questions included whether implementation of the intervention resulted in (1) EDMHCP use; (2) an increased proportion of patients with MH-specific recommendations at discharge, as documented in the medical chart; (3) MH service recommendations that aligned with patients’ risk assessments; and (4) changes in ED length of stay (LOS), admissions/transfers, and 10-day and 3-month ED revisits.

Methods

Setting

The EDMHCP was implemented in four ED and CMHA dyads in Eastern Ontario, Canada (Table 1). To facilitate transitions in care between the ED and community services, we partnered each ED site with their local CMHA to form a coordinated service dyad. The ED sites included (1) a low-volume rural hospital, (2) an urban pediatric hospital, (3) a regional hospital, and (4) an urban community hospital. Implementing the EDMHCP in hospitals and CMHAs with varying patient populations enabled us to evaluate the feasibility of the pathway in different settings.

Table 1 Site characteristics by implementation phase

The EDMHCP was first implemented at Site 1, a small rural hospital with access to a crisis intervention worker. This site had two CMHAs corresponding to two service areas across its region.

Site 2 is a pediatric academic health centre and the lead hospital site for EDMHCP implementation. During the study period, crisis intervention workers in this ED responded to MH-related visits between 7:30 AM and 11:00 PM from Monday to Friday and from 4:00 PM to midnight on weekends. ED physicians assessed patients with MH concerns outside of these hours and during peak times, and were first to assess MH patients requiring medical attention due to substance use or self-inflicted injuries. Site 2 was the only participating site with direct access to an in-house psychiatrist on-call service and a psychiatric inpatient program. Patients from the other three sites could be transferred to this site for psychiatric assessment after consulting with the on-call psychiatrists. The lead CMHA for this service area did not participate in this project; consequently, we engaged several local CMHAs related to this site.

Site 3 is a regional hospital with a 24/7 crisis team that saw MH patients over age 16 and younger patients on weekdays between 10:00 AM and 6:00 PM. Site 4 is a high-volume community hospital with an adult ED-based MH crisis team seeing patients over age 16. A crisis intervention worker was consulted for patients under 16 between 11:00 AM and 7:00 PM. The lead CMHA for this site was located within the hospital.

Table 1 describes the site characteristics in the pre- and post-implementation periods between 2017 and 2020.

Study population & participants

Inclusion criteria for the EDMHCP included (1) patients between 6 and 18 years of age (2) ED presentation during the site’s specific 8-month implementation period, (3) MH chief complaint, and (4) proficiency in English or French. Exclusion criteria included (1) triage designation requiring resuscitation, (2) medical instability, (3) need for intensive care, or (4) patients directly admitted to the hospital for medical management and observation. Patients who required medical attention were eligible for the pathway once they were medically stable.

Intervention strategy

Seven core components were integral to tailoring and implementing the pathway at each site:

• 1. Local site champions. Each ED and CMHA had several champions to promote and maintain pathway use. Champions included an ED physician, crisis intervention worker, nurse, or nurse educator at the hospitals and a clinician or administrator/Executive Director at each CMHA.

• 2. Hospital and CMHA commitment. Commitment from all hospitals and CMHAs to implement the EDMHCP was established. Sites signed a memorandum of agreement for partnerships and sharing documents, and CMHAs wrote letters of support.

• 3. Site visits. The research team visited each ED and CMHA at the start of implementation to assess readiness to implement the pathway and its fit and acceptability in each setting. Visits included an assessment of the intake and flow of pediatric patients, the role of health care providers, and infrastructure requirements. A structured form guided visits to each ED and CMHA.

• 4. Ongoing site support. Ongoing support was provided throughout both implementation phases. This involved monthly communication with site champions during implementation to support progress, and bimonthly communication during post-implementation to ensure sustainability. A process log recorded reasons for delays in implementation.

• 5. Education. The research team ensured that ED physicians, nurses, and administrators had the knowledge and skills to implement the pathway. Education included train-the-trainer workshops and training videos on how to use the HEADS-ED risk assessment tool.

• 6. Website support. The HEADS-ED website presented resources tailored for each hospital site.

• 7. Posters/reminders. Visual tools such as posters with the EDMHCP algorithm were developed and displayed in EDs to serve as reminders to use the pathway and assist with discharge dispositions to CMHAs. Posters included information about the CMHAs (e.g., referral eligibility criteria, operating hours, and phone numbers).

In addition to these core components, the Theoretical Domains Framework was used to further guide and tailor the intervention strategy. This was accomplished by assessing organizational readiness and relevant factors for change through key informant interviews guided by the Theoretical Domains Framework. Current and potential barriers and facilitators to implementation were identified through these key informant interviews at each site, and specific tailoring strategies were adopted to address these barriers and leverage facilitators.

Data collection

The Research Ethics Board approved this study at the coordinating hospital (REB #15/146X). Secondary ethics approval was obtained at the three other participating hospitals. The lead agencies and core service providers also provided letters of support to participate in the EDMHCP. This trial was registered on ClinicalTrials.gov (NCT02590302).

The EDMHCP was implemented in a staggered process in four ED-CMHA dyads (Fig. 2), with an 8-month implementation period designated for each site. A prospective chart review of all eligible patients was conducted to evaluate pathway implementation. We reviewed records for each site’s 8-month pre- and post-implementation periods. Of note, the post-implementation period began after a 3-month settling-in period to ensure the intervention activities were complete and not affecting the evaluation. The pre- and post-implementation periods were matched by time of year to control for seasonal variability in ED presentations. Sites three and four were affected by the COVID-19 pandemic during their post-implementation phase.

Fig. 2

Implementation timelines for each site

Identifying mental health presentations

Patients were identified by the following MH Chief Complaints as provided by the triage nurse in the ED: Depression/Suicidal/Deliberate Self Harm; Anxiety/Situational Crisis; Bizarre Behavior; Hallucinations/Delusions; Insomnia; Pediatric Disruptive Behaviour; Social Problem; or Violent/Homicidal Behaviour. Other Chief Complaints were also examined to determine if the visit was primarily related to a MH concern, including Laceration/Puncture, Anorexia, Concern for Patient’s Welfare, Hyperventilation, Minor Complaints Unspecified, Overdose/Ingestion, Substance Misuse/Intoxication, and Substance Withdrawal.

We grouped chief complaints into five categories for the analysis, based on the categories defined by Newton et al. [13] Mood Disorders included Depression/Suicidal/Deliberate Self Harm presentations. Anxiety/Stress-Related Disorders included Anxiety/Situational Crisis and Hyperventilation. The Substance/Alcohol Use category involved presentations for Overdose/Ingestion, Substance Misuse/Intoxication, and Substance Withdrawal. Finally, Schizophrenia and Other Psychotic Disorders included Hallucinations/Delusions and Bizarre Behavior presentations. The Behavioural or Emotional Disorders group contained all other presentations (i.e., Pediatric Disruptive Behaviour, Violent/Homicidal Behaviour, Insomnia, Social Problem, Anorexia, Concern for Patient’s Welfare, Laceration/Puncture, Behavioural Issues/Aggressive).

A study coordinator trained two research assistants to extract information from patients’ electronic medical records and input this into REDCap, an online database [14]. We designed a REDCap data collection form to record patients’ demographic and clinical characteristics, the degree to which the site completed each pathway component, disposition, and follow-up recommendations. The form also captured patients’ current MH services and subsequent ED visits, admissions, and transfers for MH concerns.

We developed a data dictionary to ensure standardized data collection between research assistants. The study coordinator and all research assistants abstracted 10% of the records to assess the inter-rater agreement on critical variables. The percentage agreement between raters was 91%.

Screening tools

As per the EDMHCP, patients were assessed with several screening tools. The triage nurse completed the Ask Suicide-Screening Questions (ASQ) [15] or the Columbia Suicide Severity Rating Scale (C-SSRS) [16], which was used at site 3. Parents whose children were under 12 completed the Caregiver Perception Survey (CPS) [17] and the Pediatric Symptom Checklist-17 (PSC-17) [18]. Patients over 12 completed the Youth Perception Survey (YPS) [17] and the Global Appraisal of Individual Needs Short Screener (GAIN-SS) [19]. Clinicians conducted risk assessments using the HEADS-ED [20].

Outcome measures

EDMHCP use

We measured use as the proportion of patients on whom the pathway was used. EDMHCP use is categorized as fully, partially, or not completed. The pathway was fully complete if the clinician completed the HEADS-ED and (i) for patients aged 6–11, the caregiver completed the CPS and PSC-17 screening tools; or (ii) for patients aged 12–17, the triage nurse administered the ASQ or C-SSRS, and the youth completed the YPS and GAIN-SS screening tools. The pathway was partially completed if at least one component was done. The pathway was considered incomplete if the patient did not receive any screening tools or assessments. For the analysis, use was reported overall, not separated by age.

MH recommendations and referrals

For patients not requiring immediate referral for potential admission, MH recommendations included any of the following: primary care (i.e., family doctor or pediatrician), secondary care (i.e., psychologist, psychiatrist, social worker, counselor), hospital outpatient services, urgent care clinics, any community MH services, crisis lines, information and strategies (e.g., apps, workbooks, information sheets), and other (e.g., peer support programs, shelters/housing, recreational programs, mobile intervention services, respite workers and service coordinators). We also evaluated the proportion of referrals to the specific CMHAs for given sites that were part of the pathway. The clinician could either make a direct CMHA referral through Epic, the electronic medical record system, or recommend self-referral to the agency or service.

Alignment of risk assessment to MH service recommendations

The HEADS-ED psychosocial screening tool was used for risk assessment [20]. This tool identifies seven areas of need: Home, Education, Activities and Peers, Drugs and Alcohol, Suicidality, Emotions, Thoughts, Behaviours, and Discharge Resources. The recommended level of action is based on one of three categories: no action needed (0), action needed but not immediately (1), and immediate action required (2). The HEADS-ED recommends that patients with a total score of 8 or greater or with a suicidality score of 2 receive a consultation from a MH clinician (e.g., a psychiatrist) and possibly inpatient admission.

Service outcomes

LOS in the ED was calculated in hours (from triage time to discharge). ED revisits and transfers/admissions were coded as any MH-related visit/transfer/admission within ten days from the index visit. Three-month revisits were visits within 11–90 days of the index visit.

Statistical analysis

The data were analyzed using R statistical software version 4.1.2 [21]. and MedCalc version 19.4 [22]. The Mann-Whitney test was used to determine whether patient age differed between pre- and post-implementation, and Fisher’s exact test (for frequency counts below 20) was used to determine differences between all other patient characteristics between pre- and post-implementation. We also conducted chi-squared tests to examine differences in the proportion of MH recommendations and referrals between the pre- and post-implementation periods. Finally, Mann-Whitney and Fisher’s exact tests examined differences in service outcomes between pre- and post-implementation.

We removed records involving patients who presented to the ED but left without being seen (LWBS) or registered (LWBR). Patients who were assessed but left against medical advice (LAMA) were excluded from the analysis on the alignment of risk assessment to MH services but included in all other analyses.

As non-lead sites (i.e., sites 1, 3, and 4) had a low number of completed pathways, data for documented MH recommendations, alignment of patient acuity to MH service recommendation, and service outcomes were combined across these sites.

Results

Demographic characteristics

Table 2 presents patient demographics and clinical characteristics at ED sites during the pre- and post-implementation periods. Several patient characteristics differed before and after pathway implementation. Patient age (p <.001) and the number of patients assigned to each chief complaint category (p <.001) were significantly different between the pre- and post-implementation periods at site 2. Site 2 also had more patients admitted involuntarily (i.e., on a Form 1) during post-implementation compared to pre-implementation (p <.001). The number of patients within each triage level changed significantly from pre- to post-implementation at site 2 (p <.001) and site 4 (p <.01). More MH patients required medical treatment in the post-implementation period at sites 3 (p <.001) and 4 (p <.05) compared to the pre-implementation period. Finally, patient disposition significantly differed between each implementation period at site 2 (p <.05) and site 4 (p <.01).

Table 2 Patient demographic and clinical characteristics

Pathway use

The proportion of completed pathways during the post-implementation period for each hospital site is presented in Table 3. EDMHCP use varied across hospital sites. Sites 1 and 4 had no pathways fully completed, and only a few were partially completed at each site (6.5% and 3.1%, respectively). Most pathways were at least partially completed at site 3 (60.2%). Site 2 had the highest pathway use, with 83% of pathways being at least partially completed.

Table 3 Pathway use during post-implementation at each hospital site

Documented MH recommendations

Table 4 presents the proportions of patients from each site with a MH recommendation or referral to a CMHA. Most patients at all sites received a MH recommendation at discharge during the pre- and post-implementation phases. Only site 2 made significantly more referrals to CMHAs post-implementation (p <.0001).

Table 4 Proportion of patients with documented MH recommendations and referrals to participating CMHAs and who they were provided by

Sites differed regarding which provider documented MH recommendations and referrals at discharge. Over half of the patients at site 2 were given recommendations/referrals by a MH clinician (crisis intervention worker or psychiatrist) or ED physician in both implementation periods, whereas most patients at sites 1, 3, and 4 received a recommendation/referral from an ED physician.

Alignment of patient acuity to MH service recommendations

Table 5 presents the alignment of patients’ HEADS-ED scores to psychiatric consultations and disposition. Of the patients with a total HEADS-ED score equal to or greater than 8 or a suicidality score of 2, 57.5% at site 2 and 45.5% at all other sites combined received a psychiatric consult, as recommended by the EDMHCP. Of these patients who received a consult, approximately one-third (31.9%) at site 2 were admitted, whereas most patients at other sites (60%) were transferred to site 2.

Table 5 Alignment of patient acuity to MH service recommendations

Of the patients who scored less than 8 on the HEADS-ED with a suicidality score less than 2, 45.9% at site 2 and 16.7% at other sites still received a psychiatric consultation. Most of these patients (93.2%) at site 2 were discharged home afterwards, whereas one-third (33.3%) at other hospital sites were discharged home. Almost all patients who did not receive a consult were discharged home, as recommended.

Service outcomes

LOS

LOS significantly increased overall between pre- and post-implementation at the lead site (p <.001) and non-lead sites (p =.02) (Table 6). Within the post-implementation period, patients at the lead site with either a partially or fully completed pathway had a significantly shorter ED LOS than patients with non-completed pathway (p <.001). However, patients at the non-lead sites had a significantly longer LOS when at least some pathway components were completed (p =.04).

Table 6 Pre- and post-implementation service outcomes by site

ED return visits

There were no significant changes in the number of times patients returned to the ED within 10 days or 3 months at any site between pre- and post-implementation or related to pathway completion.

Hospital admissions or transfers

We found no significant differences between the two implementation periods for transfers and hospital admissions within 10 days or 3 months of visiting the ED at any site. The pathway having been fully, partially, or not completed did not affect hospital admissions or transfers within 10 days or 3 months.

Discussion

We developed the EDMHCP to help address system gaps in MH care for children and youth [7, 11]. This study aimed to evaluate the implementation of the EDMHCP in four ED-CMHA dyad settings in Ontario. The EDMHCP was implemented in multiple ED settings that differed in type (i.e., specialized versus general), patient volume, and pediatric census. Thus, we could determine the generalizability of the EDMHCP and factors that may have affected its implementation and sustainability across different settings. The Theoretical Domains Framework was used to guide pathway implementation through the identification of barriers and facilitators at each site. A tailored intervention strategy at each site was developed, and the pathway was flexible to allow sites to modify pathway components to suit their unique context. For example, the pathway used the ASQ; however, site 3 had already implemented the C-SSRS. As their staff were already familiar with the C-SSRS, the pathway was adapted at this site to allow its use in place of the ASQ.

Several patient characteristics differed between the pre- and post-implementation periods. At site 2, patients were slightly younger post-implementation, and a higher proportion of patients presented to the ED on a Form 1 (i.e., a requirement to undergo an assessment from a psychiatrist [23]). Fewer patients presented with unspecified minor complaints, though the reason for this change is unclear and unlikely to have affected study results. At sites 3 and 4, more patients required medical treatment post-implementation. These differences in patient characteristics may reflect broader MH trends, where children and youth are increasingly having more complex MH needs at younger ages [4]. The increase in MH referrals at site 2 may be linked to these greater MH needs, potentially contributing to the higher referral rates post-implementation. This may also explain the longer LOS observed at both lead and non-lead sites. Although triage level and disposition were significantly different pre- and post-implementation at sites 2 and 4, we did not analyze which specific triage level or dispositions differed, so we are not able to determine how this may have impacted study results.

Sustained pathway use varied across ED sites. Only one site made significantly more MH recommendations and referrals post-pathway implementation. Additionally, there were few changes in service outcomes. LOS increased from pre- to post-implementation at the lead and non-lead sites, which may have been a secular trend for MH patient’s LOS but could be attributed to pathway use. Of note, LOS decreased at the lead site when the pathway was at least partially completed compared to when it was not completed. However, LOS significantly increased at non-lead sites when at least some pathway components were completed. These findings highlight the difficulty of sustained pathway use in complex settings such as EDs and reflect the multiple barriers and facilitators to implementation across different settings.

Our results revealed that the post-implementation use of pathway components also varied among sites, with two sites not continuing to follow the pathway after its implementation. Pathway use was greatest at site 2, where the pathway was at least partially completed for most patients. Site 3 also had an acceptable use rate of 60%. Pathway use at sites 1 and 4 was low, where only a handful were at least partially completed. Several factors may have contributed to the variability in use across sites. Site 2 integrated some of the tools (i.e., ASQ, HEADS-ED) into the electronic medical record for the EDMHCP implementation and made them mandatory, which may explain why the site had a higher completion rate. Clinicians were already familiar with these tools, and having them embedded electronically increased accessibility and ease of completion. Clinicians at other sites had to use paper copies of each pathway component and were less familiar with these tools. This additional paperwork may have increased the clinicians’ perceived workload, contributing to the lower completion rate [24]. These findings are essential to consider in implementing future clinical pathways in ED settings, suggesting that use may be greater if the site integrates pathway components into the ED’s current workflow and electronic medical record.

Other barriers may have contributed to the low pathway use, particularly at sites 1 and 4. These sites tend to see few pediatric MH patients, leading to infrequent opportunities to use and become accustomed to the pathway, which has been an identified barrier in other pathway implementation studies [25]. Since both sites are near site 2, a specialized pediatric hospital, clinicians may have opted to transfer MH patients directly to site 2 for care without using the pathway. Other factors potentially impeding EDMHCP use include the busyness of the ED, lack of time and resources to complete pathway components, competing priorities in the ED, and lack of buy-in from clinicians [25, 26]. Key informant interviews were conducted at all sites post-implementation using the Theoretical Domains Framework to better understand the barriers and facilitators to implementation. The qualitative findings from these interviews will be presented in a subsequent paper.

Clinicians at site 2 made significantly more CMHA referrals after the site implemented the EDMHCP; however, no changes were observed at other hospital sites. During the implementation period at site 2, referrals to CMHAs could be made directly using the electronic medical record. In contrast, other sites completed referrals using paper forms faxed to the given agency. Electronic referrals facilitated this process, possibly contributing to more referrals during the post-implementation period. Similar to screening tools, these findings further suggest that integrating components of the pathway into existing systems facilitates its use.

Furthermore, over half of the patients at the lead hospital site were given MH recommendations and referrals by a MH clinician (i.e., a crisis intervention worker or psychiatrist), whereas most patients at other sites received discharge instructions from an ED physician. ED physicians may not be as knowledgeable about MH services as social workers and psychiatrists, specifically trained to manage patients with MH concerns [27]. Social workers, in particular, have an essential role in providing MH care in ED settings, such as crisis counselling and making appropriate discharge recommendations, given their knowledge of MH services [27, 28]. However, general EDs are less likely to employ pediatric-trained social workers than larger pediatric EDs [27]. Further, discharge recommendations have been shown to differ between EDs depending on whether specialized MH clinicians, such as psychiatrists, are available on-site [29]. The availability of crisis intervention workers at site 2 may have also contributed to the greater proportion of documented MH recommendations and referrals.

Mixed results were obtained regarding whether clinicians’ risk assessments aligned with the pathway’s MH service recommendations. Only about half of the patients with a HEADS-ED total score greater than 8 or a suicidality score of 2 received a psychiatry consult as the tool recommended. About a third of these patients at the lead hospital site were admitted, a similar admission rate to previous reports of pediatric patients who received a psychiatric consultation [20]. There may be circumstances where a psychiatric consultation was still warranted despite having a low total HEADS-ED or suicidality score. For example, other factors may affect perceived risk among clinicians leading to these referrals, or the involvement of a psychiatrist may be necessary to help clinicians determine the most appropriate disposition for the patient.

Changes in service outcomes included increased LOS at the lead and non-lead sites during the post-implementation period. LOS decreased at the lead site when the pathway was at least partially completed compared to not completed but increased at the other sites when the pathway was at least partially completed. Several variables may have contributed to these differences in LOS. For example, LOS during the 8-month post-implementation periods at the non-lead sites may have been affected by the COVID-19 pandemic. Despite fewer patients presenting to the ED, LOS tended to be longer during the pandemic [30]. A shorter LOS at the lead site when the pathway was at least partially completed may indicate that the pathway was effective at this site, given that most patients at site 2 had a partially or fully completed pathway. As pathway tools were not embedded electronically at the non-lead sites, the need to find and complete paper-based tools may have led to increased patient LOS. Conversely, it is also possible that pathway completion differed depending on patient acuity. Patients with greater needs (e.g., on a Form 1) tend to spend more time in the ED than lower acuity patients. Clinicians may not have been able to complete the pathway components for these patients with higher needs, or these patients were being assessed by psychiatrists, who were not responsible for completing the pathway tools. Low acuity patients therefore may have been more likely to have the pathway at least partially completed. These significant findings are difficult to interpret accurately, given the many potential confounds that could have affected LOS during the implementation periods at each site.

No significant differences were found in return visits or transfers within 10 days or 3 months between pre- and post-implementation or when the pathway was at least partially completed compared to not completed. It is important to note that these post-implementation results may have been influenced by the COVID-19 pandemic, as the post-implementation period for sites 3 and 4 overlapped with the beginning of the COVID-19 pandemic lockdown. As a result, clinicians may have been less likely to transfer patients between hospitals, and patients may have been less inclined to present to EDs for MH concerns [31,32,33]. There are also various factors that influence return visits, including demographic and clinical characteristics, as well as MH care access and utilization. Therefore, addressing return visits is a complex task that requires consideration of these factors [5].

Limitations

There are several limitations to this study. The post-implementation periods at sites 3 and 4 occurred during the early phase of the COVID-19 pandemic. This may have contributed to the low number of presentations and may have affected pathway use and service outcomes because of competing priorities in the ED [31,32,33]. Given the small sample sizes at sites 1, 3, and 4, we chose to aggregate all non-lead sites. However, site 3 partially implemented the pathway. Although the sample size was likely too small to detect significant findings, analyzing this site separately may have provided insight into whether the pathway’s partial implementation affected MH recommendations and referrals, alignment of patient acuity to service recommendations, and service outcomes. The data were also collected through chart reviews, resulting in shortcomings such as incomplete documentation, difficulty interpreting information (e.g., acronyms, handwriting), and inconsistent quality of information documented by clinicians. While these are common documentation issues when conducting chart reviews [34], it was difficult to determine whether information such as discharge recommendations were provided but not documented in the chart. Documentation and referrals also varied between ED sites, where smaller EDs often had missing or less detailed information in their medical charts. This difference between ED sites may have been due to the implementation of a new electronic medical record system at site 2 prior to their post-implementation period, potentially leading to improved documentation and simplifying the referral process, as opposed to the other three sites where documentation was paper-based [35]. The integration of an electronic medical record system appeared to have a positive impact on clinical documentation and completion of pathway components.

Conclusion

The aim of the EDMHCP was to guide risk assessment and disposition decision-making, and to improve care transitions between the ED, outpatient MH care at CMHAs and hospital-based services. Implementing a novel tool such as the EDMHCP can be challenging, especially in busy and complex ED settings. We found that the lead site showed high pathway use and greater referrals to community MH agencies after implementation. The partial success of implementation at the lead site may be attributed to the availability of specialized MH staff as well as the electronic medical record, which facilitated referrals, completion of some of the pathway tools, and clinical documentation. The absence of comparable infrastructure and staffing at other sites may have been barriers to implementation. Our findings suggest that successful adoption of ED pathways requires integration into existing systems and workflows.