Screening and care for emotional and cognitive problems after stroke: a prospective mixed-methods process evaluation of a cluster-randomized controlled trial

Abstract

Background

Emotional and cognitive problems are prevalent after stroke. We developed an intervention including screening for emotional and cognitive problems, screening for participation restrictions, self-management support including education and, if needed, referral to rehabilitation services. This intervention was delivered through a nurse-led outpatient consultation at six weeks after stroke. Its effectiveness was examined in a patient-blinded, cluster-randomized controlled trial and its feasibility in a process evaluation. This process evaluation aims to (1) assess the reach, dose and fidelity of the intervention, (2) evaluate barriers and facilitators to implementation and (3) explore experiences from the perspective of healthcare providers.

Methods

A mixed-methods study was conducted. Specialized nurses offering the intervention completed registration forms to evaluate the reach, dose and fidelity of the intervention. The Barriers and Facilitators Instrument (BFAI) was used to evaluate barriers and facilitators. Furthermore, the specialized nurses participated in a group interview to explore experiences in delivering the intervention. Group interviews were analysed using inductive thematic analysis.

Results

In total, 262 of 264 patients (99.2%) attended the intervention. In 233 patients (88.9%), all screening instruments were completed. The BFAI demonstrated that the intervention was feasible and was not considered time-consuming or hindered by contextual factors. Barriers to deliver the intervention were limited to patient characteristics. During group interviews, specialized nurses reported that the intervention significantly overlapped with care as usual prior to start of the trial, yet indicated that the intervention provided a more profound understanding of post-stroke sequelae.

Conclusions

The implementation of a nurse-led intervention, comprising screening and care for emotional and cognitive problems after stroke, was generally well executed and deemed feasible.

If this intervention proves to be clinically and/or cost-effective in our upcoming primary analyses, we will advocate to incorporate it in daily clinical practice and to adhere to the implementation strategy as presented in this process evaluation.

Trial registration

This study was registered at 25 September 2018 in the Netherlands Trial Register (NL7295).

Introduction

After an ischemic stroke, a significant proportion of patients achieve a favorable functional outcome, maintaining independence and can therefore be discharged home directly after hospitalization. Despite this favorable functional outcome, these patients often experiences emotional or cognitive problems which hamper societal participation and reduce quality of life [1,2,3,4,5]. Patients who are discharged home directly generally have limited follow-up care. As a result, emotional and cognitive consequences may go undetected and untreated. Consequently, international guidelines advise to screen for these problems, especially in the early phase, such as during hospital admission [6, 7]. Nevertheless, robust evidence regarding its effectiveness is lacking [8].

In the multicenter, patient-blinded, cluster-randomized controlled ECO-stroke trial, the effectiveness of screening and subsequent patient-tailored care for emotional and cognitive problems after stroke (ECO-stroke trial) was examined, aiming to optimize referral to rehabilitation and other services such as physiotherapy, occupational therapy and speech therapy [9]. Patients discharged home after a stroke without outpatient rehabilitation were included. A detailed trial protocol has been published previously [9]. The ECO-stroke trial investigated whether this intervention is successful in terms of clinical effectiveness (i.e. primary outcome being the level of societal participation at one year post-stroke) and cost-effectiveness (i.e. from a societal perspective). The results of this trial are yet to be published. These publications, however, will neither demonstrate whether the intervention was delivered as intended, nor will it provide insight in experiences of healthcare providers offering the intervention. If an intervention is poorly conducted, an underestimation of its effectiveness is to be expected. Moreover, as the study of healthcare providers’ experiences might reveal barriers or facilitators to deliver the intervention, valuable lessons can be obtained for implementation in daily practice.

This paper describes the process evaluation that was conducted as part of the ECO-stroke trial. In accordance with the Medical Research Council (MRC) guidance a process evaluation includes the following components [10]: whether the intended target audience was reached (reach of the intervention); the quantity of the intervention implemented (dose of the intervention), i.e. duration of the intervention; if the intervention was delivered as intended (fidelity to the protocol of the intervention); and contextual factors that might have a negative or positive effect on the implementation of the intervention (context), i.e. barriers and facilitators.

Aims

Building upon the aforementioned framework, the aim of this process evaluation was to (1) measure the reach, dose and fidelity of the ECO-stroke intervention, (2) evaluate barriers and facilitators to implement the intervention and (3) explore experiences with the intervention from a healthcare provider’s perspective.

Methods

Design

This process evaluation was prospectively conducted alongside the ECO-stroke trial. The ECO-stroke trial was a multicenter, patient-blinded, cluster-randomized controlled trial (RCT) that investigates screening and subsequent patient-tailored care for emotional and cognitive problems in comparison to care as usual, with a follow-up of one year [9]. Patients aged 18 years or older, with an ischemic stroke who were discharged home without inpatient rehabilitation or treatment at an outpatient rehabilitation clinic were included in the ECO-stroke trial. Exclusion criteria included transient ischemic attack (TIA), defined as signs and symptoms lasting less than 24 h and not accompanied by ischemic lesions in the corresponding vascular territory on CT or MRI; hemorrhagic stroke; serious comorbidities with a progressive course (e.g., cancer); serious comorbidities likely to interfere with outcomes (e.g., a psychiatric disorder requiring supervision by a psychiatrist); mild cognitive impairment or dementia; a life expectancy of less than six months; inability to understand questionnaires, based on clinical judgement (for example due to insufficient command of Dutch or aphasia) and legal incompetence, on the basis of clinical judgment.

This process evaluation used a mixed-methods approach, combining quantitative data (i.e. questionnaires) and qualitative data (i.e. group interviews). Specialized nurses offering the intervention were included as participants in this process evaluation. These specialized nurses registered information in the electronic records, completed a questionnaire and participated in a group interview after performing the intervention at least ten times. An overview of the aims and measurements per aim are shown in Table 1.

Table 1 Measurements

ECO-stroke intervention

The ECO-stroke intervention consisted of a face-to-face consultation of approximately 60 min at the outpatient clinics of the neurology department in addition to usual care. Consultation was performed at six weeks after the index event. The consultation was executed by a specialized nurse, nurse practitioner or physician assistant experienced in outpatient stroke care; referred to as ‘specialized nurse’. Usual care was provided as per protocol from the participating hospital, which was different for each participating center. In general, it included at least one consultation of at least half an hour at the outpatient neurology clinic, including topics on secondary prevention, physical outcome and emotional and cognitive complaints.

The intervention included the following components:

• 1. a structured screening for emotional and cognitive problems using sensitive instruments: the Checklist for Cognitive and Emotional Consequences following stroke (CLCE-24), the Montreal Cognitive Assessment (MoCA) and the Hospital Anxiety and Depression Scale (HADS) [11,12,13].

• 2. a structured screening for restrictions in participation by the Restriction subscale of the Utrecht Scale for Evaluation of Rehabilitation on the level of Participation (USER-P-R) [14].

• 3. self-management support consisting of the following elements: (a) providing timely and individualized active oral and written information about stroke and its possible emotional and cognitive consequences based on the screening results, (b) measuring self-efficacy by the Dutch adaptation of the General Self-Efficacy Scale (GSES), (c) achieving shared decision-making with regard to the follow-up care and (d) providing contact details for possible additional questions or problems [15]. Results from the GSES questionnaire were taken into account when determining the follow-up plan; for example, a low level of patient-reported self-efficacy could be a reason to schedule an additional follow-up appointment to create a safety net for potential future issues.

• 4. a decision tool for referral to rehabilitation services based on the results of the screening instruments.

The HADS, USER-P-R and GSES were completed by the patients at home prior to the intervention consultation. The CLCE-24 and MoCA were completed by the specialized nurse during the intervention.

A description of the development of the intervention and the implementation strategy used to implement the intervention in the ECO-stroke trial is described in Supplementary Appendix 1.

Healthcare providers offering the intervention

The intervention was performed by six specialized nurses in six hospitals which were randomly allocated to the intervention group. All specialized nurses worked at the outpatient neurology clinics at different ECO-stroke trial hospitals and all had extensive experience providing stroke aftercare before participating in the RCT. The specialized nurses had, on average, 10 years (ranging from 3 to 15 years) of experience working at the stroke aftercare outpatient clinic. As part of the RCT, these specialized nurses received a training in the intervention, described in detail in Supplementary Appendix 1.

Data sources

Table 1 presents an overview of the aims and the related measurements used in this process evaluation.

Reach, dose and fidelity

The reach of the intervention was defined as the number of patients who actually received the intervention, divided by the target audience, i.e. the total number of patients included in the intervention group. These data were registered in an electronic case report form (eCRF).

The dose of the intervention was defined as the duration of the intervention consultation, which was registered by the specialized nurse in the eCRF.

The fidelity of the intervention was defined as the actual adherence to the original protocol of the intervention, thereby measuring the delivery of several components of the intervention. The following six measurable components of the intervention were examined for fidelity and registered by the specialized nurses in the eCRF:

• 1. the timing of the intervention (the protocol prescribed that timing of the intervention should be within 4–8 weeks after the ischemic stroke);

• 2. if the patient completed and brought the required patient-reported screening questionnaires (HADS, USER-P-R and GSES) to the intervention;

• 3. if the screenings instruments CLCE-24 and MoCA were completed during the consultation;

• 4. if individualized information was provided;

• 5. if shared decision-making took place between the patient and the specialized nurse according to the specialized nurse;

• 6. the duration of the intervention.

Barriers and facilitators

Contextual factors that might affect implementation were measured by the Barriers and Facilitators Assessment Instrument (BFAI) [16]. The BFAI is a 27-item questionnaire evaluating barriers and facilitators divided into four categories: innovation characteristics, care provider characteristics, patient characteristics and characteristics of organizational, social, political and societal context. The answers are rated on a 5-point Likert scale from ‘fully disagree’ to ‘fully agree’ and can be combined into percentages of respondents who ‘(fully) disagree’, ‘do not agree nor disagree’ and ‘(fully) agree’. After the specialized nurse delivered the intervention to at least ten patients and prior to the group interview, the BFAI was completed. In addition, barriers and facilitators were derived from the group interviews, as described below.

Specialized nurses’ experiences

Group interviews were held to explore and comprehend experiences of specialized nurses with the the intervention. Moreover, nurses’ opinions on perceived barriers and facilitators to implementation were explored. Last, the patients’ experiences, as perceived by nurses, were examined. All six specialized nurses who performed the intervention within the ECO-stroke trial were invited to participate. One specialized nurse did not participate because of retirement. Two group interviews were held to include all specialized nurses in the interview. The first interview was held in May 2023 and included three specialized nurses. The second interview took place in August 2023 and included two specialized nurses. Both group interviews were semi-structured and guided by a (non-pilot-tested) topic list encompassing the following topics: general impression of the intervention, healthcare providers’ experiences, patients’ experiences from the perspective of the specialized nurses, user-friendliness, fidelity to the intervention protocol and contextual factors. The topic list was created by an independent researcher (EF), master student in neuropsychology without experience in qualitative research, and revised by a postdoctoral researcher with extensive experience in qualitative research and conducting semi-structured interviews (MS). The topic list was also reviewed by another researcher (JS), PhD candidate and resident in neurology, involved in the development and conduction of the RCT. The group interviews were chaired by EF under supervision of MS. Both researchers, EF and MS, were not involved in the development of the intervention or execution of the RCT and did not have a pre-existing relationship with the specialized nurses participating in the group interviews. The participants had no prior knowledge of the interviewers. The group interviews were hosted and recorded during an online video-call using Zoom and lasted approximately two hours each. The interviews were transcribed verbatim (EF); transcripts were not returned to the specialized nurses for comments or corrections. The COREQ checklist was utilized to ensure a comprehensive and transparent reporting of the qualitative analysis [17].

Ethics

This study was conducted according to the principles of the Declaration of Helsinki (2004). The ethics committee of Medical research Ethics Committees United (MEC-U) in Nieuwegein, the Netherlands, concluded that the ECO-stroke trial was not subject to the Medical Research Involving Human Subjects Act (reference number: W18.169). Approval was obtained from the local medical ethics committees of all participating hospitals (Adrz, BovenIJ Ziekenhuis, Gelre ziekenhuizen Apeldoorn, Het Van Weel-Bethesda Ziekenhuis, Laurentius Ziekenhuis Roermond, Maasstad Ziekenhuis, Noordwest Ziekenhuisgroep locatie Alkmaar, Noordwest Ziekenhuisgroep locatie Den Helder, OLVG, Tergooi MC, Ziekenhuis Amstelland, Zuyderland Medisch Centrum Heerlen). All patients included in the ECO-stroke trial provided informed consent. Informed consent was obtained from all specialized nurses for the participation in the study. Data were handled in accordance with the EU General Data Protection Regulation.

Data-analysis

Quantitative data

Reach, dose and fidelity of the intervention and the BFAI were analysed using descriptive statistics. Descriptive statistics were conducted in IBM SPSS Statistics for Windows, Version 22.

Qualitative data

Transcripts from the group interviews were analysed by one researcher (JS) using inductive thematic analysis [18]. Analyses were performed using Atlas.ti version 9.1.3. After reading the transcripts thoroughly to become familiar with the data, open coding was applied, i.e. segments of data were labeled with codes that captured the meaning of that segment. Codes were combined into themes and subthemes. Illustrative quotes were selected and translated by the same researcher (JS).

Results

Reach, dose and fidelity

In the ECO-stroke trial, 264 patients were included by six hospitals allocated to the intervention. Of the 264 patients allocated to the intervention, 262 (99.2%) received the intervention. Two (0.8%) did not receive the intervention: one patient erroneously visited the outpatient clinics too early and refused an additional appointment and one patient was not able to visit the outpatient clinic. Due to the unforeseen consequences of the COVID-19 pandemic, seven (2.7%) patients completed the intervention by phone.

Table 2 illustrates an overview of measurements of the dose and fidelity to the intervention protocol. The HADS, GSES and USER-P-R were completed at home and brought to the intervention by 241 (92.0%) patients. The CLCE-24 was completed during the intervention in 260 (99.2%) patients and the MoCA was completed in 253 (96.6%). The MoCA was not administered in seven (2.7%) patients, since the intervention was carried out over the telephone instead of a hospital visit because of restrictions during the COVID-19 pandemic. Both questionnaires at home and the screening instruments during the consultation, were completed in 233 (88.9%) patients. The duration of the intervention was approximately 60 min in 132 (50.4%) patients, in accordance with the trial protocol, and within 45 to 75 min in 245 (93.5%) patients. The timing of the intervention was within the planned range of 4 to 8 weeks post-stroke in 245 (93.5%) patients, whereas the intervention was performed prior to 4 weeks in 9 (3.4%) patients and after 8 weeks in 11 (4.2%) patients. The reason for performing the intervention before or after this time window was not recorded. Nine (3.4%) patients were referred to physical therapy, 3 (1.1%) to occupational therapy and 2 (0.8%) to speech therapy. Twenty (7.6%) patient were referred to outpatient rehabilitation treatment and 91 (34.7%) to primary nurse-led stroke care.

Table 2 Dose of and fidelity to the intervention protocol

Barriers and facilitators

Of the six specialized nurses performing the intervention within the ECO-stroke trial, five completed the BFAI (see Table 3). One specialized nurse did not complete the BFAI because of retirement after the inclusion period was completed. In the BFAI characteristics of the intervention, healthcare providers’ characteristics and contextual factors were not identified as barriers. However, two specialized nurses indicated that it was difficult to perform the intervention in patients with a different cultural background and in patients with a low socio-economic status.

Table 3 Results of the barriers and facilitators assessment instrument (BFAI)

During group interviews, some specialized nurses noted that the intervention was not feasible in patients with insufficient command of the Dutch language or patients who were fatigued. Moreover, most specialized nurses initially found it difficult to perform the MoCA, but became proficient in the instrument over time. No other barriers to perform the intervention were identified in the group interviews.

Specialized nurses’ experiences

The following themes were identified analysing the group interviews: (1) overlap between the intervention during the trial and usual care prior to the start of the trial, (2) timing and duration of the intervention, (3) patients’ experiences according to the nurses and (4) the importance of follow-up care after the intervention. Illustrative quotes and an overview of the codes, subthemes and themes are shown in Supplementary Appendix 2 and 3.

• 1. Overlap between the intervention during the trial and usual care prior to the start of the trial

All specialized nurses stated that the intervention had considerable overlap with the usual care they delivered prior to the start of the RCT, but all acknowledged that some new and more comprehensive elements were added. For some specialized nurses, use of the CLCE-24 was new, yet for others it was not. Most nurses indicated they did not use the MoCA before the start of the RCT and stated that using the MoCA in the intervention gave them a more profound understanding of cognitive problems. Also, the specialized nurses mentioned that the intervention offered a more structured and comprehensive assessment of possible emotional and cognitive sequelae. Although the assessment of emotional and cognitive problems in the intervention led to a comprehensive understanding of these sequelae, all of the specialized nurses reported they were already familiar with assessment of such problems and evaluated emotional and cognitive impairment in the majority of their patients. Most of the specialized nurses indicated that due to the intervention they provided written information about emotional and cognitive problems more often. Prior to the RCT, none of the specialized nurses sent questionnaires to patients before the consultation. Some felt that doing so resulted in patients being better prepared for the consultation, which led to a specific questions and discussion of patient-tailored topics.

• 2. Timing and duration of the intervention

The timing of the intervention six weeks after the stroke felt right to all specialized nurses. However, four nurses mentioned the importance of having an additional appointment with patients between hospital discharge and the intervention at six weeks. Most specialized nurses noted that patients have many questions that should be addressed before six weeks after their stroke. All specialized nurses underlined the importance of sufficient time for their consultation and concluded one hour was enough to perform the entire intervention. At the same time, most specialized nurses mentioned that one hour may be too long for some patients who were fatigued. Some noticed that the intervention consultation was more efficient than their usual care, because patients were better prepared after completing questionnaires at home.

• 3. Patients’ experiences according to the specialized nurses

According to the specialized nurses offering the intervention patients’ experiences were generally positive. Three specialized nurses stated that patients specifically appreciated the attention they received. A few specialized nurses noted that some patients found the number of questionnaires too high. In addition, the MoCA could be confronting or shocking to patients according to the specialized nurses.

• 4. The importance of follow-up care after the intervention

All specialized nurses addressed the importance of adequate and long-term follow-up after the intervention. Most mentioned that problems might occur after one year as well, which would also require a safety net. Three of them reported that they referred patients to primary nurse-led stroke care very frequently. Referral to a physiatrist was mentioned by three specialized nurses. One nurse always provided a contact number for patients to reach out to.

Discussion

This process evaluation showed that screening and subsequent patient-tailored care for emotional and cognitive problems after stroke was considered feasible and well implemented. The target population was excellently reached and fidelity to the trial protocol was good. According to specialized nurses who performed the intervention, barriers to implement the intervention were limited to patient characteristics and included patients with insufficient command of the Dutch language, patients who were fatigued and patients with a different cultural or low socio-economic background. No other barriers to implement or perform the intervention were identified. Specialized nurses who performed the intervention were overall positive about the content and clinical feasibility of the intervention. Most of the specialized nurses stated that the intervention had a significant overlap with their care as usual prior to the start of the RCT. Last, specialized nurses emphasized the importance of adequate long-term follow-up for emotional and cognitive problems even after the intervention.

The implementation of screening for emotional and cognitive problems in this trial was good. Previous research determined the following barriers to implement screening for emotional and cognitive problems: limited awareness of screening tools, complexity of screening tools, insufficient time or funding to perform a screening, limited training resources and lack of a pathway if the screening turns positive [19]. In the current study, specialized nurses were trained in using the screening instruments and, as part of the process evaluation, registered if these instruments were completed or not completed. Consequently, we might have reduced the experienced complexity and increased awareness to use screening instruments. Also, specialized nurses had dedicated time during the consultation to perform the intervention as it was designed, checked whether the screening instruments had been completed as intended, and if not, they administered them during the consultation. This undoubtedly contributed to the high completion rates. Moreover, a decision tool was provided to empower the specialized nurses to take action when a screening turned positive. As a result of these interventions, the aforementioned barriers were probably overcome. The good fidelity could also be explained by the finding that most specialized nurses stated to be familiar with many components of the intervention and partly performed aspects of the intervention before the start of the RCT. Although the fidelity to the trial protocol was considered good, one component may have been not adequately adhered to: referral rates to outpatient rehabilitation treatment, physical therapy, speech therapy and occupational therapy were rather low. Referring patients was not the primary aim of the intervention. Instead, the goal was to identify potential emotional and cognitive issues and, if necessary, refer patients to appropriate follow-up care. Several explanations exist for the low referral rates. This may indicate that adequate follow-up care had already been initiated before the intervention, for instance, during hospitalization, leaving no indication for additional care during the intervention session. Alternatively, it could reflect sufficient recovery, negating the need for further care. However, it may also suggest that insufficient follow-up actions were taken, despite potential indications for such actions, which might have resulted in better care [20]. An additional appointment was planned at the stroke-after care outpatient clinic for 26.0% of the patients who participated in this trial, suggesting that specialized nurses preferred follow-up care to referral to other rehabilitation services. Possibly, referral to specialized therapists may have been more suitable. If insufficient follow-up actions were undertaken, it likely decreases the effectiveness of the intervention. Last, the reach of the intervention was excellent: 262 out of 264 patients (99.2%) actually visited the intervention consultation, which might be explained by previous research showing that patients highly prioritize emotional and cognitive problems after stroke [21, 22].

This process evaluation was conducted alongside the ECO-stroke trial. Should the intervention demonstrate clinical and/or cost-effectiveness, this process evaluation corroborates that our strategy for implementation of the intervention in the trial is generally adequate for implementation in daily practice. Moreover, healthcare providers familiar with the intervention do not anticipate significant barriers to the execution of the intervention. If the intervention demonstrates to be effective, we advocate to implement it as delineated in the current process evaluation. If the intervention demonstrates to be ineffective, it is unlikely attributable to poor reach or the overall fidelity to the trial protocol. Rather, a potential lack of effectiveness might be explained by the fact that specialized nurses perceived considerable overlap between the intervention and usual care, potentially diminishing the contrast within the trial. In addition, low referral rates after the intervention contribute to a potential lack of effectiveness. Therefore, in future implementation endeavors, either in research or clinical practice, the importance of initiating prompt follow-up actions following positive screening results should probably be emphasized. The results of the clinical effectiveness analyses are currently being analysed and expected to be completed in the near future.

With regard to clinical implications for daily practice, the process evaluation illustrates an efficacious strategy for implementation (as described in Supplementary Appendix 1) of screening for emotional and cognitive problems in the trial that could also be used for implementation of this type of stroke after-care in daily practice. Specialized nurses who offered the intervention determined that conducting a screening at six weeks after stroke is suitable and emphasized the importance of allocating sufficient time (i.e. one hour) to address topics of emotional and cognitive problems after stroke, next to other topics that should be discussed. Moreover, the specialized nurses who participated in this process evaluation strongly recommended to add a (telephonic) consultation shortly after discharge from the hospital, for example at two weeks, as patients often encounter many unanswered questions. In addition, these specialized nurses underlined the importance of creating a safety net by adequate follow-up, for example by providing contact details and reassurance of the availability of long-term follow-up.

Strengths of this process evaluation are the pre-planned, prospective design and its mixed-methods design, which provides in-depth insights in the experiences of the healthcare providers who offered the intervention. The following limitations can be identified. First, no qualitative data from patients were gathered because it would lead to patients being unmasked to their treatment allocation [23]. Focus groups or interviews after the follow-up period were ruled out, as we assumed that the one-year period between the intervention and the interviews would likely compromise recall accuracy. This aspect limits our understanding of the responsiveness of patients to the intervention. Next, information acquired from the specialized nurses might overestimate true fidelity and underestimate barriers as a result of social desirability towards the research team. Lastly, in this process evaluation we observed low referral rates, but the reasons for non-referral were not recorded. Not documenting these reasons is a limitation, as they could have provided valuable insights.

Conclusion

In general, this intervention was well implemented and considered feasible. Should our upcoming primary analyses reveal clinical and/or cost-effectiveness of the intervention, we intend to recommend incorporation of screening and subsequent care for emotional and cognitive problems after stroke in daily clinical practice. Implementation should adhere to the implementation strategy evaluated in this process evaluation, with specific emphasis on the importance of follow-up actions in the event of a positive screening result.