Implementation status, drivers and barriers to the sick children quality of care improving interventions in the Oromia region, Ethiopia: case study design

Abstract

Background

Improving quality of sick child care is the primary aim of Integrated Community Case Management services and implementation fidelity is critical to translating an evidence-based intervention to practice. A community-based complex intervention was implemented at the health posts of four agrarian regions of Ethiopia from 2017 to 2018 to improve the coverage and utilization of quality child health services. This study aimed to examine the implementation status of child health care quality improvement interventions in program areas of Oromia region, Ethiopia.

Methods

A case study design using quantitative and qualitative research methods was conducted from September to October 2018. Implementation data were collected using observations, document reviews, and program staff interviews. Program staffs including health extension workers and their supervisors were included in surveys, and purposefully selected key informants from health posts to zonal health office level were included in the qualitative component of the research. The analysis framework was focused on the fidelity of the interventions’ content, frequency, duration, and coverage, as well as the potential moderating factors of implementation using the model proposed by Carroll et al. conceptual framework for implementation fidelity.

Results

Performance Review Clinical Mentoring was implemented according to the plan (every 6 months) in all of the districts and around nine in ten (88%) of the core contents were implemented. Though mentoring was provided by trained mentors, the duration of mentoring was less than the plan to fully implement all of the core activities. Overall, the mentoring program has reached 88% of health extension workers. Slightly greater than three-fourths of health extension workers have been supervised (76%) according to the plan and 80% of health posts were supplied with required iCCM medicines regularly. Staff turnover, topographical challenges, lack of transportation, competing priorities, weak support and feedback from the District health office, and security problems were frequently mentioned challenges to implementation. Whereas, the existence of continual partner support, the presence of integration and coordination of activities, and changes observed were the facilitators of implementation.

Conclusions

The implementation status of the Performance Review Clinical Mentoring Meeting was sufficient, while moderate adherence was observed in supportive supervision and supply of medicines. All of the providers were reached with sick children management training. Therefore, the implementation of community-based interventions should be aware of operational challenges in order to improve and sustain the program’s performance.

Introduction

In the last decades, notable progress has been made in reducing under-five morbidity and mortality worldwide. The under-five mortality rate declined by 53% in the period between 1990 and 2018 [1]. The rates of reduction, however, were not even or uniform across different parts of the world. The sub-Saharan Africa region disproportionately had high a burden of under-five mortality [2, 3]. In the past two decades under-five mortality was reduced by 67% in Ethiopia [4].

The integrated community case management (iCCM) was recommended by the World Health Organization and United Nations Children’s Fund in the 2000s as one of the main intervention for addressing child mortality in resource-limited settings. iCCM is a strategy to train, support, and supply community health workers (CHWs) to provide diagnostics and treatment for multiple illnesses-specifically pneumonia, diarrhea, and malaria-for sick children of families with difficult access to case management at health facilities. It is an extension of the integrated management of childhood illnesses (IMCI) [5, 6]. This intervention is provided by community health workers and aimed to link community health services to facility-based services. The major causes of child mortality in resource-constrained settings can be addressed at the community level largely by engaging communities and supporting community-level workers. A review of randomized control trials showed both statistically significant and operationally important effects in this regard [7].

Previous evaluations done on the national scale community health worker program in sub-Saharan Africa showed positive results in under-five mortality reductions, though only one study showed a statistically significant acceleration [8]. The lack of statistically significant effect may be because such a program doesn’t work, is not implemented properly (type III error) or the community health worker program design was not optimized to achieve the maximum effect (e.g., via the best or most efficient combination and dose of intervention components) [9]. Several other qualitative assessments also suggest poor outcomes are related to suboptimal program design and implementation [10].

In the early implementation period of iCCM in Ethiopia, a study found relatively high level of quality of care and implementation strength [11]. However, household survey showed that care seeking for childhood illness was low and similar in both study arms at baseline and end line, and has only increased marginally [12]. Following the scale-up of the program, service utilization remained low and lagged. The main reasons for low health service utilization were the demand side and supply side barriers [13]. The community’s unfavorable attitude towards curative care and irregular supply or unavailability of medicines at health posts caused low service utilization [14]. Sub-optimal quality of child health care was also reported in the recent studies done in the same setting as this article as part of a larger baseline survey. These studies were done after iCCM fully scale-up nationally, and integrated to the routine system [15,16,17].

The government of Ethiopia in collaboration with partners developed and introduced a community-based complex intervention to optimize the health extension program in 2017. The community-based complex intervention is aimed to improve the coverage and utilization of child health services in Ethiopia through three main strategies: demand creation, capacity building, and ownership and accountability [18, 19]. Capacity-building interventions are intended to improve the quality of care and health services at health posts. It includes training, regular clinical mentorships and performance reviews, supervision, and supply of iCCM medicines and commodities [19, 20].

The performance review clinical mentoring meetings (PRCMM) is a strategy used to improve the clinical management skills and motivations of HEWs. The changes in the capacity of health workers should contribute to improved quality of sick child care [21]. PRCMM is conducted biannually in the central District town and engages all HEWs and their supervisors. It involves a review of the health post iCCM activity performance, followed by clinical mentoring to HEWs. The program is used as a post-training strategy in public health interventions. PRCMM creates an opportunity to provide clinical knowledge and skills transfer among HEWs and builds a supportive atmosphere [22, 23]. When implemented effectively, mentoring resulted in improvements of clinical management skills of health workers [24,25,26] and the quality of sick child care [27, 28]. Similarly, studies found that high-quality supervision with components of supportive approaches, problem-solving and feedbacks are effective to improve health workers skills [29,30,31,32,33].

In Ethiopia, fidelity hasn’t measured since early in the scale-up of iCCM [34, 35]. There was scanty evidence on how and in what context these interventions can lead a positive outcomes (improved competency of community health workers and improved quality of care). Therefore, this study aimed to explore whether the packages of quality improvement (QI) strategies were implemented as intended, in what context and to describe the facilitators and barriers of the implementation. The evidence generated will be used to improve quality of child health services in Ethiopia.

Methods

Study setting

The study was conducted in districts targeted for the implementation of the Optimizing Health Extension Program in the Oromia region. The Optimizing Health Extension Program was implemented in four districts of the West Hararghe Zone and seven districts of the Guji Zone in the Oromia regional state. Overall, there were 1,151,145 people living in these intervention districts, with 522,272 people in Guji Zone and 628,873 in West Hararghe Zone. The total number of children under-five years old was 188,788. The districts were home to 567 HEWs (379 in Guji Zone and 188 in West Hararghe Zone) across 257 health posts and 50 health centers (26 in Guji and 24 in West Hararghe Zones). The map of the study area is presented in Fig. 1.

Fig. 1

Map of the study area

Description of the intervention

Figure 2 presents the Theory of Change (TOC) for the child healthcare quality improvement interventions within the community-based complex intervention. The interventions included iCCM training, PRCMM, supportive supervision, and the supply of iCCM medicines and commodities.

Fig. 2

Theory of change of quality improving interventions

iCCM trainings

iCCM trainings were conducted by skilled trainers with Training of Trainers (TOT) certification for six days. Theoretical training was provided during classroom sessions, followed by practical training at the health center level.

PRCMMs

The PRCMMs are conducted bi-annually in a central town in the district for two days. This session comprises of two main activities: performance review and clinical mentoring. Participants include HEWs, supervisors of HEWs, IMNCI focal persons at health centers, district health office family health experts, and heads of Primary Health Care Unit (PHCU) and DHO. In addition, family health experts from ZHDs and OHEP focal points also participate. The PRCMM sessions are led and facilitated by iCCM-trained and experienced mentors who are knowledgeable about the local context, including the local language. Each mentor offers guidance and technical support to 4–6 mentees (HEWs).

Supportive supervisions

Supportive supervisions with a component of iCCM are conducted every three months using standard supervision checklists. These supervisions are provided by experts from the DHO, HEW supervisors from health centers, and OHEP program coordinators. The supervisors assess and document the care process, as well as the availability of materials for service provision at the health posts. Availability of iCCM medicines, medical equipment, and supplies are assessed during supervisory visits. In addition, supervisors review iCCM registers and observe consultations to ensure that HEWs are properly providing services to sick children. Finally, supervisors provide feedback.

iCCM medicine and commodities supply

Medicines are supplied to health posts monthly based on their needs. Every month, the HEWs complete the iCCM medicines Request and Resupply Form (RRF) and send it to PHCU. The PHCU then check the completed form and resupplies medicines from the district store. Supplies, such as iCCM register, chart booklets, and field books, are provided by the implementing partner supporting the program in the districts.

According to the TOC of the program, when interventions are implemented as planned, different levels of results (short, intermediate, and long-term) will be observed among the target groups. The intermediate effects (outcomes) of the interventions include improved clinical knowledge and skills, increased motivation of HEWs, improved coverage and uptake of quality sick child care, and increased satisfaction of caretakers at the health post level. The long-term effects (impacts) of the intervention are reductions in under-five child morbidity and mortality attributed to common childhood illnesses such as pneumonia, diarrhea, malaria, and malnutrition.

Study design and participants’ selection

A case study design utilizing both quantitative and qualitative research methods was conducted from October to November 2018. This design allows for a comprehensive examination of the program’s operation and the surrounding context, providing valuable insight. By analyzing deviations from the program model or plan across various settings, it offers evidence on the effectiveness of the program and helps understand the reasons behind its success or failure. Additionally, it enables the testing of theories regarding the impact of different program components on outcomes [36].

The study participants included HEWs; supervisors of HEWs; health center heads; iCCM/IMNCI focal points at the health centers, DHOs and ZHDs; and program focal points from partner organizations in all implementing districts across the two zones. All health centers, DHOs, and both ZHDs were included in the study.

Ideally, each PHCUs has up to five health posts. We randomly sampled two health posts from each PHCU and included them in the study (approximately 40% of health posts in the study districts). We believe that this sample size was sufficient to assess the implementation fidelity of QOC improving interventions. From the selected health posts, one HEW was randomly selected and included. Family health experts or iCCM focal points from the DHOs and ZHDs were interviewed. Additionally, staff from the implementing partner located at the ZHD and DHO levels were included in the study.

A survey was conducted with program staff to assess the implementation status of QI strategies, specifically iCCM training and PRCMM. Health Extension Workers were interviewed to evaluate their participation in clinical trainings and their perceptions of the quality of PRCMM sessions held in the 12 months prior to the survey period. HEW mentors and supervisors were also asked about the HEWs’ responsiveness to PRCMM sessions. Additionally, four PRCMM sessions were planned to be observed in order to assess the fidelity of the programs implementation (two observations in each zone).

Program record reviews were conducted to assess the implementation status of supportive supervision, and facility audits were conducted to examine the supply of iCCM medicines and commodities to health posts. Program records, such as reports and supportive supervision checklists, were reviewed, and this was supplemented with interview results. Key informant interviews were conducted with purposefully selected HEWs, HEW supervisors, iCCM focal points at DHO and ZHDs, and individuals from NGOs to understand the implementation processes, as well as barriers and enablers to the implementation of QI strategies.

Measurement

The implementation fidelity framework by Carroll et al. (2015) was utilized to evaluate the implementation fidelity of iCCM quality improvement interventions [37]. Implementation fidelity was assessed using the dimension of adherence, which refers to the degree to which program components (QI strategies) are carried out as outlined in the program plan. This includes aspects such as program content, frequency, duration, and coverage. Additionally, we analyzed the quality of program delivery and the responsiveness of program participants. Program delivery quality pertains to how effectively the QI strategies are delivered to and received by participants, while participant responsiveness refers to the level of engagement and reaction from participants during program activities (Fig. 3).

Fig. 3

Conceptual framework for evaluating the implementation fidelity of quality improvement interventions.

The core intervention components, fidelity dimensions, and indicators used in measuring fidelity are presented in Table 1.

Table 1 Quality improvement interventions, fidelity dimensions and measurement indicators with data sources

Data collection

The study utilized techniques such as observations, record reviews, and interviews to collect data. Both quantitative and qualitative interview techniques were employed. Observations were used to assess the implementation fidelity of PRCMMs content adherence. The entire process of PRCMM sessions was observed, and the findings were documented. Trained data collectors silently observed activities during PRCMM sessions and documented the overall process without interfering with the implementers activities. The coverage and frequency of PRCMM sessions held in the past 12 months were determined through interviews with program staff and a review of program records, such as two rounds of PRCMM attendance registration forms and reports. Additionally, program staff were asked to evaluate the delivery quality of PRCMM sessions and the responsiveness of recipients. Delivery quality was examined through interviews with HEWs, and participants responsiveness was assessed by asking HEW supervisors.

A review of program records was conducted to assess the implementation status of QI strategies, including supportive supervision and the supply of iCCM medicines. Supportive supervision checklists and reports were analyzed to determine the content, frequency, and coverage of supervisions conducted in the past 12 months. This information was further supported by interviews with HEWs. Drug ledgers were also examined to evaluate the supply of iCCM medicines to health posts, with additional insights gained from interviews with HEWs. Key informant interviews were conducted with purposefully selected program staff to identify barriers and enablers to the implementation of QI strategies.

The questionnaires were designed based on the study’s purpose after reviewing program records. In line with the data collection techniques applied, document review templates, observation checklists, a quantitative interview tool, and a key informant interview guide were developed and used (see Supplementary file). The interview tools were translated into local language (Afan Oromo) and then back translated into English. A pilot test was conducted in a district in Oromia region that was not selected for the study, and the questionnaire contents were updated accordingly.

Two data collection teams participated in the study. Each team consisted of two data collectors, a supervisor, and a driver. The data collectors were health professionals with a bachelor’s degree or higher, recruited from outside the study zones. Both data collectors and supervisors underwent a 3-day training on the study’s purpose, questionnaires, data collection processes, and research ethics. They were also oriented on the appropriate use of audio tape recorders and note-taking during interviews.

Data quality control

Experienced data collectors and supervisors who have also received training on the field-level data collection techniques and the questionnaire participated in the study. A standard operating procedure was developed and provided to data collectors and supervisors to standardize the data collection process. Field data collection was closely monitored by supervisors, and daily feedback was given. The scientific rigor and trustworthiness of qualitative data were ensured using credibility, dependability, confirmability, and transferability criteria [38].

Credibility

We used multiple data sources to cross-verify the findings. Additionally, we undertook member checks to ensure the consistency and completeness of respondent ideas at the end of each interview. Parts of the audio recorded data were heard by the study participants to confirm whether it represented their ideas and views.

Transferability

We provided detailed contextual information about the study. The research context, participants, and methods were thoroughly described to allow readers to evaluate the similarities or differences between their context and the study.

Dependability

A pilot-tested questionnaire was used, and we followed rigorous procedures throughout the research process, including participant selection, interviews and analysis.

Confirmability

The investigators considered the impacts of their preconceptions about the subject area during data analysis and interpretation of findings.

Data processing and analysis

Quantitative data were checked for consistency and completeness and entered into SPSS version 21 for analysis. The data were analyzed descriptively and presented using frequency and proportion. Implementation fidelity data were analyzed and interpreted using the sub-dimensions and indicators of adherence presented in Table 1.

The quality of the QI strategy (PRCMM) delivery was measured with Likert scale questionnaires through interviews with HEWs, and participants responsiveness was assessed by interviewing HEW supervisors. HEWs were asked to evaluate the delivery quality of PRCMM, while their supervisors were asked to evaluate the responsiveness of health extension workers towards PRCMM. The delivery quality questions asked about the satisfaction with the mentor’s preparedness, enthusiasm, respectfulness, and competency during PRCMM sessions. Participant responsiveness was assessed through one question regarding satisfaction with the interaction and attentiveness of mentees during PRCMM sessions. The moderators of implementation fidelity (delivery quality and participants responsiveness) were analyzed descriptively.

Qualitative data were transcribed into the original language (Afan Oromo) and then back translated into English. Audiotape records were compared with notes to ensure consistency and completeness. Content and thematic analysis techniques were used to analyze the qualitative data. The themes of analysis results included enablers and barriers to program implementation, as well as fidelity sub-dimensions (content, frequency, and coverage).

Results

A total of 178 program staff members were interviewed, with 88 from West Hararghe Zone and 90 from Guji Zone. Thirteen participants were district health office experts, 49 were iCCM/IMNCI focal points at health centers, 106 were HEWs, 2 were experts from the zonal health office and 8 were NGO at the district and zonal levels (Table 2). All HEWs (n = 106) were trained in iCCM, with 87% trained more than 12 months before the survey and the remaining 13% trained within the 12 months prior to the survey period.

Table 2 Distribution of respondents by district and health institution type

Three PRCMMs were observed, out of the planned four. Additionally, 71 key informant interviews were conducted with program staff, and a review of program records was undertaken.

Implementation fidelity: content adherence

PRCMM sessions were organized in such a way that HPs were grouped according to their catchment health centers. When the number of health posts or health extension workers per catchment health center was low, up to 2 health centers were assigned to one mentoring group. During each PRCMM session, HEWs sat in a circle, facing each other. On average, the ratio of mentors to HEWs was 1:7, with a range of 1:4 to 1:10. This did not meet the standard of one mentor for 4–6 HEWs. All mentors were trained in iCCM and mentoring techniques.

Observation data indicated that most of the expected components of the PRCMM were conducted in both zones. These activities included presenting past iCCM performance of HEWs; reviewing iCCM registers; completing quality grid sheets (checking consistency and completeness of care provided by HEWs); discussing key iCCM performance, strengths and weaknesses; conducting problem root cause analysis; providing clinical mentoring to HEWs, and action planning. However, mentoring was only provided using case scenarios. In none of the PRCMM sessions were HEWs observed and mentored while managing real patients (sick children) in the health center or hospital. The mentors reviewed every HP iCCM registration book. The review process involved collecting and recording data on under-five cases managed at health posts by HEWs on data compilation sheets, and then comparing them with the plan. A sample of two cases/illnesses managed by HEWs in the last six months was reviewed for each category of child illness following the clinical care pathway: assessment, classification, treatments, referral, and counseling. The strengths and weaknesses of assessment and management of cases by HEWs were identified by HEWs themselves and their supervisors, with coaching and support from assigned mentors. Nearly nine out of ten (88%) of the core activities of PRCMM were implemented in both zones.

Quality of delivery and participants responsiveness

The HEWs were asked to rate the preparedness of mentors including the delivery quality of PRCMM. Almost all of the HEWs were satisfied with the delivery quality of PRCMM sessions, as measured in terms of mentors preparedness, enthusiasm, respectfulness, and competency during the PRCMM sessions (Fig. 4).

Fig. 4

Health extension workers’ satisfaction on PRCMM quality of delivery

Mentors and supervisors of HEWs were interviewed to assess the responsiveness (active interaction and attentiveness) of HEWs during PRCMM sessions. The majority (45 out of 72 or 63%) of mentors were satisfied, 18 (25%) very satisfied and 9 (12%) dissatisfied with the responsiveness of HEWs. The main reasons mentioned for dissatisfaction were lack of punctuality, limited interactions during discussions, and misbehavior.

Implementation fidelity: frequency and duration

PRCMMs

PRCMMs were conducted biannually at the district level in both zones, in accordance with the standard. However, most participants felt that the meetings were too infrequent to effectively improve their performance. They suggested that review meetings and mentoring sessions be held at least once every three months.

‘Whenever the frequency of meetings and mentoring sessions was shorter than it currently is, we would have the opportunity to detect performance gaps in a timely manner and take prompt corrective measures’. Head of Health Center, Male, 26 years old

Regular cluster-level review meetings and mentoring were not conducted in the majority of health centers in the study area. Only a few health centers and districts have been conducting monthly and quarterly performance reviews by involving HEWs in their catchment areas.

The PRCMM sessions were completed in an average of one and a half days, which was not consistent with the standard duration. The first day of PRCMM activities often started late due to participants arriving delayed.

This finding was further supported by qualitative data, as key informants noted that delayed attendance was a common problem among participants in previous PRCMM sessions.

‘Often we launch review meetings lately as participants don’t arrive on time. Despite prior communication, participants consistently show up late. This has resulted in lack of sufficient mentoring and review meeting hours’. ICCM/IMNCI focal of health center, a 28-year-old male

Supportive supervision

On average, health posts in the study area received three supervisory visits in the last 12 months, falling short of the recommended minimum of four visits. The supportive supervisory visits held at health posts ranged from 2 to 4 in the last 12 months. During these visits, HEWs were supervised on iCCM components, including checking the availability of supplies and medicines, as well as assessing clinical case management performance. Three-quarters of health extension workers (76%) had received these supportive supervisions in the 12 months prior to the assessment period. The supervisions were conducted by staff from NGOs, DHOs, and/or PHCUs.

The qualitative research participants responded that the supportive supervision provided by health centers and DHOs was inconsistent, untimely and inadequate. Often, supportive supervision was carried out solely by NGOs or in partnership with government bodies. Only a small number of instances involved health centers and/or DHOs in the process. The primary factors contributing to the irregularity of supportive supervision were security issues, transportation shortages, and a lack of incentives for supervisors.

‘In our catchment health center, each health post has assigned focal persons who provide supportive supervision on a weekly basis or at least once per month. However, due to a shortage of human resources in our health center, we are unable to cover all health posts in the specified schedule. Additionally, we have transportation challenges. There is a health post in our catchment that is 30 km away from the health center. How can a supervisor reach this health post without transportation support?’. Head of the health center, Male, 30 years old

The supportive supervision provided to HEWs has mainly focused on collecting iCCM performance data. Only a few have discussed major problems related to clinical service provisions and included coaching support in areas that need improvement. The vast majority of interviewed HEWs were not satisfied with the supportive supervision provided to them.

‘Our supervisors visit us, but they only stay at the health post for a limited time to discuss the major problems related to the services we provide. They [supervisors] mainly focus on collecting data about the services provided using a checklist’. Head HEW, 28 years old

iCCM medicines and commodities supply

The average monthly frequency of iCCM medication deliveries to health posts was nine months during the past 12 months of the survey period. There was variation in the supply of iCCM medicines and commodities among the HPs in the study area. This variation was related to gaps in timely completing and submitting medicine request and resupply forms (RRFs) by HEWs.

‘Since the intervention [OHEP] began in our catchment area, the problem of iCCM medicines has been greatly resolved. However, there are gaps in some of the health posts when it comes to in timely completion and submission of drug request and resupply forms (RRFs). This has hindered the on-time distribution of medicines to health posts’. Head of Health Center, Male, 36 Years old

The HEWs have also noted the improved availability of medicine since the introduction of the HEP optimization program.

‘Since the optimization program started at our health post, we have not faced any challenges related to medicines. iCCM medicines are supplied by health centers and sometimes by district health offices. Currently we have iCCM medicines. We do also have CBNC medicines for newborns zero to two months’. Head of HP, 28 years old

Implementation fidelity: coverage

Out of the 106 HEWs interviewed, 93 (88%) participated in both rounds of PRCMMs. The most commonly stated reasons for missing review meetings by HEWs included being on leave (study leave, maternity leave, and sick leave).

Three quarters (76%) of HEWs had received supportive supervisory visits focused on iCCM in the last 12 months prior to the survey from any of the supervisory bodies (NGOs, PHCUs, DHOs). Moreover, eight out ten (80%) of HPs were monthly supplied with iCCM medicines in the last 12 months prior to the survey (Fig. 5). Additionally, almost all (98.5%) of the HPs were supplied with iCCM registers, field books, and chart booklets.

Fig. 5

The coverage of quality improving interventions

Moderating factors

Intervention complexity

The key informants (program staff) stated that the complex intervention is understandable and can be easily applied in their contexts. One of the major factors that makes it easier to implement is that the interventions are implemented within the existing system, targeting critical problems that require partners support. The research participants (program staff) have also elucidated that the timing of program initiation was convenient for their organization and staff, primarily because the interventions aim to bring substantial improvements to child health services utilization, which has been reported as very low in the study area. Orientations (short briefs) about the program activities were provided during the program’s inception period, and formal trainings were given before and during program implementation. Additionally, manuals or guides to support program implementation were distributed to the program staff.

Context

The study zones share similar contextual factors, including infrastructure, availability of human resources, health system characteristics, and demographic and population characteristics (such as language, religion, and culture). In both zones, challenges related to road infrastructure, electricity, and water supply were identified as impediments to the implementation of health services at health facilities. Shortages of human resources at health centers, along with administrative issues (such as insecurity) in some districts, were also identified as barriers to the implementation of iCCM.

Drivers and barriers to program implementation

The analysis of in-depth interviews with participants revealed two main factors affecting program implementation. These factors included those that facilitated program implementation, as well as those that hindered it. The enablers of program implementation frequently mentioned by the respondents were:

• Existence of strong support from partners or NGOs, both technical and material supports. These supports were critical in enhancing the quality of care provided to sick children.

• There was an increased number of trained staff on iCCM, particularly the HEWs. In the study areas, a large majority of HEWs were trained in iCCM case management.

• Active engagement and participation of local community members in the iCCM program implementation: participants mentioned that community leaders (‘Aba Gada’) and religious leaders were actively involved in mobilizing communities for health post maintenance and renovations to ensure their functionality.

• Active engagement and participation of district-level government sectors and institutions at Kebele level in the iCCM program implementation: the key informants mentioned that district administration, the women and child affairs office, kebele administration, schools, and the agriculture sector supported program implementation in various ways.

• Existence of Women’s Development Army structure at the community level: the women’s development army supported program implementation through identifying service utilization bottlenecks, educating the community about childhood illnesses, and linking clients to health posts.

With regards to barriers to program implementation, the following were frequently mentioned by the study participants:

• Variations in the level of commitment among HEWs: the key informants revealed that there are still some HPs that are closed and not regularly providing services, partly due to inappropriate planning for field visits, trainings, and personal leave.

• Competing priorities (overlapping schedules): participants reported that competing priorities at the district level were major barriers to effectively implementing program activities, such as supportive supervisions, clinical mentorships, and review meetings.

• Weak support given to health posts by the health center and district health offices: the majority of HEWs stated that supportive supervisory visits from health centers and districts were inadequate.

• Administrative problems: Some participants mentioned that security problems in their catchment areas were a major obstacle to implementing program activities, including supportive supervisions, mentorships, and other community-level activities.

• Topographical problems and lack of transportation: the majority of participants stated that the difficult topography of the districts in the zone, coupled with a lack of transportation, has significantly challenged them to sufficiently support program implementation at the health post and community levels.

Discussion

This study has shown that the implementation of Performance Review Clinical Mentoring Meetings (PRCMMs) had good adherence to the protocol. However, there were gaps in the implementation fidelity of supportive supervision and the supply of iCCM medicines to health posts. A previous study on iCCM in Ethiopia found better implementation strength of iCCM in terms of supportive supervision coverage (87%), and comparable results in relation to PRCMM (89%) and iCCM training coverage (98%). However, the availability of essential iCCM commodities was comparatively lower in the prior study (69%) [11].

Most of the core contents of PRCMM were implemented as intended, but mentoring of HEWs was conducted using case scenarios or existing data. Research shows that mentoring done with real patients leads to better knowledge, skills, and confidence for health extension workers [39, 40]. Allowing HEWs to practice on actual patients will provide them with a better opportunity to enhance their skills [39].

The coverage, frequency, and duration of exposure to the intervention contribute to positive outcomes in target groups [41, 42]. Earlier studies conducted on iCCM in Ethiopia also found positive relationships between exposure to the PRCMM intervention and improvements in the quality of sick child care at health posts [25, 43]. PRCMM was implemented biannually according to protocol, and a large majority (88%) of the HEWs attended PRCMM twice per year. However, the duration of exposure was too short to be adequate. On average, PRCMM sessions were conducted for one and a half days, which is lower than the minimum standard. This finding was supported by qualitative data where participants mentioned that the short duration of PRCMM, combined with delays in session starting times, affected the intensity of program implementation.

The approach to mentoring, as well as preparation and adequacy of mentors (the ratio of mentors to mentees), significantly impacts the outcome of mentorship [44]. In our study, HEWs were mentored by trained mentors who were well-versed in the local context, including the local language and culture. However, the ratio of mentors to mentees exceeded the recommended guidelines of the program. As a result of this, many HEWs did not have their registers reviewed or received a quality grid assessment on several occasions. This affects the quality of clinical mentoring provided to HEWs.

Supportive supervision, incorporating elements of coaching, problem identification, and on-site support, improves the skills and motivation of health workers and can lead to positive outcomes [45, 46]. Regular supervision with feedback has a high impact on changing the knowledge and skills of health workers and healthcare managers [47,48,49]. The performance of community health workers is highly correlated with the quality of supervision and their relationships with other healthcare workers [50]. Supervision can bridge the gap between PRCMM sessions. However, in the study area, supportive supervision was provided less frequently than recommended, had moderate coverage, and lacked a coaching component. Qualitative research participants revealed that supervision from health centers and district health offices was irregular and focused more on data collection than problem identification and solving. Reasons for these shortcomings included supervisors lacking expertise and motivation, a shortage of human resources or unavailability of clinical supervisors due to understaffing at the supervising health facility, lack of funds for transport, and absence of incentives. This is consistent with findings in other low-income countries worldwide, where supportive supervision is often low in coverage, irregular, unsupportive, and demotivating [45]. Similarly, a study conducted in Ethiopia found that supervision of health posts, including case management observation, had low coverage [51].

Availability of iCCM medicines and job aids are critical components of quality sick child health care [11, 52]. In the study area, eight out of ten health posts were supplied with essential iCCM medicines on a monthly basis over the 12-month period examined. Nearly all of the health posts had the necessary job aids, such as service registration books, chart booklets, and family health cards.

Appropriately designed, implemented, and governed community health programs can improve access to quality healthcare and reduce mortality [53,54,55,56]. CHWs play a significant role in filling the gaps of health workforce shortages, particularly in settings like SSA [57]. Most SSA countries have policies supporting community health programs and are working to expand coverage of community health services. However, gaps have been observed in translating policy into practice. The main concerns of community health program implementation in such settings include drug supplies, quality of care, and CHWs’ incentives, training, and supervision [58, 59].

The success of community health programs is attributed to the strong integration of interventions into well-designed and adequately funded health systems [60,61,62]. This can be achieved by incorporating community health programs into the national healthcare system, investing in building the skills of CHWs, and supporting them as valued members of the health team [56, 63].

In summary, the study utilized mixed research methods (observation, interviews, and document reviews) to assess the implementation fidelity of quality improvement interventions. The study was limited to the Oromia region and doesn’t addressed the variation in the implementation of the quality improving strategies in other regions of Ethiopia. However, due to the comparable health system context, we believe that implementation fidelity may be comparable in other regions with similar contexts. Certain aspects of fidelity metrics, such as supervision quality and the delivery quality of PRCMM, were examined through interviews with HEWs. Providers may not provide accurate information to avoid criticizing supervisors and mentors. However, document evaluations (supervision checklist reviews) supported the findings of the supervision interviews. Furthermore, there may limitations related to observing PRCMM sessions as mentors and mentees could alter their behavior when being observed.

Conclusions

The current study found that there was moderate adherence to the implementation fidelity of supportive supervision and the delivery of critical iCCM medications, and good adherence to Performance Review Clinical Mentoring Meetings. A significant proportion of HEWs were not supervised, and the same proportion of health posts were irregularly supplied with iCCM medicines. The supportive supervision provided to health posts by the health center was irregular, weak, and lacked a coaching component when conducted. The main implementation barriers in the study area were security problems, lack of incentives and transportation support, gaps in timely submitting drug request and resupply forms, and a shortage of skilled human resources at health centers. The findings imply that efforts should be made to improve the implementation strength of iCCM quality improvement strategies. In addition to coverage, enhancing supervision and mentorship quality is crucial. Avoiding fragmentation in program implementation and optimizing the Health Extension Program is critical for improving child health and well-being.