Checklists are a low-cost safety practice with well-documented benefits in both low- and middle-income countries (LMIC) and high-income countries (HIC) [1, 2]. They are now fully integrated into the routines of most hospitals in HIC and the workflows of surgical teams, aiming to improve the quality of care. On the other hand, limited resources are seen as a key barrier to completing points of the checklist in an LMIC [3]. These concerns are multiplied in areas of armed conflicts, where the entire workflow is disrupted.
The Russian invasion of Ukraine has become the biggest war in Europe since World War Two and has led to hundreds of thousands being killed and injured [4]. This has caused huge damage to all medical infrastructure, and increased the number of patients with trauma and burns by hundreds of times. Medical care in these extremely stressful conditions, with continuous risk for both patients and personnel, limited resources, and overload, could significantly impair patient safety [5]. Therefore, use of the WHO Surgical Safety Checklist and the Anesthesia Equipment Checklist could improve patient outcomes during wartime.
We expected the lack of personnel, time, and other resources to negatively influence the checklist implementation process, as found previously by other researchers [6–8]. On the other hand, we had reasons to expect a net positive impact: the WHO Surgical Safety Checklist and checklists in general are designed to prevent simple mistakes, which are often brought on by a lack of vigilance caused by overexertion, staff shortage and disruptions in the usual workflow – all invariably present in a war-affected hospital [9].
The aim of our study was to assess the impact of the WHO Surgical Safety Checklist and Anesthesia Equipment Checklist implementation on patient outcomes and adherence to safety standards in low-resource settings, affected by an ongoing war.
METHODS
A prospective multicenter study was conducted in 6 large Ukrainian hospitals: Kyiv City Clinical Hospitals #1, University Clinic of the Bogomolets National Medical University, Kyiv City Maternity Hospital #5, National Cancer Institute, Shalimov National Institute of Surgery and Transplantology, Saint Martin Hospital in Mukachevo. Institutions were selected among those that aimed to improve their adherence to the Safe Surgery Saves Lives campaign recommendations but had not yet introduced the WHO Surgical Safety Checklist. On each study site, 1 or 2 operating rooms were designated for data collection. All surgical procedures observed by the data collectors in those operating rooms were included. The study was conducted in two phases: a control period, lasting from March 1, 2022, until August July 31, 2022, followed by a week-long educational period during which the WHO Surgical Safety Checklist and the Anesthesia Equipment Checklist (available in English as supplementary materials) were used, but no data were collected. The intervention period began on August 8, 2022, and lasted until December 31, 2022. Hospital administration and members of all surgical teams were informed about the nature of the study and the proper checklist use procedure. The WHO Surgical Safety Checklist and the Anesthesia Equipment Checklist were provided in the designated operating rooms. Both checklists were translated into Ukrainian. The ethical committee of Bogomolets National Medical University waived the requirement for written informed consent from the patient. The study design was approved by the ethics committee of the Bogomolets National Medical University (protocol #148, 07.09.2021) and retrospectively registered at clinicaltrials.gov on 22/03/2023 (NCT05798000).
Data collection
The anesthesiologists recorded data into a standardized data sheet after every included surgical procedure. The patients’ contact information was recorded for a 30-day follow-up. Collected data included: the date of surgery, patients’ name, age, gender, diagnosis, type of intervention, safety measures performed (airway check, sufficient intravenous access, timely antibiotic prophylaxis, identity confirmation, tool accounting, postoperative mana-gement discussion), hospital, postoperative adverse events, 30-day outcome, and patients’ contact data. Postoperative complications included: massive bleeding (over 1000 ml), acute kidney injury, deep vein thrombosis, myocardial infarction, coma, cerebral stroke, pulmonary artery thromboembolism, surgical infection, pneumonia, sepsis, reoperation, and “other major” (defined as those that qualified as such based on the Clavien Score). Obtained data were then entered into an Excel spreadsheet and transferred to the primary investigators for further monthly analysis.
Outcomes
The primary outcomes were any major complications, including death, during 30 days after surgery.
The secondary outcomes were adherence to six safety measures: airway evaluation and documentation before anesthesia, presence of two peripheral intravenous catheters or a central venous line before the incision if the expected blood loss was 500 mL or higher, timely antibiotic prophylaxis (at least 30 minutes before the incision), identity and surgical site oral confirmation before the incision, sponge, and instrument number confirmation, postoperative management discussion.
Data analysis
Statistical analysis was performed using online calculators [www.calculatorsoup.com]. Frequencies of safety measure performance and postoperative complications were calculated for each site to minimize the effect of differences in the numbers of patients at each site. After that logistic-regression analysis was used to estimate the intervention effect in each group. To evaluate the significance level Pearson’s χ2 test was performed, with a P-value < 0.05 being considered statistically significant.
RESULTS
Overall, 2237 surgical procedures were recorded – 1178 in the control group and 1059 in the intervention (checklist) group. The mean patients’ age was 51.4 and 49.2 years in the intervention and control groups respectively (Table 1). The overall male to female ratio among the patients was 1/0.98, with two sites presenting a male (site 4 – 90.4% and 87.3% in the intervention and the control group respectively) or female (site 3 – 98.1% and 99.3% in the intervention and the control group respectively) predominance. The groups showed no significant differences in demographic characteristics.
TABLE 1
Characteristics of the included patient population. Intervention (After) and control (Before) groups are shown, split based on the study site. The odds ratio, 95% confidence intervals, and P-values obtained via Pearson’s χ2 test for the control and intervention groups are presented
In the control group, 16 (1.3%) patients died within 30 days after surgery, which was significantly higher than in the intervention group, where 5 (0.3%) patients died (OR = 0.35 [0.13–0.95], P = 0.03). Major complications occurred more frequently in the control group as well (OR = 0.32 [0.19–0.52], P < 0.001). Reoperation and surgical infection incidence decreased significantly after checklist implementation (OR = 0.3 [0.1–0.8], P = 0.009, and OR = 0.3 [0.1–0.8], P = 0.01, respectively). However, the incidence of blood loss exceeding 1000 ml and other complications remained unchanged. Theresults are presented in Table 2.
TABLE 2
Local incidence of major postoperative complications. Intervention (After) and control (Before) groups are shown, split based on study site. Odds ratio, 95% confidence intervals and P-values obtained via Pearson’s χ2 test for the entire control and intervention groups are presented
Checklist implementation also was associated with a higher incidence of performed safety measures (Table 3). The investigators reported that preoperative airway assessment and documentation had risen from 74.7% to 95.4% (OR = 4.5 [3.3–6.3], P < 0.0001), antibiotic infusion at least 30 minutes before the incision was performed more often (OR = 23.5 [16.2–34], P < 0.0001), patient identification and surgical site confirmation frequency increased from 82.7% to 98.1% (OR = 10.7 [6.7–17], P < 0.0001). Instrument and sponge count were completed more often (OR = 1.1 [1.0–1.2], P = 0.02), and postoperative management was properly discussed more regularly (OR = 6.7 [4.9–9.3], P = 0.0005). We analyzed the adequacy of the venous access during surgical procedures where blood loss exceeding 500 mL was expected. Adequate venous access was placed in 52.1% (n = 61) of patients in the control group and 90.4% (n = 95) in the intervention group (OR=8.72 [4.1–18.4], P < 0.0001). Detailed data are presented in Table 4.
TABLE 3
Number of included patients and local incidence of successfully performed safety measures in intervention (After) and control (Before) groups, split based on study site. Odds ratio, 95% confidence interval and P-value obtained via Pearson’s χ2 test for the entire control and intervention groups are presented
TABLE 4
Number of patients and local incidence of adequate venous access (2 peripheral or 1 central lines) during procedures, where over 500 mL of blood loss are expected. Intervention (After) and control (Before) groups are shown, split based on study site. Odds ratio, 95% confidence interval and P-value obtained via Pearson’s χ2 test for the entire control and intervention groups are presented
DISCUSSION
Our findings confirm the previous data on the effect of routine Surgical Safety Checklist and Anesthesia Equipment Checklist use on major postoperative complications [9, 10]. However, the results of our study emphasize the importance of introduction of both checklists especially during an ongoing war and in limited resources areas. A significant improvement in the performance rate of periope-rative safety actions provides numerous potential explanations for the effect of checklists on patient outcomes. In this study, adequate intravenous access rate increase was associated with a decrease in 30-day mortality. In contrast, massive bleeding rates have remained unchanged, suggesting a possible contribution of this practice to patient survival. While the effect of the discussed minor safety measures may appear insignificant concerning other modern practices, one may examine their unique place as a fundamental activity involving the entire surgical team, necessitating comprehensive communication even during the simplest procedures [11, 12]. Being a method consistently proven to have a positive impact in almost any surgical setting, the Surgical Safety Checklist is a reliable way to introduce modern safety standards. Of particular interest to us was the evaluation of checklist reception in settings traditionally more opposed to the concept, such as military surgery. Despite reasonable concerns regarding checklist use in combat conditions, the practice is also gradually gaining a foothold in that area [13]. In our research, Site 4 has reported a marked improvement in adherence to standards of antibiotic prophylaxis.
As the WHO Surgical Safety Checklist is a brief reflection of the Safe Surgery Saves Lives recommendations, its implementation has been described as one of the tools for the promotion of the campaign’s messages in LMIC [14]. In low-resource settings, surgical safety checklists may be met with skepticism due to a marked initial difference in expected and existing routine approaches, further amplified by the fact that elimination of such obstacles would require cooperation between the administration of multiple hospital departments. While being a great challenge, this peculiarity may also indirectly explain the positive impact of the practice on problems related to the surgical safety checklists. Such standardized rearrangements often highlight concerns unique to the setting, helping document and rectify the otherwise ignored routine issues [15]. Despite the presence of more significant barriers to proper implementation, it is in the low-resource settings that use of the Surgical Safety Checklist results in the most impressive improvements [16].
It should also be noted that even though for the medical personnel the local development of safety culture may be a protracted uphill battle with seemingly minuscule or non-existent daily benefits, in the eyes of a surgical patient, measures such as checklist reading are perceived as a more professional approach and may have a reassuring effect before a major intervention [17]. This side effect is also a step towards a culture of open communication between the healthcare providers and the patient, contributing to the development of greater trust and an overall more humane approach to the hospitalization process.
It is currently known that the impact of the WHO Surgical Safety Checklist is greater in LMIC [18]. These data encouraged us to take a closer look into its effects in such settings during wartime, since the amount of information available on the matter is limited, notably so when compared to the volume of existing evidence supporting general Surgical Safety Checklist use. Our results indicate that surgical safety checklists still positively influence the quality of care during an armed conflict, suggesting that further research into this topic is ethically justified. Other authors, who described their experience in Syria, also highlight the importance of minimal safety standards while maintaining improvised surgical theaters in the warzone [19].
Study limitations
There was no possibility to perform an interrupted time series analysis [20]. The chosen time frame coincided with other safety policy introductions at some sites, which, combined with consecutive group examinations, likely resulted in some confounding. The entire period of the study also coincided with a Russian invasion, which had a considerable effect on patient profile, availability of medication, personnel, and other prerequisites of adequate medical care. Proper documentation of every significant restriction that influenced the quality of treatment (such as temporary lack of water or electricity) has proved impossible for the allocated staff. Criteria for safety check fulfillment could differ for specific sites and the study protocol, which may have led to underreporting of safety checks performed according to institutional regulations.
CONCLUSIONS
Routine use of the WHO Surgical Safety Checklist and the Anesthesia Equipment Checklist significantly improves the performance rate of basic safety actions and decreases the incidence of major postoperative adverse events. Implementation of checklists during wartime still results in an improvement in patient outcomes.
