Comparison of Modified Early Warning Score (MEWS), Simplified Acute Physiology Score II (SAPS II), Sequential Organ Failure Assessment (SOFA), and Acute Physiology and Chronic Health Evaluation II (APACHE II) for early prediction of septic shock in diabetic patients in Emergency Departments

Abstract

Introduction

Sepsis is a severe medical condition that can be life-threatening. If sepsis progresses to septic shock, the mortality rate increases to around 40%, much higher than the 10% mortality observed in sepsis. Diabetes increases infection and sepsis risk, making management complex. Various scores of screening tools, such as Modified Early Warning Score (MEWS), Simplified Acute Physiology Score (SAPS II), Sequential Organ Failure Assessment Score (SOFA), and Acute Physiology and Chronic Health Evaluation (APACHE II), are used to predict the severity or mortality rate of disease. Our study aimed to compare the effectiveness and optimal cutoff points of these scores. We focused on the early prediction of septic shock in patients with diabetes in the Emergency Department (ED).

Methods

We conducted a retrospective cohort study to collect data on patients with diabetes. We collected prediction factors and MEWS, SOFA, SAPS II and APACHE II scores to predict septic shock in these patients. We determined the optimal cutoff points for each score. Subsequently, we compared the identified scores with the gold standard for diagnosing septic shock by applying the Sepsis-3 criteria.

Results

Systolic blood pressure (SBP), peripheral oxygen saturation (SpO2), Glasgow Coma Scale (GCS), pH, and lactate concentrations were significant predictors of septic shock (p < 0.001). The SOFA score performed well in predicting septic shock in patients with diabetes. The area under the receiver operating characteristics (ROC) curve for the SOFA score was 0.866 for detection within 48 h and 0.840 for detection after 2 h of admission to the ED, with the optimal cutoff score of ≥ 6.

Conclusion

SBP, SpO2, GCS, pH, and lactate concentrations are crucial for the early prediction of septic shock in patients with diabetes. The SOFA score is a superior predictor for the onset of septic shock in patients with diabetes compared with MEWS, SAPS II, and APACHE II scores. Specifically, a cutoff of ≥ 6 in the SOFA score demonstrates high accuracy in predicting shock within 48 h post-ED visit and as early as 2 h after ED admission.

Introduction

Sepsis is a severe medical condition characterized by a systemic inflammatory response that frequently impacts various organ systems. [1, 2] In Thailand, the annual incidence of sepsis falls within the range of 75 to 150 cases per 100,000 individuals. Notably, sepsis is a significant contributor to mortality, accounting for 6% of all deaths, with 22% of these fatalities attributed to concurrent sepsis. [3] When sepsis progresses to the more critical state of septic shock, the mortality rate substantially escalates to approximately 40%, a stark contrast to the 10% mortality rate observed in cases of sepsis. [4]

Patients diagnosed with diabetes mellitus (DM) face an elevated susceptibility to infections and sepsis. [4] Furthermore, the management of sepsis in individuals with diabetes presents inherent complexities. [5] This issue is exacerbated by the growing prevalence of diabetes, as evidenced by data from Thailand’s National Health Examination Survey, which has documented a consistent annual rise in the incidence of DM cases. [6]

The Emergency Department (ED) typically serves as the initial point of contact for patients experiencing deteriorating health conditions. Timely recognition and management of sepsis within this critical setting are paramount, as delayed intervention can exacerbate the condition. Therefore, adhering to sepsis guidelines, which emphasize the importance of early fluid resuscitation and the prompt initiation of appropriate antibiotics within the first hour of presentation, is of utmost significance.

Numerous screening tools have been developed to aid in the identification of sepsis, including Systemic Inflammatory Response Syndrome (SIRS), Quick Sequential Organ Failure Assessment Score (qSOFA), Modified Early Warning Score (MEWS) [7], and the Ramathibodi Early Warning Score (REWS). [8] Additionally, scoring systems such as MEWS [9], Simplified Acute Physiology Score (SAPS II) [10], Sequential Organ Failure Assessment Score (SOFA) [2, 11], and Acute Physiology and Chronic Health Evaluation (APACHE II) [12, 13] are employed to predict the severity or mortality risk associated with the disease. Early recognition of sepsis severity prior to the onset of septic shock enables more vigilant monitoring and precise resuscitative efforts.

The principal objective of this investigation was to conduct a comparative assessment of the efficacy of MEWS, SAPS II, SOFA, and APACHE II scoring systems for the early prediction of septic shock in diabetes patients within the ED setting. Additionally, we sought to determine the optimal cutoff values for detecting septic shock. Our secondary aim entailed the analysis of data pertaining to patients presenting with concurrent diagnoses of diabetes mellitus and septic shock at Ramathibodi Hospital’s ED.

Methods

Study design and setting

We conducted a retrospective cohort study focusing on diagnostic accuracy within the ED of Ramathibodi Hospital in Bangkok, Thailand. This study received approval from the Human Research Ethics Committee at the Faculty of Medicine, Ramathibodi Hospital, Mahidol University. Ramathibodi Hospital is a tertiary care medical facility located in Bangkok, and the study was conducted following their ethical guidelines (COA.MURA2021/878).

Study participants

We enrolled patients visiting Ramathibodi Hospital’s ED between 1 January 2017 and 31 December 2021. We included those aged > 18 years with DM. Inclusion criteria mandated a sepsis diagnosis according to the International Classification of Diseases, 10th Revision (ICD-10), within 48 h of their ED admission. The diagnostic criteria for sepsis were aligned with the latest international consensus guidelines, explicitly following the Sepsis-3 criteria.

Predetermined parameters were implemented and required the identification of a confirmed source of infection and the presence of at least one of the following criteria: qSOFA score of ≥ 2, SIRS score of ≥ 2, or REWS of ≥ 2. [8] Furthermore, septic shock was defined as the presence of a confirmed source of infection, accompanied by hypotension necessitating vasopressor use to maintain a Mean Arterial Pressure (MAP) ≥ 65 mmHg and adequate fluid resuscitation. [8] The exclusion criteria were as follows: presentation of sepsis after 48 h of triage in the ED, incomplete laboratory results, non-diabetic mellitus types 1 or 2, palliative care with the refusal of inotropes or an endotracheal tube in the ED, revisit of clinically unimproved cases, and a diagnosis of septic shock within 15 min after the ED visit.

Data gathering

We collected data from 552 of 1,150 patients (66 patients had no data, and 532 had incomplete laboratory values). Data collection comprised a comprehensive set of variables, including patients’ demographic information, age, sex, comorbidities, glycemic control of DM patients categorized by the American Diabetes Association (ADA) guidelines for the year 2022 [14, 15], mode of transportation to the ED, initial vital signs in ED, Emergency Severity Index (ESI) triage, source of infection and initial laboratory findings such as white blood cell count, platelet count, serum creatinine, point-of-care glucose, electrolyte levels, albumin, bilirubin, arterial blood gas parameters, and serum lactate. Moreover, each patient’s SIRS, qSOFA, REWS, MEWS, SOFA, SAPS II, and APACHE II scores were recorded.

Outcome

The primary outcome of this study was the diagnosis of septic shock within a 48-hour timeframe. Secondary outcomes included the duration from triage to the progression of septic shock, admission to the intensive care unit (ICU), discharge from the ED, ED length of stay (LOS), overall hospital length of stay, 28-day mortality, in-hospital mortality, and the use of mechanical ventilation. The protocol is illustrated in Fig. 1.

Fig. 1

Study flow

Sample size and statistical analyses

The sample size of this study was calculated by PM-sampsize statistical software on the basis of our pilot unpublished data. [16] We included 215 patients with diabetes. Using the C-statistic, the area under the curve (AUC) was 0.68 with three parameters (from qSOFA score). We assumed a 0.05 acceptable difference in the apparent and adjusted R-squared and a 0.05 margin of error in the estimation of intercept. When using the Events per Predictor Parameter (EPP), we assumed a prevalence of progression to septic shock of 0.10 by the STATA statistical program. The minimum sample size required for new model development was 693 with 70 events. Descriptive data are presented as the means with standard deviations (SD) or medians (interquartile ranges) for continuous variables and as the percentages for categorical variables. Chi-square and Wilcoxon’s rank-sum tests were used to compare categorical variables and continuous variables, respectively. We used the Youden index to determine a cutoff value for predicting progression to septic shock within 48 h for each score. Sensitivity, specificity, and positive likelihood ratio (LR+) were calculated to present the predictive performance separately for each score.

A discrimination result was considered satisfactory if the Receiver operating characteristic (ROC) curve area was > 0.7, with statistical significance denoted as p ≤ 0.05. The AUC, with a 95% CI, was used to evaluate the discrimination performance of each score. All tests were two-sided, and values with a p ≤ 0.05 were statistically significant. All data analysis was performed using Stata version 16 (StataCorp LLC, College Station, TX, USA).

Results

A total of 1150 adult diabetic patients were presented to the ED of Ramathibodi Hospital and were diagnosed with sepsis between January 1, 2017, and December 31, 2021. Patients with missing data (n = 66) and incomplete laboratory results (n = 532) were excluded from the analysis. Consequently, 552 patients remained eligible and were included in the analysis (Fig. 1). Among these, 152 individuals (27.54%) progressed to septic shock within 48 h.

Table 1 shows the demographic and clinical characteristics of the patients. A higher incidence of septic shock was observed in males than in females, 77 (50.66%) vs. 137 (34.25%) P < 0.001. Patients with septic shock had significantly higher percentages of underlying diseases of myocardial infarction, Parkinson’s disease and dementia or Alzheimer’s disease than sepsis (49 (32.24%) vs. 78 (19.50%) p = 0.001, 11 (7.24%) vs. 10 (2.50%) P = 0.009, 15 (9.87%) vs. 11 (2.75%) P < 0.001, respectively). However, patients with septic shock had significantly lower percentages of underlying diseases of hypertension, metastatic cancer and old cerebrovascular accident (CVA) than sepsis group (76 (50%) vs. 271 (67.75%) p < 0.001, 22 (14.47%) vs. 116 (29.00%) p < 0.001, 47 (30.92%) vs. 191 (47.75%) P < 0.001, respectively).

Table 1 Baseline characteristics of patients with diabetes categorized by shock

There were no significant differences in the mean age, body mass index (BMI) > 30 kg/m², mean hemoglobin A1C (HbA1C) value, poorly controlled DM patients, Emergency Medical Service (EMS) route, surgical service, or the source of infection between the two groups. The genitourinary tract infection and lower respiratory tract infection were the first and second most common sources of infection in both groups. Physical examinations and laboratory parameters showed that systolic blood pressure (SBP), diastolic blood pressure (DBP), the shock index (SI), peripheral oxygen saturation (SpO2), Glasgow Coma scale (GCS), urine output, blood urea nitrogen (BUN), creatinine (Cr), bicarbonate (HCO₃⁻) values, serum albumin, pH, and lactate concentrations were significantly different between two groups (all p < 0.05). The median scores for triage parameters: Emergency Severity Index (ESI), qSOFA, SIRS, REWS, MEWS, SAPS II, SOFA and APACHE II significantly differed between two groups (all p < 0.001).

Table 2 reveals that patients with septic shock had a significantly more extended hospital stay than those in the sepsis group (p < 0.001). We found a significantly higher rate of admission to the intensive care unit (ICU) in the septic shock group than in the sepsis group (p < 0.001). Moreover, the in-hospital mortality rates was approximately 2% for patients with poorly controlled DM who did not progress to septic shock and 20% for those who did. In patients with well-controlled DM, the in-hospital mortality rate was approximately 3% for those who did not progress to septic shock and 15% for those who did. Mortality rates were significantly higher in the septic shock group compared to the sepsis group for both poorly controlled DM (p < 0.001) and well-controlled DM (p = 0.039). When lactate concentrations exceeded ≥ 2 mmol/L and ≥ 4 mmol/L, the in-hospital mortality rates were significantly higher in the septic shock group than in the sepsis group (22.88% vs. 2.67%, p < 0.001 and 36.59% vs. 6.52%, p = 0.001, respectively). The in-hospital mortality rates, when antibiotics were administered within 1 h and delayed longer than 1 h, were significantly higher in the septic shock group than in the sepsis group (both p < 0.001). The median length of stay in the ED in patients discharged from the ED was not significantly different between the groups.

Table 2 Outcomes of diabetes patients with sepsis categorized by shock

Table 3 shows the predictive performances of scores for septic shock onset within 48 h using the area under the Receiver operating characteristic (AUROC) curve. Notably, a SOFA score ≥ 6 (supplementary Table 1) showed robust discrimination at 0.866 (95% confidence interval [CI] 0.84–0.90, p < 0.001), with an optimal cutoff of 5.5 points and a maximum Youden index of 0.732. In contrast, MEWS, SAPS II, and APACHE II scores lacked effective discrimination. The MEWS score (cutoff: 5.5 points) had a Youden index of 0.208, the SAPS II score (cutoff: 44.5 points) had a Youden index of 0.369, and the APACHE II score (cutoff: 20.5 points) had a Youden index of 0.322. Their AUROC values were 0.647 (95% CI 0.60–0.70), 0.707 (95% CI 0.66–0.76), and 0.668 (95% CI 0.61–0.72), respectively. These results are shown in Fig. 2 and supplementary Fig. 1 for predicted shock 48 h post-ED visit. Additionally, the score predicting shock after a 2-hour ED visit was significantly good for a SOFA score ≥ 6 (AUROC = 0.840, 95% CI 0.81–0.87, p < 0.001), but the other scores showed insufficient discriminatory power (Fig. 3, supplementary Table 2, supplementary Figs. 2–3). A subgroup analysis for well controlled and poorly controlled DM identified the SOFA score as the best predictor, with AUROCs of 0.938 (95% CI 0.91–0.96) and 0.941 (95% CI 0.90–0.99), respectively (supplementary Table 3). Multivariable logistic regression analysis for septic shock showed significant results for SBP, SpO2, GCS, pH, and lactate concentrations (r² = 0.25, all p < 0.05, supplementary Table 4).

Table 3 Diagnostic accuracy of MEWS, SAPS II, SOFA, and APACHE II scores to predict shock in patients with diabetes within 48 h after visiting the ED

Fig. 2

Diagnostic accuracy of MEWS, SAPS II, SOFA, and APACHE II scores to predict shock in patients with diabetes within 48 h after visiting the ED

Fig. 3

Diagnostic accuracy of MEWS, SAPS II, SOFA, and APACHE II scores to predict shock in patients with diabetes 2 h after an ED visit (n = 130)

Discussion

In our study, patients in the sepsis group had a mean age of 76.43 years, and those in the septic shock group had a mean age of 74.27 years, with no significant difference between them. These findings are consistent with a study by Esposito et al. These findings suggest an increased risk of sepsis in individuals aged older than 60 years. Additionally, this study showed an association between male sex and septic shock, suggesting that male sex hormones impair cell-mediated immune responses. [17]

This study shows nearly 40% of the patients (from both cohorts) did not receive antibiotics within 1 h and almost 10% of septic shock patients were discharged from ED. Early prediction of septic shock could be utilized in these cases and hopefully alter the mortality. There were 66.45% of patients with septic shock admitted to the ICU, while the remainder discontinued vasopressor/inotropic drugs in the ED, awaiting ICU admission. Notably, patients with sepsis showed a higher direct discharge rate from the ED than those with septic shock. Additionally, there was no significant difference in the length of stay of patients discharged from the ED between the two groups. The median time of ED-LOS with discharge from ED in patients with sepsis was 50.5 h (IQR; 15.5, 77) and in septic shock group was 51 h (IQR; 31.5, 78). Alternatively, this finding could have been limited by our study’s single-center design and small sample size, especially compared with a study by Calvert et al. on 29,083 patients who progressed to septic shock. [18] In our study, the in-hospital mortality rate of septic shock group was 19.08%, consistent with Wardi et al.’s study. [19] Poorly controlled diabetes or point-of-care-testing glucose; ≥ 200, < 70 mg/dL, did not show a correlation with the overall mortality rate. However, this study shows the mortality rate for patients with poorly controlled diabetes was ten times higher than that for patients with sepsis who developed septic shock. In contrast, the mortality rate for patients with well controlled diabetes was five times higher compared to these groups. Costantini et al. suggested ongoing controversy regarding the effect of diabetes on sepsis mortality. [4] The high mortality rates in diabetic patients with sepsis and septic shock underscore the need for early recognition, prompt treatment, and tight glycemic control in these high-risk individuals. Furthermore, optimizing diabetes management is crucial to minimizing infection risk and severity. Notably, in our study, metastatic cancer cases were significantly more prevalent in patients with sepsis than in those with septic shock. This difference between studies may be due to the exclusion of palliative care in septic shock cases, affecting SAPS II and APACHE II scores.

We found that SBP, SpO2, GCS, pH, and lactate concentrations were predictors of septic shock, which is consistent with the findings of Wardi et al., who have utilized these parameters to predict septic shock, particularly in emergency settings. These parameters serve as critical indicators of patient status and can stratify patients based on their risk for severe outcomes, thereby guiding clinical decisions regarding the level of care required, such as ICU admission. Notably, various triage scores (ESI, qSOFA, SIRS, REWS, MEWS, SOFA, SAPS II, and APACHE II) were significantly different between patients with sepsis and those with septic shock, which suggested their usefulness in predicting the severity of sepsis. [19]

This study showed that the SOFA score was the optimal predictor in predicting septic shock, with a cutoff point at ≥ 6 and an impressive AUROC range of 0.8–0.9 within 48 h post-ED visit and within 2 h for patients with diabetes. When diagnosing septic shock, physicians must consider the infection source, hypotension requiring vasopressors to maintain mean arterial pressure > 65 mmHg, and adequate fluid resuscitation (evaluated by clinical and ultrasound criteria). Additionally, recognizing the operator-dependent nature of ultrasound is essential. Importantly, in our study, the operators (most were residents trained in emergency medicine) obtained approval through resuscitation and ultrasonography courses.

MEWS, SAPS II, SOFA, and APACHE II scores involve SBP. The SOFA score, which integrates values related to vasopressor/inotropic drug use and mean arterial pressure, remains relevant, even though these values were collected in our study before the onset of septic shock. Based on these factors, we recommend using the SOFA score with a cutoff point of ≥ 6 because of its high sensitivity and specificity in predicting septic shock. Regular monitoring of this score could facilitate the early identification of septic shock within 48 h and, notably, as early as 2 h post-ED visit.

Conclusion

SBP, SpO2, GCS, pH, and lactate concentrations are crucial for the early prediction of septic shock in patients with diabetes. The SOFA score is a superior predictor for the onset of septic shock in patients with diabetes compared with MEWS, SAPS II, and APACHE II scores. Specifically, a cutoff of ≥ 6 in the SOFA score demonstrates high accuracy in predicting shock within 48 h post-ED visit and as early as 2 h after ED admission.

Limitations

A study at a specific tertiary care hospital may not fully represent the broader population because of given the generally less complex nature of patients in rural hospitals. Therefore, it is unclear whether the patient characteristics, treatment protocols, and outcomes would be representative of diabetic patients presenting with sepsis at other institutions or in different geographic regions. The limited sample size, falling short of 693 events, could affect our results, especially non-statistical outcomes. This influence effect could be mitigated through an increase in sample size. Diagnosing septic shock involves clinical and operator-dependent ultrasound assessments, contributing to the overall complexity of the diagnosis. For improvement, data collection mandates approval from operators with completed resuscitation and ultrasonography courses, enhancing methodological rigor.