One- to 10-year Status Epilepticus Mortality (SEM) score after 30 days of hospital discharge: development and validation using competing risks analysis

Abstract

Background

Status epilepticus (SE) is an emergency neurological disorder that affects quality of life and is associated with high mortality risk. Three scores have been developed to predict the risk of in-hospital death, but these scores are poor discrimination of mortality after discharge. This study aimed to develop and validate a simple risk score for long-term mortality in SE patients.

Methods

This retrospective cohort study was conducted using SE patient data collected from Thailand’s Universal Coverage Scheme database between the fiscal years of 2005 and 2015 and followed-up to 2016. Patients who died in hospital or within 30 days after discharge were excluded. Data were divided at random into either a derivation or validation set. A proportional hazards model for the sub-distribution of competing risks was fitted with backward stepwise method. The coefficients from the model were used to develop a point-based scoring system. The discrimination ability of the model was evaluated using a time-dependent receiver operating characteristic (ROC) curve.

Results

A total of 20,792 SE patients (with ages ranging from the first day of life to 99 years at first admission) were randomly separated into two groups: 13,910 in the development group and 6882 in the validation group. A sub-distribution hazard model was used to determine nine predictors to be included in the final model, which was, in turn, used to develop the scoring system: age (0–19 points), male (two points), brain tumor (12 points), stroke (three points), cancer (11 points), diabetes (three points), chronic kidney disease (five points), pneumonia (five points), and urinary tract infection (four points). The possible total score ranged from zero to 64 and the cumulative incidence function was used to determine the probability of mortality associated with each total score within the first 10 years after the first admission. The area under the ROC curve (AUC) of the first to last time point ranged from 0.760 to 0.738.

Conclusion

A nine-factor risk score for predicting 10-year mortality in SE patients was developed. Further studies should focus on external validity and including a range seizure types and duration of seizure as the predictors.

Background

Status epilepticus (SE) is an emergency neurological disorder that affects patients’ quality of life and is associated with high mortality risk. A recent meta-analysis found that the pooled crude annual incidence rate of SE was 12.6 per 100,000 person-years (95% confidence interval [CI]: 10.0 to 15.3) and the case fatality rate was 14.9% (95% CI: 11.7 to 18.7%) [1].

There are several factors that contribute to the risk of death in SE patients, but the key factors are age, duration of SE, and etiology [2, 3]. Moreover, previous studies found that comorbidities and complications were also major factors associated with mortality, for example, brain tumors, central nervous system (CNS) infection, septicemia, pneumonia, and shock [4,5,6,7]. Because death is a crucial outcome in SE, identifying the risk of death in patients with this condition can allow physicians to optimally manage their patients’ care on an individual basis. Risk scores have been developed to assess the risk of death in SE based on several important factors in order to assist health care professionals in the therapeutic decision-making process.

Currently, there are three available scores to predict a patient’s risk of in-hospital death. The most commonly used score is the Status Epilepticus Severity Score (STESS). This score was developed based on four predictors: consciousness, seizure type, age, and history of previous seizure [8]. Another is the Epidemiology-Based Mortality Score in Status Epilepticus (EMSE), which takes epidemiological data, including etiology, age, electroencephalogram (EEG), and comorbidity, into account [9]. Finally, the modified Status Epilepticus Severity Score (mSTESS) was developed from the STESS to more accurately predict mortality at discharge by including the patient’s Modified Rankin Scale (mRS) in with the other predictors used in the STESS [10].

The STESS has also been applied to predict long-term mortality in SE patients, but was found to be a poor discrimination of mortality after discharge (the area under the receiver operating characteristic curve [AUC] was 0.676 (95%CI: 0.516 to 0.833)) [11].

Most of the published studies mentioned above only followed SE patients through their hospital stay and lacked data predicting long-term mortality. The outcome of interest in this study is time-to-death after 30 days of hospital discharge. It was defined as time from the patient’s first admission with a primary diagnosis of SE to death which including both direct (e.g. seizure, epilepsy, SE) and indirect (e.g. SE complications, suicide, accidents, or underlying diseases [12]) causes of deaths by taking into account for the competing risks (e.g. senility, cardiovascular collapse). Thus, this study aimed to develop and validate a score to predict long-term mortality in SE patients that accounts for the presence of competing risks in order to avoid overestimating the of probability of death in SE, which tends to be a flaw in the conventional statistical methods that are used for this purpose [13, 14].

Methods

Design and setting

This was a retrospective cohort study. Data for this study was retrieved from the Thai Universal Coverage Scheme electronic database that recorded the information of over 75% of Thai citizens who admitted to hospitals within Universal Coverage Scheme. Data between the fiscal years of 2005 and 2015 (October 1, 2004 to September 30, 2015) were used and followed up for 1 year (until September 30, 2016).

Patients

Patients were eligible for inclusion in this study if their data were in the Thai Universal Coverage Scheme database and they were admitted to hospitals with a primarily diagnosis of SE based on the guidelines described in the International Statistical Classification of Diseases and Related Health Problems – 10th Revision (ICD-10) code G41 (Status epilepticus). The SE diagnosis was followed by the guideline of the International League Against Epilepsy in each version over past 10 years. There were no restrictions based on age, sex, or SE type. Eligibility was limited to those patients who had not died in hospital (discharge status of death) or within 30 days after discharge. Patients with incomplete data regarding their date of birth, date of admission, date of death, or cause of death were also excluded. All patient’ data were anonymized and de-identified prior to extraction and analysis.

Study variables

The outcome in this study was time from the patient’s first admission with a primary diagnosis of SE to death (including both direct and indirect causes). Deaths from the following causes according to the patients’ death certificates were considered to be event of interest: (a) seizure, epilepsy, SE, (b) accident, suicide, (c) SE complications (e.g. pneumonia, septicemia), and (d) comorbidities (e.g. cancer, diabetes mellitus; Additional file 1: Figure S1). Competing risks events were senility, cardiovascular collapse, etc. (Additional file 1: Table S1 provides the details regarding event of interest and competing events in this study). Patients who were still alive at the end of the study (September 30, 2016) were censored data.

Predictors in this study included baseline demographic characteristics (sex and age at first admission), comorbidities, and complications (occurred during the follow-up period). ICD-10 codes were utilized to identify comorbidities and complications (Additional file 1: Table S2).

Statistics

The eligible study subjects were randomly split into a derivation set and a validation set at a 2:1 ratio. Characteristics of patients in both sets were reported as frequency and percentage for categorical data and median with minimum and maximum value for continuous data. A sub-distribution hazard function was used to develop a predictive model with competing risks [15, 16]. Sensitivity analysis was performed based on selection algorithms, and the modified Bayesian information criterion for competing risks was used to select predictors for the final model [17]. The coefficients in the final sub-distribution hazard model were used as a scoring system that followed by the method of Austin [14]. Although the concordance index (c-index) has been widely used to evaluate the performance of predictive models with time-to-event outcomes, the c-index tends to be misleadingly high when used to predict t-year risk. Thus, we deemed the time-dependent receiver operating characteristic (ROC) curve to be more appropriate to assess differences at each year point [18, 19]. Statistical analyses were performed using the R statistical programming language [20] with the “crrstep” package [21] to select predictors, the “cmprsk” package [22] for fitting the sub-distribution hazard model, and “riskRegression” package [23] for assessing the time-dependent ROC.

Results

Among the 24,818 SE patients who were admitted to hospitals included in the Thai Universal Coverage Scheme database between the fiscal years of 2005 and 2015, 4026 cases were excluded because the patient died in hospital, died within 30 days after discharge, had an unknown cause of death, or had missing or erroneous data. Of the remaining 20,792 SE patients, 13,910 were randomly allocated to the derivation set and 6882 to the validation set (Fig. 1). There were 3594 cases of death due to direct and indirect causes (derivation = 2446, validation = 1148), and 1073 died due to competing risks events (derivation = 701, validation = 372). The median age of all participants was 31 years (range: 0–99), and 64% were male. Characteristics of patients in the derivation set and those in the validation set were comparable (Table 1).

Fig. 1

The inclusion flowchart

Table 1 Characteristics of admitted SE patients on the national database during fiscal year 2005–2015

Developing a predictive model from the derivation set

There were 17 candidate predictors that were included in the initial crude analysis with sub-distribution modelling (Table 1; Items marked with an asterisk [*]). After application of the forward or backward stepwise procedure, nine predictors remained relevant in the final model (Table 2).

Table 2 Predictors associated with long-term mortality in SE^a according to the sub-distribution hazard model

Developing a risk score from the predictive model using competing risks analysis

Of the nine predictors included in the final model, age at first admission, a continuous predictor, was classified into age categories at five-year intervals (Table 3). After following the steps required for the point-based scoring system, scores were assigned to all predictors. The possible total score ranged from zero to 64 (Table 4). The total score was used to determine the probability of event occurrence based on the cumulative incidence function for each year point (Table 5).

Table 3 Point-based risk scoring system for long-term mortality in SE^a

Table 4 Risk score for long-term mortality in SE^a

Table 5 Predicted mortality rates in SE^a with individual score totals

Validating the predictive model from both the derivation and validation sets

The AUCs in the derivation set were 0.760, 0.745, 0.742, 0.734, 0.741, 0.739, 0.741, 0.743, 0.742, and 0.738 at each year from years one to 10, respectively. In the validation set, the AUCs for each year from years one to 10 were 0.761, 0.743, 0.740, 0.734, 0.722, 0.725, 0.733, 0.737, 0.733, and 0.740 (Table 6).

Table 6 Discrimination in the derivation and validation sets

Discussion

This study developed and internally validated a new 10-year prediction score for mortality in SE patients after their first admission using a competing risks approach. This simple prediction risk score relied on demographic data, comorbidities, and complications (age, sex, brain tumor, stroke, cancer, diabetes, chronic kidney disease, pneumonia, and urinary tract infection). These finding should be considered with caution, because this study focused on the long-term mortality (outside hospital), we cannot guarantee that the indirect causes of death (accident, suicide, SE complication, and comorbidity) were due to SE. We found that age was a major predictor of mortality in SE patients, which is in accordance with results found using the STESS and EMSE [8, 9]. Similarly, brain tumor, stroke, diabetes, and chronic kidney disease have been identified as predictors using the EMSE [9]. Because this study focused on long-term mortality in SE patients, gender, cancer, pneumonia, and urinary tract infection scores differed from those found in previous studies. However, prior articles have found clear associations between these predictors and mortality in SE patients [4, 6, 7, 24], except the urinary tract infection which shown as a protective factor in previous study [7]. This is due to the difference group of patients – inpatients and after discharge within 30 days, and the outcome – all-cause mortality in SE.

The discriminatory capability of the proposed prediction model using the AUC at each year from years one to 10 were all over 70% for both the derivation and validation sets, which is slightly lower than that of the STESS (AUC = 0.760; in-hospital death) [8]. However, a recent study examining long-term mortality utilized the STESS and found that it had a poor discriminatory ability (AUC = 0.676) [11]. Another study found that the discrimination capability of the EMSE (AUC = 0.902; in-hospital death) [9] decreased when it was applied to 30-day mortality (AUC = 0.832) [25]. Although the performance of our model was generally consistent, the 95% CI of the AUC was wider in the later periods than earlier on due to the lower death rates in the later years [26].

There were three major strengths of this study as a result of our use of Thailand’s Universal Coverage Scheme dataset to develop the risk score. First, it allowed us to access a large sample size consisting of patient data from multicenter and follow them over 10 years. Second, as the database contains the information of patients in all age ranges, the score we developed can be generalized to SE patients of all ages. Lastly, the Universal Coverage Scheme database is connected to that of the Ministry of the Interior, which allowed us to follow up on all patients. However, the death of patients in this study is out-of-hospital mortality. If the patients died with natural causes, the definitive cause of death would not be identified because they were not performed by autopsy, even so, the sensitivity analysis found that the AUCs of first to last time points ranged from 0.722 to 0.692 that decreased by 0.038 to 0.046 when the patients who died with comorbidities were excluded. Furthermore, most of the healthcare units lacked the capability to perform EEG (difficult to specify the type of seizure). Therefore, most of patients in this study were diagnosed with convulsive SE and our study included all type of SE in order to avoid misclassification of SE types which be lead to underestimation or overestimation. Moreover, the database did not contain information about duration of seizure and etiology, which are the important predictors associated with death in SE patients. Thus, further studies should include seizure types, duration of seizure, and etiology as predictors. In addition, we lacked of external validation due to this database containing insufficient longitudinal data. Therefore, external validation should be performed in order to evaluate the generalizability of this study.

Conclusions

In this study, we developed the first simple clinical scoring system that considers competing risks to predict long-term mortality after first admission in SE patients. The score is based on demographic data, comorbidities, and complications (age, sex, brain tumor, stroke, cancer, diabetes, chronic kidney disease, pneumonia, and urinary tract infection). The AUCs of the first to last time point ranged from 0.760 to 0.738. This user-friendly score requires only simple information of patient that contained in the medical records; where clinical or laboratory data was not available. In addition, the risk score can assist the patients to realize their own risk and helps clinicians in their decision-making process about long-term plan for treatment or reduce risk of death, especially patients with high risk. Although this score can assist in the estimation of a prognosis in an individual patient after discharge as well as SE management, it still needs improvement and external validation.