ONLINE SUPPLEMENT
Title: Reporting of Sepsis Cases for Performance Measurement versus for Reimbursement In New York State
Contents
eTable 1: Four Methods to Identify Severe Sepsis and Septic Shock Page 2 in Discharge Records
Appendix 1: Methods Supplement Page 3
eTable2: Hierarchical Matching Algorithm to Link Sepsis Cases
to Discharge Records in SPARCS Page 5
eTable 3: ICD-9-CM Coding of Matched Sepsis Cases Page 6 eTable 4: Patient Characteristics of Discharges
with a Diagnosis of Severe Sepsis or Septic Shock
(Coded Sepsis Discharges), by Reporting Status Page 7 eTable 5: Crude in-hospital mortality of Discharges meeting Page 10 Dombrovskiy or modified Angus criteria that were reported
versus not reported to New York State Department of Health
eFigure 1 Correlation between Percent of Discharges Sepsis Cases
Reported and Protocol Initiation among Eligible Reported Cases Page 11 eFigure 2: Crude In-Hospital Mortality Over Time Page 12
eTable 1: Methods Used to Identify Sepsis Discharges in SPARCS Database
Name Conceptual Definition Operational Definition Test Characteristics
Principal Severe Sepsis/Septic Shock Diagnosis Code
Discharges that are primarily for severe sepsis or septic shock.
Discharges with a principal or primary ICD-9-CM diagnosis code of 995.92 (severe sepsis) or 785.52 (septic shock).
High specificity, but low sensitivity.
Any Severe Sepsis/Septic Shock Diagnosis Code
(Coded Sepsis)
Discharges with recognized sepsis, but the primary reason for hospitalization may be another diagnosis.
Discharges with an ICD-9-CM diagnosis code of 995.92 (severe sepsis) or 785.52 (septic shock) in any diagnosis field.
High sensitivity (100%), but low specificity (9.3%)1.
Domborovskiy Criteria
(Possible Sepsis)
Discharges that are likely for severe sepsis or septic shock, although we infer that the acute organ dysfunction is due to a dysregulated host response to infection.
Discharges with an ICD-9-CM diagnosis code for sepsis, severe sepsis, or septic shock and an ICD- 9-CM diagnosis code for acute organ dysfunction.
High specificity with modestly improved sensitivity over explicit diagnosis.
Angus Criteria (Possible Sepsis)
Discharges that are likely for severe sepsis or septic shock, although we infer that the acute organ dysfunction is due to a dysregulated host response to infection.
Discharges with an ICD-9-CM diagnosis code for infection and an ICD-9-CM diagnosis code for acute organ dysfunction; or an explicit diagnosis of 995.92 (severe sepsis) or 785.52 (septic shock).
Slightly lower specificity, but greater sensitivity than severe sepsis / septic shock diagnosis code or Dombrovskiy approach (50.3%)1.
1. Iwashyna, T. J. et al. Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care 52, e39-43 (2014).
Appendix 1: Methods Supplement
Study Populations
Our primary analysis examined inpatient and ED discharge records with a primary or secondary ICD-9-CM diagnosis code for severe sepsis (995.92) or septic shock
(785.52)—which we call “coded sepsis discharges”. In a prior study, this approach had a positive predictive value of 100.0% (95% CI 76.8%, 100.0%) relative to structured chart review by trained hospitalists12.
We also examined alternative approaches including (1) discharges with a principal diagnosis of severe sepsis or septic shock, and (2) “possible sepsis discharges”, defined by Dombrovskiy criteria14 (diagnosis codes for both sepsis and acute organ dysfunction) and modified Angus criteria12,15 (diagnosis codes for both infection and certain acute organ dysfunctions, or a code for severe sepsis or septic shock). We considered discharges meeting Dombrovskiy or Angus criteria as possible sepsis discharges, as positive predictive values (PPV) of these definitions are estimated to be around 70% and 30%, respectively12,13.
Matching Reported Sepsis Cases to Discharge Records
We used a hierarchical deterministic algorithm to match cases in the Sepsis Clinical Database to discharge records in SPARCS using unique patient and facility identifiers, and other variables (e.g. dates of service, discharge location) common to both datasets (eTable2). We first matched cases by enhanced unique patient identifier, admission date, discharge date, unique facility identifier, date of birth, sex, and disposition. We then performed a series of additional matches on the remaining cases using relaxed criteria, e.g. admission date +/-3 days, discharge date +/- 3 days, etc. Most cases (65%) were matched using complete data (eTable2).
Comparing Characteristics of Reported versus Unreported Sepsis Discharges To explore patterns of incomplete reporting, we compared the characteristics, crude in- hospital mortality, and adjusted in-hospital mortality of reported versus unreported
sepsis discharges. Adjusted analyses accounted for age, gender, race, payer, and individual Charlson comorbidities18,19. In a separate analysis, we also accounted for individual acute organ dysfunctions coded during the hospitalization, applying the acute organ dysfunction codes used by Dombrovskiy16.
Sensitivity Analyses
To estimate the potential number of true sepsis discharges that were unreported (including those without a diagnosis code for severe sepsis or septic shock), we multiplied the number of possible sepsis discharges by the respective PPVs (70% for Dombrovsky and 30% for Angus) to estimate the true number of sepsis discharges, then subtracted the number of reported discharges:
(1) Number of Possible sepsis
discharges in SPARCS x PPV = Estimated number of true sepsis discharges
(2)
Estimated number of true sepsis
discharges -
Number Reported
(Number of SPARCS discharges matched to Sepsis Clinical Database
plus the number of unmatched Cases in Sepsis Clinical Database)
=
Estimated number of true sepsis discharges that were
not reported
To understand differences in reported versus unreported discharges, we calculated crude in-hospital mortality of reported versus unreported discharges meeting
Dombrovskiy criteria, and separately for reported versus unreported discharges meeting modified Angus criteria.
ICD-9-CM to ICD-10-CM conversion
In October 2015, New York State hospitals switched from International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes to International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) to report diagnoses and procedures. Through forward and backward mapping, we converted ICD-10-CM codes to ICD-9-CM codes using CMS’s general equivalency mapping12, prior to employing ICD-9-CM-based definitions of severe sepsis and septic shock. We manually reviewed all ICD conversions to ensure their appropriateness.
eTable 2: Hierarchical Matching Algorithm to Link Cases in the Sepsis Clinical Database to Discharge Records in SPARCS
Round UPIDE
Modified UPID
(No SSN)
Patient Control Number
Medical Record Number
Admit Date
Disch.
Date PFI DOB Sex Disp.
Number of Matched
Clinical Cases
Percent of Total
Clinical Cases
Cumulative Total of Matched
Clinical Cases
Cumulative Percentage of Total Clinical Cases
Records in SCD at Start
of Round
Records in SPARCS at Start of Round Merge to Hospital Discharge Records
1 X X X X X X X 72,269 64.6% 72,269 65% 111,816 4,503,703
2 X +/- 3 X X X X X 11,964 10.7% 84,233 75% 27,583 4,419,470
3 X X +/- 3 X X X X 477 0.4% 84,710 76% 27,106 4,418,993
4 X +/- 3 +/- 3 X X X X 71 0.1% 84,781 76% 27,035 4,418,922
5 X X X X X X 8,872 7.9% 93,653 84% 18,163 4,410,050
6 X +/- 3 X X X X 1,714 1.5% 95,367 85% 16,449 4,408,336
7 X X +/- 3 X X X 168 0.2% 95,535 85% 16,281 4,408,168
8 X +/- 3 +/- 3 X X X 35 0.0% 95,570 85% 16,246 4,408,133
9 X X X X X X 4,133 3.7% 99,703 89% 12,113 4,404,000
10 X X X X X 2,321 2.1% 102,024 91% 9,792 4,401,679
11 X X +/- 3 X X X 51 0.0% 102,075 91% 9,741 4,401,628
12 X +/- 3 X X X X 696 0.6% 102,771 92% 9,045 4,400,932
13 X +/- 3 +/- 3 X X X 13 0.0% 102,784 92% 9,032 4,400,919
14a X X X X X 274 0.2% 103,058 92% 8,758 4,400,645
14b X X X X X 715 0.6% 103,773 93% 8,043 4,399,930
15a X X X X X 100 0.1% 103,873 93% 7,943 4,399,830
15b X X X X X 233 0.2% 104,106 93% 7,710 4,399,597
16a X +/- 3 X X X 332 0.3% 104,438 93% 7,378 4,399,265
16b X +/- 3 X X X 122 0.1% 104,560 94% 7,256 4,399,143
17a X X +/- 3 X X 21 0.0% 104,581 94% 7,235 4,399,122
17b X X +/-3 X X 6 0.0% 104,587 94% 7,229 4,399,116
18 +/- 3 X X X X 171 0.2% 104,758 94% 7,058 4,398,945
19 X +/- 3 X X X 13 0.0% 104,771 94% 7,045 4,398,932
20 +/- 3 +/- 3 X X X 14 0.0% 104,785 94% 7,031 4,398,918
21 X X +/- 3 +/- 3 ** X X 208 0.2% 104,993 94% 6,823 4,398,710
Merge to Emergency Department Discharge Records
23 X X X X X X X 527 0.5% 105,520 94% 6,296 12,337,707
24 X X X X X X 121 0.1% 105,641 94% 6,175 12,337,586
25 X X X X X X X 27 0.0% 105,668 95% 6,148 12,337,559
26 X X +/- 3 X X X X 54 0.0% 105,722 94.5% 6,094 12,337,505
Abbreviations: DOB: Date of birth; PFI: a unique facility identifier; SCD: Sepsis Clinical Database; SPARCS: Statewide Planning and Research Cooperative System;
UPIDE: an expanded unique personal identifier; UPID: a unique patient identifier.** One hospital had discrepant unique facility identifiers in SPARCS and the Sepsis Clinical Database, which was manually corrected.
eTable 3: ICD-9-CM Coding of Matched Sepsis Cases (cases reported to Sepsis Clinical Database and matched to a discharge record in SPARCS)
ICD-9-CM Coding in Matched Discharge Record N, (% of all 105,722
matched cases) Principal diagnosis code for severe sepsis or septic shock, N (%) 629 (0.6%) Diagnosis code for severe sepsis or septic shock in any position, N (%) 95,355 (90.2%)
Positive Dombrovskiy criteria, N (%) 93,009 (88.0%)
Positive modified Angus criteria, N (%) 100,633 (95.2%)
No diagnosis code for severe sepsis / septic shock, negative Dombrovskiy, and negative modified
Angus criteria, N (%) 4,704 (4.2%)
No diagnosis code for infection, N (%) 1,147 (1.0%)
eTable 4: Patient Characteristics of Discharges with a Diagnosis of Severe Sepsis or Septic Shock (Coded Sepsis Discharges), by Reporting Status
Patient Characteristics
Unreported Discharges (Coded Sepsis
Discharges in SPARCS that were not
matched to Sepsis Clinical Database) (N=27,329 Total
Discharges) (N=26,748 Inpatient
Discharges)
Reported Discharges (Coded Sepsis
Discharges in SPARCS Matched to
Sepsis Clinical Database, and therefore confirmed as
reported) (N=95,355 Total
Discharges) (N=94,969 Inpatient
Discharges) p
Age, years, median (IQR) 72 (60-83) 72 (60-83) 0.005
Male, N (%) 13,911 (50.9%) 49,394 (51.8%) 0.01
Race, N (%) <0.001
Hispanic 2,567 (9.4%) 9,090 (9.5%)
Non-Hispanic, Asian 1,094 (4.0%) 3,033 (3.2%)
Non-Hispanic, Black 5,366 (19.6%) 15,487 (16.2%) Non-Hispanic, Multi-Racial 118 (0.4%) 691 (0.7%) Non-Hispanic, Other 2,782 (10.2%) 9,115 (9.6%) Non-Hispanic, White 15,402 (56.4%) 57,939 (60.8%)
Primary Payer <0.001 Payer <0.001
Medicare 19,419 (71.1%) 69,529 (72.9%)
Medicaid 4,801 (17.6%) 15,079 (15.8%)
Commercial 2,545 (9.3%) 8,760 (9.2%)
Self-Pay 304 (1.1%) 1,006 (1.1%)
Other 256 (0.9%) 860 (0.9%)
Unknown 4 (0.0%) 121 (0.1%)
Charlson Comorbidity Index, med
(IQR) 2 (1, 3) 2 (1, 3) 0.005
Charlson Comorbidity Index 0.07
0 4,409 (16.1%) 15,171 (15.9%)
1 to 4 17,214 (63.0%) 60,845 (63.8%)
5 to 9 740 (2.7%) 2,380 (2.5%)
10 to 14 4,948 (18.1%) 16,907 (17.7%)
15 to 22 18 (0.1%) 52 (0.0%)
Acute Organ Dysfunctions
Respiratory, N (%) 11,188 (40.9%) 41,133 (43.1%) <0.001 Cardiovascular, N (%) 14,155 (51.8%) 60,099 (63.0%) <0.001 Coagulation, N (%) 4,323 (15.8%) 17,930 (18.8%) <0.001
Renal, N (%) 15,302 (56.0%) 57,085 (59,9%) <0.001
Hepatic, N (%) 1,615 (5.9%) 6,972 (7.3%) <0.001
Central Nervous System, N (%) 6,002 (22.0%) 20,752 (21.8%) 0.48 In-Hospital Mortality
Crude, mean (95%CI) 26.1% (25.6%,26.6%) 30.2% (29.9%,30.5%) <0.001 Partially adjusted*, mean (95%CI) 26.6% (25.5%, 27.8%) 30.7% (29.4%, 32.1%) 0.002 Fully adjusted**, mean (95% CI) 28.8% (28.8%, 29.7%) 30.1% (29.2%, 31.0%) 0.20 Diagnosis-Related Grouping (DRG), with Severity of Illness (SOI), N (%) (For Inpatient
Discharges Only)
<0.001 APR-DRG 720: Septicemia and
Disseminated Infections 15,521 (58.0%) 59,734 (62.9%) SOI Level 4 (Extreme) 8,800 (32.9%) 36,385 (38.3%) SOI Level 3 (Major) 5,778 (21.6%) 20,568 (21.7%)
SOI Level 2 (Moderate) 891 (3.3%) 2,646 (2.8%)
SOI Level 1 (Minor) 52 (0.2%) 135 (0.1%)
APR-DRG 710: Infectious & Parasitic Diseases Including HIV with O.R.
Procedure
2,316 (8.7%) 9,114 (9.6%)
SOI Level 4 (Extreme) 1,731 (6.5%) 7,410 (7.8%)
SOI Level 3 (Major) 542 (2.0%) 1,574 (1.7%)
SOI Level 2 (Moderate) 43 (0.2%) 116 (0.1%)
SOI Level 1 (Minor) 0 (0.0%) 14 (0.0%)
APR-DRG 005: Tracheostomy with MV 96+ Hours Without Extensive Procedure
876 (3.3%) 2,521 (2.7%)
SOI Level 4 (Extreme) 844 (3.2%) 2,468 (2.6%)
SOI Level 3 (Major) 31 (0.1%) 51 (0.1%)
SOI Level 1 (Minor) 0 (0.0%) 0 (0.0%) APR-DRG 004: Tracheostomy with
MV 96+ Hours with Extensive Procedure or ECMO
676 (2.5%) 2,177 (2.3%)
SOI Level 4 (Extreme) 670 (2.5%) 2,161 (2.3%)
SOI Level 3 (Major) 5 (0.0%) 16 (0.0%)
SOI Level 2 (Moderate) 1 (0.0%) 0 (0.0%)
SOI Level 1 (Minor) 0 (0.0%) 0 (0.0%)
Other 7,359 (27.5%) 21,423 (22.6%)
Abbreviations: APR-DRG: All Patient Refined-Diagnosis Related Group; SOI: Severity of Illness
Partially adjusted mortality adjusted for age, sex, race, pater, and, Charlson co-morbidities.
Charlson comorbidities and index were calculated from each SPARCS discharge record using the method of Deyo1. Fully adjusted mortality also adjusted for acute organ dysfunctions, abstracted from each SPARCS discharge record using the method of Dombrovskiy2.
eTable 5: Crude in-hospital mortality of Discharges meeting Dombrovskiy or modified Angus criteria that were reported versus not reported to New York State Department of Health
Sepsis Identification Methods
Unreported Discharges
(Sepsis Discharges in SPARCS that were not matched to Sepsis Clinical Database)
Reported Discharges (Coded Sepsis Discharges in SPARCS Matched to
Sepsis Clinical Database, and therefore confirmed as reported)
p
Discharges meeting
Dombrovskiy criteria 16.1% 31.5% <0.001
Discharges meeting
modified Angus criteria 8.6% 29.7% <0.001
eFigure 1: Correlation between Percent of Discharges Sepsis Cases Reported and Protocol Initiation among Eligible Reported Cases
eFigure 2: Crude In-Hospital Mortality Over Time
0%
10%
20%
30%
40%
50%
60%
1 2 3 4 5 6 7 8 9
Crude In-Hospital Mortality
Quarter Following Initiation of Mandated Reporting
Coded Sepsis Discharge in SPARCS, matched to Sepsis Clinical Database, no protocol initiated Coded Sepsis Discharge in SPARCS, matched to Sepsis Clinical Database and protocol initiated Coded Sepsis Discharge in SPARCS, not matched to Clinical Sepsis Database
