The ISARIC WHO Clinical Characterization Protocol: Risk Stratification of Patients Admitted to Hospital with COVID-19

Background: SARS-CoV-2 infection has resulted in a high mortality rate, with the majority of deaths resulting from respiratory failure. As waves of the pandemic continue to overwhelm healthcare systems throughout the world, a pragmatic risk stratification tool that would allow for the early identification of patients with COVID-19 infection who are at the highest risk of death could help guide the management of individual patients as well as resource utilization.  In a systematic review from April 2020, Wynants et al found that prediction models have rapidly entered the literature since the start of the covid-19 pandemic, but that they are of questionable quality and at high risk of bias, and as such are not ready for general use.  A prediction model based on a large cohort with high quality methods would be of great value to the medical community.  

Article: Knight SR, et al; ISARIC4C investigators. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ 2020. PMID: 32907855.

Clinical Question: Can an easy to use risk stratification score accurately risk stratify patients with COVID-19 infection as being at low, intermediate, high or very high risk of suffering in-hospital mortality?

What They Did:

• The International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) World Health Organization (WHO) Clinical Characterisation Protocol UK (CCP-UK) study is an ongoing prospective cohort study performed by the ISARIC Coronavirus Clinical Characterisation Consortium (ISARIC-4C) in 260 hospitals across England, Scotland, and Wales
• Prospective observational cohort study
• Develop and validate a risk score to predict mortality in patients admitted to the hospital with COVID-19
  • Derivation Cohort: Feb 6th to May 20th, 2020
  • Validation Cohort: May 21st to June 29th, 2020
• Potential predictor variables were selected based on three criteria:
  • Patient and clinical variables known to influence outcome in pneumonia and flu-like illness
  • Clinical biomarkers previously identified within the literature as potential predictors in patients with COVID-19
  • Values available for at least two thirds of patients within the cohort
• Compared 4C model with existing risk stratification scores:
  • Scores identified through a systematic literature search
  • Used area under the receiver operating characteristic curve statistic for comparison

Outcomes:

• Primary: In-hospital mortality

Inclusion:

• Consecutive patients aged ≥18 years
• Admitted to the hospital with COVID-19 at least 4 weeks prior to data extraction 

Exclusion: 

• Patients who were admitted within four weeks of data extraction

Results:

• 35463 patients were included in the derivation dataset 
  • Mortality rate = 32.2%
  • Median age = 73
  • Female = 41.7%
  • At least one comorbidity = 76.0%
• 22361 in the validation dataset 
  • Mortality rate = 30.1%
• 41 candidate predictor variables measured at hospital admission for model creation 
• After composite variable creation, 21 variables remained
• Eight important predictors were identified:
  • Age 
  • Sex
  • Number of comorbidities
  • Respiratory rate
  • Peripheral oxygen saturation
  • Glasgow coma scale
  • Urea level
  • C reactive protein
• These eight factors became the 4C Mortality Score – score range 0-21 points
• 4C Mortality Score showed high discrimination for mortality
  • Derivation cohort: area under the receiver operating characteristic curve 0.79, 95% confidence interval 0.78 to 0.79
  • Validation cohort: area under the receiver operating characteristic curve 0.77, 95% confidence interval 0.76 to 0.77 

• Compared discriminatory performance with 15 other pre-existing risk stratification scores:
  • 4C Score AUC: 0.61 to 0.76
  • Other COVID-19 Scores AUC: 0.63 to 0.73
  • Performance of the 4C model compared well against existing risk stratification scores

Strengths:

• Large study population
• Asks clinically important question
• The 4C Mortality Score uses patient demographics, clinical observations, and blood parameters that are routinely collected and available at the initial presentation, allowing for a prediction score to be generated early in the patient’s course.
• Temporal and geographical validation was performed to account for the changes in patient characteristics and baseline infections rates than can be expected in an ever changing pandemic, further increasing the clinical applicability of the score. 
• Used all available data to maximize the power and generalizability of results
• Model reliability was assessed using a temporally distinct validation cohort with sensitivity analyses
• Restricted analysis outcomes to patients who were admitted ≥4 weeks before final data extraction to enable most patients to complete their hospital admission
• Transparent and thorough explanation of variables included in derivation of 4C model

Limitations:

• The list of included comorbidities was not exhaustive and excluded some potentially relevant conditions, including hypertension, previous myocardial infarction and stroke.
• The 4C Morbidity Score was not compared to existing scores that required a large number of parameters (APACHE II & scores requiring advanced imaging modalities). While this makes the 4C Morbidity Score more clinically applicable, its value compared to these scores cannot be determined in this paper.
• Patients included were those admitted to the hospital, having a higher mortality and more advanced age than the general community population with COVID-19 infection.  The score is for inpatient use and cannot be applied in the outpatient setting and potentially cannot be generalized to younger patients.
• Predominantly caucasian population in a non-low income healthcare system.  These results may not extrapolate to lower income countries where preventative medicine is not as robust
• Patient population older (age ≈73 years) making application to younger populations not ready for primetime until further validation in this cohort of patients occurs

Discussion:

• Chose to give each comorbidity equal weight as previous evidence suggested an additive effect of comorbidity in patients with COVID-19.  Increasing number of comorbidities is associated with poorer outcomes
• Online calculator can be found at: https://isaric4c.net/risk/
• Discriminatory performance of existing COVID-19 scores applied to this cohort of patients was lower than reported in derivation.  This is most likely due to use of small inpatient cohorts resulting in generous results.  
• The authors were only able to apply other risk scores in less than 10% of their validation cohort because lactate dehydrogenase is not a routinely measured laboratory value in the UK

Author Conclusion: “An easy-to-use risk stratification score has been developed and validated based on commonly available parameters at hospital presentation. The 4C Mortality Score outperformed existing scores, showed utility to directly inform clinical decision making, and can be used to stratify patients admitted to hospital with covid-19 into different management groups. The score should be further validated to determine its applicability in other populations.”

Clinical Take Home Point: This risk-stratification score to predict risk of mortality for patients with COVID-19 infection was developed using a large COVID-19 dataset and has been thoroughly internally validated.  The patient population enrolled, however, will limit the widespread applicability at this time.  Once externally validated, this tool could prove useful in guiding resource allocation and determining disposition for COVID-19 patients.

For More Thoughts on This Topic Checkout:

• St. Emlyn’s Blog: Possibly the Best COVID-19 Risk Prediction Tool to Date
• PulmCrit: Oscar Awards for the Best COVID Prognostic Models

References:

1. Knight SR et al; ISARIC 4C investigators. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020. PMID: 32907855.
2. Wynants L et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020. PMID: 32265220

Post Peer Reviewed By: Salim R. Rezaie, MD (Twitter: @srrezaie)