The HOUR Trial: Clinical Decision Rule for Opioid Overdose Patients in the Emergency Department

Background Information: In 2017, more than two-thirds of the 70,000 drug overdose deaths in the United States involved an opioid.¹ Many emergency departments are affected as opioid overdoses increased 30% from July 2016 through September 2017 in 52 areas in 45 states.² With the half-life of naloxone being between 60-90 minutes the appropriate disposition and observation time of these patients following naloxone reversal continues to be debated in the literature.^3,4, A study on heroin overdose patients treated and released by pre-hospital providers showed no deaths in the one-year period studied.⁵ While this only applies to isolated heroin use, other studies have shown no increased incidence of death within 48 hours of patients treated with naloxone for non-fentanyl opioid overdoses.^6,7 Many of these studies, however, suffer from poor follow up which is unsurprising given the difficulty in tracking patients with opiate use disorders. It is important to note that the increased presence of synthetic and long-acting opioids further complicates this topic. A systematic review of a clinical prediction rule known as the St. Paul’s Early Discharge Rule, concluded that ambulatory patients with normal vital signs, and a Glasgow Coma Scale (GCS) of 15 only needed 1 hour of observation prior to discharge.^8,9 The authors of this study sought to validate this single center derived rule and its six criteria.

Clinical Question:

• Is the St. Paul’s Early Discharge Rule a valid tool in safely discharging opiate overdose patients from the emergency department after a 1-hour observation period following the administration of naloxone?

What They Did:

• This was a prospective observational study at a large single urban academic center to determine if clinical judgement and/or the six-component clinical prediction rule could predict adverse events in patients 1 hour following prehospital naloxone administration. After patient discharge, the hospital record was reviewed by three medical students for the presence of adverse events. One of two board certified emergency physicians reviewed all charts with one or more unclear adverse events.

Inclusion Criteria:

• Adult patients ≥ 18 years old
• Treated by emergency medical services, firefighters, police or laypersons with naloxone

Exclusion Criteria:

• Patients who:
  • Were prisoners or under arrest
  • Did not receive a 1-hour evaluation
  • Had an incomplete or otherwise normal 1-hour evaluation
  • Received in-hospital naloxone prior to the 1-hour evaluation
  • Requested to be withdrawn from the study
  • Could not have their study data linked to their hospital records

Outcome:

• To predict the risk of adverse outcomes using:
  • The prediction rule and each of its individual components
  • Provider judgement
  • Combination of prediction rule and provider judgements
• Local county medical examiner records were queried for patient deaths within 48 hours. All deaths within 48 hours were considered an adverse effect
• The authors divided adverse events as clearly defined or unclear as depicted in the table below:

Results:

• A total of 690 patients were screened for inclusion from May 2016 to September 2017.
• 538 of the total screened patients (78%) met inclusion/exclusion criteria.
• 82 of these (15.4%) had adverse events
• 4% of the enrolled patients had ED length of stays greater than 4 hours
• Review of medical examiner records, no patients were confirmed to have died within 48 hours

Critical Results:

• Prediction rule’s ability to predict adverse effects:
  • Not having the ability to mobilize as usual had the greatest sensitivity (58.0%)
  • Not having a normal temperature had the greatest specificity (99.1%)
  • Overall negative predictive value of 95.6%
  • Failed to predict adverse events in 13 (2.4%) of the total 538 enrolled patients
• Provider judgement to predict adverse effects:
  • Overall negative predictive value of 95.8%
  • Failed to predict adverse events in 12 (2.3%) of 529 cases

• Provider judgement plus prediction rule:
  • Overall negative predictive value of 96.0%
  • Failed to predict adverse events in 10 (1.9%) of 529 cases

Strengths:

• Large sample size that mirrored that of the original derivation study.
• The authors recognize and elaborate on the similarities and differences between their validation study and the original derivation study, such as the route of naloxone administration.
• Similar to the derivation study, inclusion criteria were not limited based on the drug or route of administration used thus making it a more reliable validation study
• Had a very well defined and systematic process for reviewing charts for adverse effects after patient was discharged which included blinding of both the medical students and emergency physicians
• Reported out clearly defined and unclear adverse events and a process for reviewing those records
• Attempted to limit systematic bias by assessing for hospital wide adverse events in 50 excluded cases
• Acknowledge and repeatedly emphasize that the rule should be used with caution in cases of known oral or mixed overdose

Limitations:

• Study does not have adequate follow-up to assess whether an adverse event occurred after discharge. Investigators reviewed the patient charts from the visit and checked the local medical examiner records for deaths only. Looking for adverse effects in this high-risk patient population during the small snapshot of time they are in the emergency department is not sufficient enough to claim there were no deaths or other serious life-threatening adverse effects.
• The authors did not specify if enrollment was 24-hours or during typical business hours (Monday-Friday, 8am – 5pm). This in addition to not specifying the route and type of opioid involved makes the population of the study a convenience sample rather then an accurate representation of these high-risk patients
• Having taken place at a single urban academic tertiary care center, this limits the study’s overall generalizability as a large number of emergency physicians work in smaller community hospitals.
• The opiates abused or heroin supply in the region of this study may be markedly different than that in the rest of the northeast, the south or midwestern US.
• EMS transporting a disproportionate number of overdose patients to the study site due to availability of specialized substance abuse and psychiatric services possibly introduces a selection bias.
• Due to a broad inclusion criterion, the authors acknowledge it is not possible to determine the performance of the rule on patients following parenteral opioid overdose
• Some of the patients treated with naloxone for presumed opioid overdose may not have actually overdosed on opioids and had other substances contributing to their symptoms.
• While the authors utilized a Cohen’s Kappa Coefficient to assess for inter-rater agreement both among the medical student and emergency physician reviewers, they did not mention what this value was or any interpretation of this assessment.
• The authors fail to mention what dose of naloxone these patients received either in the pre-hospital or emergency department setting.
• Although blinded, one of the reviewing emergency physicians was the author of this study thus possibly introducing experimenter bias

Discussion:

• The lack of appropriate, exhaustive follow up in this study is a major issue. Patients with opiate abuse issues are unlikely to only be seen in a single ED or even in a single county. Additionally, these patients often will present without identification or under pseudonyms. Finally, without proper identification, a patient death may be registered as a “Jane/John Doe” and, thus, not be captured as an event. It is very likely that this study grossly underestimates the risk of an adverse event and the risk of death.
• Even if the numbers here are correct, a sensitivity in the low 80s is unacceptable particularly when we see that the lower end of the CI drops the sensitivity down to 76%.
• The original derivation study had an adverse event rate of 16% and a negative predictive value of 99%. Despite differences in study design and population, this study had a similar adverse event rate.
• One important difference to note between the two studies is that the majority of naloxone (85%) in the validation study was given intranasally vs the derivation study. Nearly 100% of naloxone was administered subcutaneous, intravenously and intramuscularly.
• Another difference the authors recognize, is that a larger number of patients had a greater than 4-hour ED length of stay when compared to the derivation study and that it may predispose them to receive additional care.
• The prediction rule had lower sensitivity and negative predictive value when compared to the derivation study due to this study being a different patient population. The authors emphasize the application of the rule on various patient populations as being a necessary to establish generalizability of the rule.
• The observation period of opioid overdose patient varies throughout the literature from 2-hours to 4-6 hours.^3,4 The study hospital utilizes a 4-hour observation period following the administration of naloxone.
• The authors conclude by stressing that what they deem as low risk patients based on the lack of adverse effects, still have the potential to deteriorate.
• The authors emphasize that clinical prediction rules are best for answering binary questions. However, in this instance, we recommend that instead of using this rule, more time should be invested in obtaining a thorough history and physical to not exclude other possible substances overdosed on. This may help prevent premature closure and anchoring bias often associated with clinical improvement following administration of naloxone.
• Finally, overdoses are rarely a single agent and often due to polysubstance ingestion, with synthetic and/or long-acting opioids being involved. Therefore, it is very difficult to say that an observation time of anything less than 4 hours, especially 1 hour, is a sufficient in ruling out adverse events.

Author’s Conclusions:

The St. Paul’s Early Discharge Rule appears to be useful for identifying suspected opioid overdose patients treated with naloxone who are safe for discharge one hour after administration. This prediction rule works when naloxone is administered intra-nasally in a population where synthetic opioids are more common than the original study. Further studies are needed to determine the rule’s performance in the context of drug combinations and different routes of opioid administration.

Our Conclusion:

The St. Paul Early Discharge Rule has very limited utility and was found to be no better than clinical gestalt at detecting adverse effects in overdose patients who received intranasal naloxone. We do not recommend the use of this rule as it may be harmful to the patient and implies a focus on throughput and disposition. Opiate overdose patients should be observed for 4-6 hours to allow for safe disposition. The additional time should be used to counsel patients on safe use, referral for treatment, distribution of naloxone to them or their friends, opiate education, risk modification, safe-injection site locations and other patient-centered factors.

REFERENCES:

1. Scholl L, et al. Drug and Opioid-Involved Overdose Deaths — United States, 2013–2017. MMWR Morb Mortal Wkly Rep 2019. PMID: 30605448
2. Vivolo-Kantor, AM, et al. Vital Signs: Trends in Emergency Department Visits for Suspected Opioid Overdoses–United States, July 2016-September 2017. Centers for Disease Control and Prevention. PMID: 29518069
3. Boyer EW. Management of opioid analgesic overdose. N Engl J Med 2012; PMID: 22784117
4. Clarke SF, et al. Naloxone in opioid poisoning: walking the tightrope. Emerg Med J 2005; PMID: 16113176
5. Vilke GM, et al. Are heroin overdose deaths related to patient release after prehospital treatment with naloxone? Prehospital Emergency Care, 1999. PMID: 10424852
6. Wampler DA, et al. No Deaths Associated with Patient Refusal of Transport After Naloxone-Reversed Opioid Overdose, Prehospital Emergency Care 2011. PMID: 21612385
7. Greene JA, et al Incidence of mortality due to rebound toxicity after ‘treat and release’ practices in prehospital opioid overdose care: a systematic review Emerg Med J.  PMID: 30580317
8. Christenson J, et al. Early discharge of patients with presumed opioid overdose: development of a clinical prediction rule. Acad Emerg Med 2000. PMID: 11015242
9. Eggleston W, Clemency BM. In response to: do heroin overdose patients require observation after receiving naloxone? Clin Toxicol 2017. PMID: 28140683

Peer Review by Anand Swaminathan, MD (Twitter: @EMSwami)

This article suffers from a number of major issues as detailed above. I see the following major issues:

• The recorded adverse event rate within 24 hours was 15%. This is unacceptably high particularly in a high-risk group
• The sensitivity of the rule was just 84% with a lower limit of the CI of 76%. Clinical decision instruments (CDIs) should be designed with high sensitivity and this is woefully low.
• The CDI performed no better than clinical judgement and when combined with clinical judgement, was only marginally better. If a CDI doesn’t improve upon what we are already doing, what’s the point? This CDI would use clinicians time without actually improving the ability to catch events.
• There was no rigorous follow up performed. The authors here simply reviewed charts of patients seen in their department for events occurring within 24 hours of discharge and examined the county ME records. This is woefully inadequate for any study and even worse in a study with this population. Typically, if you were looking for adverse events, you would examine records at all area hospitals since we know that patients seen in one ED may very well seek repeat care in a different ED. This is even more important in a group of opiate abusers as trips to many EDs is common. Additionally, only looking at county ME records is inappropriate. Again, area ME records should be reviewed. Making things harder is the fact that many opiate use disorder patients will register under different names or present as “Jane/John Doe” secondary to the stigma associated with the disease as well as the fact that many will present while intoxicated and be unable to give proper identification. This means that patients presenting with opiate overdose may re-present and die without ever being identified. The inadequate follow up attempt from this group as well as the difficulties with following up these patients makes the reported adverse event and death rates highly likely to be underestimates.

CDIs are most valuable when they meet a defined need and help clinicians reduce unnecessary testing or guide management. This CDI doesn’t really do any of this. It was found to be no better than provider judgement alone and a combination of provider judgement and the CDI was only marginally better than judgement alone. This CDI attempts to create faster discharge. The question is whether faster discharge helps patients with opiate use disorders and I can’t see how it does. Not only does rapid discharge open patients up to the risks of recurrence of the opiate toxidrome when the naloxone wears off but, it also misses an opportunity to engage patients in risk modification, substance abuse counseling and possible steps towards recovery. The observation time for these patients can be spent by giving them resources for recovery, discussing medication-assisted-therapy (MAT), educating about safe injection, assessing risk for infectious diseases (HIV, Hep C) etc.
How would we recommend doing this better? First, patients with opiate toxidromes do not require full reversal with naloxone. The goal of naloxone use is to restore adequate breathing and circulation and often this can be done with doses that don’t result in a fully awake patient in withdrawal. The patient who has been fully reversed, who is vomiting and actively trying to leave the department is the reason a CDI like this was created – to determine which of these patients are safe to go home. If we don’t fully reverse, this won’t be an issue.
That being said, sometimes patients are unintentionally fully reversed or, a large dose of naloxone is given to a patient with no vital signs in an attempt to save their life resulting in full reversal. All of us have done this and, sometimes we simply have to deal with the results. Early discharge isn’t the answer. We can mitigate symptoms of withdrawal and then engage the patient as above.
For those patients who have been reversed and appear stable, we recommend observing for 4-6 hours. This allows us to watch the patient for a couple of half-lives of naloxone and feel confident that the long-acting opiate that the patient took either intentionally or unintentionally won’t rear its head and cause a repeat situation.
Finally, we would like to remind readers of the purpose of CDIs. CDIs should be created to reduce testing and guide treatment where possible in order to help patients and clinicians. If the CDI isn’t designed to help patients, there’s no purpose in the CDI.

For More on This Topic Checkout:

• Ken Milne at the SGEM: Wake me up Before you go, go – Using the HOUR Rule
• Justin Morgenstern at First10EM: The HOUR Rule for Opioid Overdose (Clemency 2018)