Measuring resiliency in hospital quality performance during the pandemic era

COVID-19

Measuring resiliency in hospital quality performance during the pandemic era

By
Joel Streed

January 3, 2023

Hospital quality rankings and ratings, including the U.S. News & World Report’s Best Hospitals and Centers for Medicare & Medicaid Services (CMS) Overall Hospital Star Ratings have faced an unusual challenge in recent years: To what extent should hospitals be held accountable for traditional quality measure performance, given the unknown effects of the COVID-19 pandemic on hospital operations?

Some national ranking and rating groups have chosen to withhold reporting from several months, or even multiple years falling with in the COVID-19 pandemic. However, at Mayo Clinic, and across many leading academic medical centers, health services researchers and hospital quality leaders believe that hospital quality measurement becomes even more important during times of health system crisis.

Indeed, we believe our patients deserve to continue to receive high-quality care regardless of pandemics or other outside factors.

This conundrum in fact provides an opportunity for quality reporting stakeholders. Namely, the chance to develop a hospital quality resiliency measure or index. A recent article in JAMA discussed some potential basic criteria for measuring resiliency, or a hospital's ability to continue to:

Deliver high-quality care to patients admitted with COVID-19.
Deliver high-quality care to patients admitted with other acute care needs.
Provide service to their community, including elective surgeries, while mitigating disparities.
Protect staff well-being.

In the Science of Quality Measurement Program in the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery we have a data use agreement permitting researchers to use retrospective Medicare inpatient claims data to explore risk-adjusted trends in readmission and mortality among Medicare beneficiaries.

This data allowed us to identify some simple quality measurement options to assess the first three criteria. For example, 30-day mortality or readmissions among COVID-19 patients as well as the CMS 30-day readmission and mortality measures could be used to evaluate a hospital's ability to maintain a high quality of care for patients hospitalized for all acute care needs.

Or, as shown in the Figure 1 below, using Medicare claims data from 2020 we could assess inter-hospital 30-day mortality rates among non-COVID patients comparing all months of pandemic-era data versus only the months in which hospitals had below national average COVID-19 burden (i.e., percentage of patients admitted for COVID-19 compared to all other causes).

Figure 1: Intra-hospital correlation in risk-adjusted 30-day mortality among non-COVID patients with inclusion versus exclusion of high-COVID-burden data months

Figure 1 legend: Bubble size corresponds to hospital’s total volume of non-COVID Medicare inpatient encounters April 1, 2020, through Nov. 30, 2020. Intra-hospital performance on risk-adjusted 30-day mortality was highly correlated regardless of whether hospital months with higher-than-average COVID-19 burden were included (x-axis) or excluded (y-axis).

Likewise, Medicare inpatient claims could be used to assess volumes of common elective procedures to assess whether usual community needs were being met.

There isn't widely available national staff well-being data, so hospital-level measures would require more discovery. However, it is clear that evidence-based measurement of resiliency in hospital quality performance during the COVID-19 pandemic is possible and may yield surprising and informative results.

We look forward to investigating this further, and leading efforts to ensure continued high-quality patient care for today and future generations.

-- Ben Pollock, Ph.D., is the Robert D. and Patricia E. Kern Scientific Director for the Science of Quality Measurement.

###

Editor's note: This project illustrates one of several ways Dr. Pollock's team is using the CMS data, examining the following factors:

Patient-level factors for both targeted and nontargeted conditions.
Patient-level factors for inpatient service lines such as cardiology or neurology
Patient-level factors for diagnosis-related groups
Institution-level factors such as geography, patient demographics and overall patient volume

The data analysis will allow researchers to document granular trends or differential effects of patient outcomes that have not been previously examined in sufficient detail.

###