The Making of a Quality Measure: A Peek Behind the Curtain

There’s a lot that goes into developing a new quality measure before your quality department is required to collect and report it. Take a peek behind the curtain in this Q&A with two experts on quality measure development and testing.

A lot goes into a new quality measure before you collect and submit it for the first time—similar to what goes into a new prescription drug before a doctor prescribes it to a patient for the first time.

We put together a virtual roundtable discussion with a measure developer and a measure implementer to walk you through all the testing that happens to a single measure before it goes live at hospitals across the country.

Our guest quality measure implementer is Ken McCormick, executive vice president of client services at Medisolv. Our guest quality measure developer is Grace Glennon, research project manager and project lead at the Yale New Haven Health Services Corporation Center for Outcomes Research and Evaluation.

Medisolv: When does someone start developing a measure?

Ken McCormick: Most of the time, a quality measure is reacting to something that’s already been identified as a problem. A simple example is stroke discharge instructions measure. It’s important to give every single stroke patient instructions after discharge to help prevent another stroke or readmission to the hospital.

Medisolv: Let’s pick up on that. Where do measures come from?

Grace: From a measure developer perspective, it starts with, what we call ESLRs, which is short for environmental scan and literature reviews. We’re looking for issues that are rising in priority in the healthcare space, such as current news, new government policies or issues that come from patients, communities or providers. Then we investigate whether there’s a measure to assess that issue.

Ken: Once someone has identified a problem in a specific patient population, it’s traced back to a gap in care, or a missing or broken structure or process. The measure then tracks behaviors that ideally close the gap in care or create a new structure or process.

The stroke discharge instruction measure started with patients not being educated correctly and being readmitted to the hospital.

Medisolv: How much testing goes into a measure before it goes live with hospitals and health systems?

Ken: It depends on the measure. If it’s an actual, approved, endorsed measure (meaning it’s recognized by organizations like the National Quality Forum or NCQA) then it goes through a formal process.

The formal process revolves around a five-part lifecycle:

1. Measure Conceptualization
2. Measure Specification
3. Measure Testing
4. Measure Implementation
5. Measure Use and Maintenance

For more information, check out eCQI’s page on the eCQM lifecycle. 

If it’s a measure you’re developing for internal use only, you go through the same steps. You just don’t have to go through all the bureaucracy to get everyone’s stamp of approval.

Grace: The amount of testing also depends on the complexity of the measure. The more complex it is, the more time it will take to test.

Medisolv: What do you test a quality measure for?

Grace: There are two important aspects to testing measures. One is making sure they’re valid and reliable. The second is making sure a measure is valuable to patients and providers.

Ken: Data is also critically important to testing. Does the data you need for your measure exist in your EHR system or another system already? If so, can you easily harvest it? If the data doesn’t exist in your EHR system or anywhere else, can you add a data field to capture it? And if you add a field, will your doctors and nurses input the data?

You could come up with a measure that’s a great measure, but if you don’t have the data or if you don’t have anyone who will input the data, it’s useless.

Medisolv: Is there an ideal length of time it takes to test a measure?

Grace: A lot of variables affect how long it takes to test a measure: access to data, the efficiency of the technology you’re using, stakeholder buy-in. But it really comes down to how complex the measure is. You’re always trying to develop a new measure as quickly as possible, but you have to make sure that it's valid and reliable.

Ken: You can, however, accelerate the testing timeframe if the circumstances warrant it. COVID-19 is a great example. At Medisolv, we came up with and tested our COVID-19 inpatient mortality measure fairly quickly, because our clients needed it. We didn’t convene an expert panel of 10 physicians to talk about it for six months.

Medisolv: Last question. You’re both passionate about quality measure development and implementation. If there were a Quality Measure Hall of Fame, what do you think should be in it?

Ken: I think the new hybrid readmission measure is awesome. We’ve been working on it for seven years, and it’s going to be mandatory for hospitals starting on July 1, 2023. I like that CMS will publicly report the data. And I like the fact that it will be risk-adjusted. Hospitals will be on a level playing field, making comparisons valid. No more saying your readmission rates are higher because your patients are sicker.

Grace: Generally speaking, I appreciate outcomes measures because they hold someone accountable for a health outcome. Not for something that’s outside of their control but for something they’re supposed to do. In terms of specific outcomes measures, readmission and mortality measures will always be near and dear to my heart. That’s because from a patient’s perspective, they can understand them and, obviously, they’re important to them. Either you don’t make it out of the hospital, or you make it out and have to go back.

Medisolv: Thank you, Grace, and thank you, Ken. It’s great to talk with people who are passionate about quality measures and whose development and testing work behind the scenes makes the role of quality directors and departments so vitally important to patients everywhere.

Related: Learn more about the quality measures for hospital stroke care by reading our new blog post, “The Differences Between the 5 Major Stroke Measure Sets.”