How CMS Will Calculate Your 2021 Star Rating

CMS made significant changes to the way the Hospital Star Ratings are calculated. They finalized the calculation in the CY 2021 Medicare Hospital Outpatient Prospective Payment System and Ambulatory Surgical Center Payment System Final Rule.

Here’s a high-level summary of the major changes that came out of that final rule.

• CMS overhauled the way they calculate your Star Rating. The new calculation is a simple average of the measures in the group. (Details below.)
• There were seven measure groups that make up the Star Rating calculation. CMS has reduced the groupings to five groups. (Details below.)
• CMS is placing hospitals in one of three peer groups based on the number of measure groups you submit. Your Star Rating is based on which peer group you are in. (Details below.)
• Critical Access Hospitals and Veterans Association Hospitals are included in the Star Rating system (although they may still opt out).

Note: CMS did not choose to peer group readmission measure scores based on dual-eligibility.

Ok, let’s dive into what the heck that all means.

Background and basic concept of the Star Ratings

The Star Ratings system began in 2016. In general, the new star ratings are published once a year during one of the quarterly Care Compare releases, but since inception, CMS has never consistently released during one specific quarter. 

The Star Ratings are calculated based on a hospital’s performance on certain measures found on the Care Compare website. In 2022, Star Ratings are updated in July. 

Hospitals submit data to the Hospital IQR, OQR, Value-Based Purchasing, Readmission Reduction and HAC Reduction programs. That information is published on Care Compare. CMS then takes certain measures submitted in each of these programs and calculates a Star Rating which also appears on the Care Compare website. So, your individual measure performance is on the website and the Star Rating attempts to summarize your performance by stars, so the public gets a general feel for your quality of care - one star is the worst and five stars is the best.

Here’s what the public sees.

How CMS Calculates your Star Rating

Step 1: Hospitals submit data to CMS quality programs.

You, as a hospital, go about your business of submitting measures to the CMS quality programs: Hospital IQR, OQR, Readmission Reduction, HAC Reduction and HVBP programs.

Step 2: CMS identifies which measures will be used to calculate the score.

CMS selects which measures they will use to make the Star Rating score. They have identified 51 quality measures. You can find the measures on the QualityNet website.

Step 3: CMS standardizes the measure scores

CMS takes your hospital’s performance on those CMS-selected measures and standardizes the measure score so it makes sense in the overall equation. Think about an inverse measure where lower scores are better. You wouldn’t want that score to count against a total score, so they standardize all measure scores. This method of standardizing is called Z-scoring.

Understanding Z-scoring

A Z score is statistical technique. It’s a standard deviation score which shows the amount of variation in a dataset. For the Overall Star Rating, Z-scores were produced by subtracting the national mean score from each hospital’s measure score and dividing that by the standard deviation across hospitals. Standard deviation is a number that measures how far data values are from their average.

For Example:

In addition to that, CMS changed the direction of all inverse measures to reflect better performance for those lower scores.

Step 4: Measures are grouped

The measures are put into groups based on the category. The categories were reduced to five groups. Because of the Meaningful Measures Initiative that CMS launched they cut out quite a few measures over time which means that some categories didn’t have a lot of measures in them. So, CMS combined Effectiveness of Care, Timeliness of Care and Efficient Use of Medical Imaging groups into one group called Timely and Effective Care.

The five measures groups are:

1. Mortality
2. Safety of Care
3. Readmissions
4. Patient Experience
5. Timely and Effective Care

Step 5: CMS calculates the measure group score using a simple average

In each measure group, CMS will calculate a simple average to determine a measure group score. This is where CMS made the biggest change. In previous years, CMS used something called a Latent Variable Model and a Winsorization strategy to come up with the group score.

Honestly, I never understood this calculation and turns out, I wasn’t the only one. In fact, the feedback to CMS was that the calculation was too difficult for hospitals to replicate and predict. How is a hospital supposed to know what to improve upon to adjust their star rating if the calculation is so complex?

This year they dropped the Latent Variable Model and the Winsorization strategy and went to a simple average.

A simple average takes the sum of measures with equal weights and divides by the number of measures a hospital reports.

For example:
If a hospital submits all eight measures in the Safety of Care measures group, the calculation would look like this.

There are 8 measures in the Safety of Care category. Let’s say a hospital reports all 8 measures.

100 divided by 8 is 12.5 so each of the 8 measures in this category would be worth 12.5% of the overall Safety of Care category score.

100% / 8 reported measures = 12.5%

Measure group score =
[(-1.13*0.125) + (-0.75*0.125) + (0.09*0.125) + (1.21*0.125) + (0.97*0.125) + (0.98*0.125) + (0.46*0.125) + (0.02*0.125)] = 0.23

Step 6: Group scores are applied a weight

A weight is applied to that measure group score. Your final score is calculated according to the weight of the category. The weights for the five categories are as follows:

1. Mortality = 22% weight
2. Safety of Care = 22% weight
3. Readmissions = 22% weight
4. Patient Experience = 22% weight
5. Timely and Effective Care = 12% weight

For Example:
Summary score = [(-0.70*0.22) + (0.23*0.22) + (-0.76*0.22) + (-1.13*0.22) + (-0.25*0.12)] = -0.55

Example of Summary Score Calculation and Star Rating Assignment

Step 7: Hospitals are excluded or score reweighted depending on the number of measures submitted.

A hospital must report at least three measure groups with at least three measures in each group, (must include groups Mortality or Safety of Care) in order to receive a star rating. Once the minimum measure and measure group thresholds are met, any additional measures a hospital reports would be included in the Overall Star Rating calculation.

Let’s say you met the minimum threshold, but you didn’t report any Timely and Effective Care measures, that measure group is subtracted from the total.

For Example:
100 (100-12=88)

Mortality weight: 22/88 = 25%

Safety of Care weight: 22/88 = 25%

Readmission weight: 22/88 = 25%

Patient Experience weight: 22/88 = 25%

Example of Reweighting for a Hospital Which Does Not Report Timely and Effective Care Measure Group

Step 8: Hospitals are put into peer groups based on the number of measures submitted

Hospitals are put into one of three peer groups depending on how many measures they submit. CMS believes that by grouping hospitals by how many measure groups they report, this will help capture key differences between hospitals like size, patient volume, case mix and service mix. Larger hospitals, for example, generally submit the most measures and smaller hospitals submit the fewest.

The three peer groups are:

1. Five measure groups submitted
2. Four measure groups submitted
3. Three measure groups submitted

Step 9: Within the peer groups, scores are split into 5 groups using the k-means clustering method

Within the peer grouping, scores are split into five groups using the k-means clustering method. Essentially the k-means clustering is an algorithm method which aims to make hospital scores results within a cluster more similar to each other than to a different cluster.

Step 10: Whichever group your score puts you in becomes your star rating

Whichever k-means clustering group your score puts you in becomes your star rating.

Note: Since hospitals in a peer group are being compared only to each other that means hospitals with the same score, but a different peer group, could receive different star ratings.

What to do next

Even with all the changes this year, your star rating is still based on your performance across the quality measures that make up the program. And how many stars you have may influence the credibility, efficiency, culture and financial state of your organization.

It’s important to understand that Star Ratings are similar to other CMS Hospital Quality Initiative programs in the fact that performance is based on retrospective data from previous years, which means your current performance isn’t reflected on Care Compare. For instance, the 2021 readmission performance published now is your performance between July 1, 2016 – June 30, 2019.^iv You are long past your ability to affect change to your 2022 or even 2023 Star Rating. Even as you read this you have less than a year to affect your performance in many measures for 2024!

Read more: Five Tactics to Improve Your Star Rating

Quality data is so important but unless you can harness the data it can be overwhelming. You must use your data to identify the factors that drive your hospital’s performance in these categories. You need a dashboard that helps you understand how your problem measures are trending over time. Most importantly your data must be actionable. You must be able to analyze the data and create strategies around improvement. By partnering with a strong analytics vendor like Medisolv, hospitals can focus on a quality improvement plan that will help them succeed with their value-based care initiatives, now and into the future.

With the window to influence Star Ratings closing, you must take action now. Don’t wait any longer to improve your performance. Start monitoring your performance now to protect your revenue and reputation two years down the road.

Sources:
i. https://public-inspection.federalregister.gov/2020-26819.pdf
ii. https://www.aha.org/system/files/media/file/2020/12/cms-issues-hospital-outpatient-asc-final-rule-including-maintenance-340b-cuts-bulletin-12-3-20.pdf
iii. https://www.sheppardhealthlaw.com/2020/12/articles/centers-for-medicare-and-medicaid-services-cms/new-criteria-hospital-quality-star-rating/
iv. https://qualitynet.cms.gov/inpatient/public-reporting/overall-ratings/data-collection