Preparing for eCQM Validation (Audit!!)

It’s my guess that CMS’s decision to use the word Validation instead of Audit wasn’t an accident. I don’t know anyone who throws a party when they hear they’re being audited. Even reading about the audit process below will probably trigger some anxiety. So, I apologize in advance for any undue distress this may cause. I want you to have the most recent information at your fingertips in case you happen to be one of the “lucky” ones selected by CMS for Electronic Clinical Quality Measure (eCQM) validation. 

For the first time ever, eCQM data will be validated for accuracy. Starting with the eCQM data collected in calendar year 2025, you will need to achieve a 75% or better agreement rate to pass validation. If you don’t meet this threshold, your annual payment update will be affected. 

So, what does this mean? Take a deep breath and read on. I’m here to help you understand how we got to this point and what steps you need to take so you can be better prepared (and hopefully a little less anxious) for CMS validation. 

As always, I like to start with the basics.

Let’s review the definition and purpose of CMS Validation

CMS describes data validation as “Verification that data submitted can be reproduced by a trained abstractor using a standardized protocol.” Pretty straightforward, right? Someone abstracts your data to make sure it’s the same as what you submitted. There are a few reasons why this is important to CMS.

1. It holds hospitals accountable for the accuracy of data. 
2. It supports public reporting of data and ensures consumers are provided with accurate and complete information that reflects the quality of care provided at the hospital. 
3. Policy requires it. 

A little background on the history of CMS validation: 

CMS has been validating Quality measures for the past 10 years, and although eCQMs still feel “new” to a lot of us, they’ve been a part of the validation process for almost eight of those years. 

• 2013: CMS finalizes validation of chart-abstracted measures for the Hospital IQR Program, effective for Fiscal Year (FY) 2015. 

• 2018: CMS finalizes validation of eCQMs, requiring selected hospitals to submit at least 75% of sampled medical records timely and completely. Records are validated for completeness only, not data accuracy. 

• 2021: CMS aligns validation scoring processes, combining the scores for chart-abstracted measures and eCQMs. Requires 100% of the sampled eCQM medical records to be submitted. eCQM validation carries a weight of 0%. 

• 2024: Annual selection of up to 200 hospitals randomly and up to 200 hospitals using targeting criteria to participate in both eCQM and chart-abstracted validation. 

• 2025: CMS finalizes eCQM validation scoring based on the accuracy of data beginning with CY 2025, affecting the FY 2028 payment determination. Requires hospitals to achieve a minimum passing score of 75 % to pass validation (more on this later). 

eCQM Validation Prior to 2025 

For the past few years, hospitals selected for eCQM validation were asked to provide medical records for eight cases (randomly selected by CMS from the eCQM QRDA files submitted) per quarter for the calendar year. 

The cases were then reviewed to confirm all requested records were provided. This is what CMS refers to as auditing for “completeness.” Hospitals passed the audit by simply submitting all requested cases. If a hospital was unable to submit all cases, eCQM validation carried zero weight in scoring (i.e., no penalty) so “failing” eCQM validation was not a deal-breaker. 

And even though CMS only held hospitals to completeness to pass the pre-2025 audits, they went ahead and validated the eCQM data for accuracy anyway. Accuracy review involved comparing abstracted data elements from the submitted medical records to the data in the eCQM QRDA files. This served two purposes: 

1. Allowed CMS to provide confidential feedback to hospitals highlighting areas for improvement ahead of mandatory validation based on accuracy.
2. Gave CMS insight into how well hospitals were managing data accuracy. 

In summary, hospitals previously met eCQM validation requirements simply by providing the medical records requests. Hospitals were not penalized if the auditor abstracted data didn’t match the eCQM data in the QRDA files. 

In the last seven years, CMS has gathered and studied a lot of validation information. This has helped them understand how accurate and reliable eCQM data is. They determined the accuracy of the data by calculating an agreement rate. An agreement rate is how often the eCQM data submitted in QRDA files matches the medical record data reviewed by CMS during the validation process. 

In the early days, eCQM agreement rates were low. In recent years, hospitals selected for eCQM validation have consistently high agreement rates. In other words, eCQM data is now accurate. Congratulations to all of you who were previously selected for eCQM validation. Your commitment has paid off; you’ve finally managed to boost the accuracy (and hopefully the performance) of your eCQM data. 

And for the rest of you? Feeling a bit salty? Well, you can always send your complaints to the hospitals that paved the way. The high bar they set was the primary reason CMS decided it was time to flip the audit switch to accuracy and hold hospitals accountable for their eCQM data.  

Here’s a bit more information to help you understand the decision-making process: 

• For the most recently submitted eCQMs, the overall agreement rate across all measures was about 90%. Meaning, most measures were very close to being 100% accurate. 

• On the low end of agreement rates was STK-3, Anticoagulation Therapy for Atrial Fibrillation/Flutter. The average agreement rate was ~84%. 

• At 94%, the measure with the highest average agreement rate was STK-5, Antithrombotic Therapy by the End of Hospital Day Two. 

• You’ll note that even the low end of the average accuracy is well above a passing threshold of 75%. This supported CMS’s move to scoring hospitals’ eCQM data based on the accuracy of the data submitted.  

2025 Validation

There are several key pieces to the validation process that you must know and prepare for in case you are selected for audit. 

1. They will evaluate you based on the data you began collecting on January 1st, 2025.
2. CMS will provide selected hospitals with a random sampling of eight cases per quarter that must be submitted for validation. 
3. CMS auditors (Clinical Data Abstraction Center or CDAC) will abstract the data in the records submitted based on the specifications for each eCQM you submitted within the IQR program. The medical record must contain sufficient information for CDAC to determine measure eligibility and/or outcome. Meaning, you must submit enough data for CDAC to confirm which measures the case qualifies for as well as the measure population the patient qualified for. 
4. There is a new scoring system. The combined validation score for eCQMs and chart-abstracted measures is being replaced with two separate validation scores. This reflects the distinct reporting requirements, procedures, and nature of the measures. Each score will require a minimum of 75% to pass. 
   1. Your chart abstracted measure score must be >75% 
   2. Your eCQM score must be >75% 
5. The eCQM scoring uses the same methodology for calculation as chart abstracted measures. 
6. If the score is below 75%, the hospital fails validation and will be included in the targeted sample for validation in the following year. 
7. CMS removed the requirement for hospitals to submit 100% of eCQM medical records to pass validation, BUT, any missing eCQM medical records will be treated as mismatches and will count against the agreement rate. So, basically you're still required to submit all records. 
8. The changes to the data validation policies will have a significant impact on the FY 2028 payment determination. Hospitals must achieve at least a 75% validation score for eCQM data to receive the full annual payment update. Additionally, you must pass both the chart-abstracted measure validation and eCQM validation. If a hospital fails either validation requirement, it will not receive the full annual payment update. 

The table below compares the old validation policy to the new one:  

What this means for hospitals:

The new scoring system sets a high standard for data accuracy, requiring hospitals to ensure that their eCQM data meets the 75% agreement rate. To achieve this, hospitals must implement well-defined processes for managing eCQM data, ensuring that it is accurate, complete, and meets the validation requirements.  

Preparing for eCQM Data Validation 

Successful eCQM data validation requires a structured data management plan. In the ever-evolving landscape of healthcare, with information exchange, artificial intelligence, and digital measurement topping the list of hot topics, the accuracy and integrity of your data is becoming more important. And with this new eCQM validation policy, successful data management isn’t something that you can sweep under the rug and worry about next year. 

Not sure what data management means, don’t worry, I had to phone a friend to make sure I fully understood it. Justin DiStefano is Medisolv’s long-time VP of Engineering, and he spends his days dealing with the challenges of data management. Here’s his take as it applies to EHR / Health IT 

"Data management involves organizing and handling information to ensure it is collected, stored, processed, and used effectively. In the context of healthcare, particularly with Electronic Health Record (EHR) systems, data management includes gathering all relevant patient information, structuring it in a specific format, applying relevant codes, safely storing it in a database, ensuring it is in the correct format for various uses such as generating regulatory files like QRDA I, and keeping track of where the information came from within the EHR system to ensure accuracy and compliance. This process helps healthcare organizations maintain accurate and usable patient records and ensures that the information meets regulatory requirements." 

Hospitals will need an annual plan to review eCQM data capture, accuracy, completeness, mapping, storage, and QRDA file components. To be successful, representatives from both the IT and Quality teams must be involved. The most successful hospitals have clear processes in place with timelines, frequency, and ownership mapped out.  

Here are a few tips to help you get started: 

1. Educate team members. Make sure all stakeholders understand the basics of eCQMs, the significance of failing an audit, and are committed to your data management goals. 
2. Track your eCQM processes. Develop a system for tracking your eCQM processes that can be used as a central resource for eCQM management and for audit prep. At a minimum, you will need to track the following.
   1. The eCQMs being tracked and submitted by your hospital every year. 
   2. The EHR documentation process for each data element in each measure. Specifically, where each data element is documented and by whom. 
   3. The mapped codes for all data elements (e.g., is every data element mapped? Are they mapped to the correct codes? And do those codes align with the reporting year?). 
   4. The measure-related documentation that is captured in free text (notes) or paper. Come up with a plan to move this document to structured, mapped fields in your EHR. 
3. Agree on and commit to timelines for eCQM data review and frequency of review — it is imperative your review processes align with: 
   1. Annual regulatory updates. Know when new measures are added, and when specifications and codes are updated annually. 
   2. The eCQM reporting period start and end date. 
   3. Submission plans and timing. 
4. Communicate and discuss other changes that might impact your eCQM data 
   1. EHR updates 
   2. Mapping vendor updates 
   3. Documentation changes 
5. Consider conducting your own audit. Abstract a few patients across your eCQMs, does your data match? Is the result the same? 

Conclusion 

The Hospital IQR Program's data validation rules for 2025 are being changed to make the eCQM data submitted by hospitals more accurate and reliable. The change from focusing on completeness to accuracy shows the program's commitment to data integrity, hospital accountability, and open public reporting. 

While the risk of a penalty is worrisome, hospitals can handle these issues by teaching and training their staff, setting up clear and simple processes, and making sure they are well-prepared and communicate effectively. By doing this, hospitals can meet the new standards and improve how they manage data. This will increase trust in the eCQM results and show ways to improve patient care, which is good for both hospitals and the communities they serve.

Want to dive deeper? Connect with our eCQM experts, who can help you fine-tune your data capture process and ensure you’re fully prepared for CMS validation. Additionally, don’t miss our upcoming webinar, 'Navigating CMS's 2025 eCQM Validation Changes’, on May 15 at 1:00 pm EST. For the first time, we’re offering CPHQ CE credits, making this an excellent opportunity for professional development.