- 1
-
-
Quality Measures Subcommittee
Partnership for Quality Measurement (PQM) E&M Guidebook Comments Due July 30
-
Partnership for Quality Measurement (PQM) E&M Guidebook Comments Due July 30
On June 30, Battelle, whose Partnership for Quality Measurement (PQM) recently took over the NQF’s contract as the consensus-based entity that vets measures for use in federal programs, hosted a webinar titled, “Introduction to PQM and the Endorsement & Maintenance (E&M) Process.” If interested, copied below are notes from this session. Feel free to share with other members who might be interested. After sitting through this 2-hour webinar, my overall impression is that they’ve done little to move away from the NQF’s administratively complex and bureaucratic process, other than a few minor improvements, adjustments, and name changes (but I guess the devil is in the details).
Reminders about upcoming comment and nomination periods:
- The PQM
recently released an E&M Guidebook that presents a more detailed
description of the processes summarized on today’s webinar. This
Guidebook is open for public comment through July 30.
More information can be found here. - The PQM
also released a PRMR-MSR Guidebook, which is open for public comment
through July 21. More information can be found here.
The PRMR process will replace the Measures Application Partnership
(MAP) process. The MSR process is an extension of an existing process that
results in recommendations about measures that should be removed from
federal programs. On July 10, PQM will host a webinar
on these two processes. Registration for that webinar can be
found here. - The PQM
also just announced that it is accepting committee nominations through July
30. Individuals can self-nominate to serve on the E&M
Committee. Organizations and individuals can submit a nomination to serve
on committees supporting PRMR and MSR. More information is
available here.
Note that all nominees must be members of the PQM, but membership
is free.
*If Dr. Tinloy is interested in continuing to participate in the MAP process, then he should nominate himself for the PRMR, noting his past appointment.
Notes from 6/30/23 Introduction to PQM and the E&M Process Webinar
To achieve a 6-month Evaluation and Maintenance (E&M) process while maintaining high standards for transparency and rigor, Battelle has enacted several key enhancements over NQF process: 1) leveraging the Scientific Methods Panel (SMP) to advance measure science; 2) retiring the Consensus Standards Approval Committee (CSAC); 3) establishing a more robust and transparent appeals process 4) leveraging a Novel Hybrid Delphi and Nominal Groups (NHDNG) technique (rigorous, evidence-based approach to build consensus and prevent loud voices from taking over); 5) reducing the number of E&M committees; and 6) conducting the pre-evaluation public commenting concurrent with other necessary E&M activities.
Overall, there is a bigger emphasis on public comment, more timely and transparent decision-making, fewer committees, more experts vetting, more robust vetting process. Measure reviews will take place within 6 months.
Structure:
- There were
14 E&M Committees under NQF, now will be 5, which follow the patient’s
journey: 1) Primary Prevention; 2) Initial Recognition and
Management; 3) Management of Acute Events, Chronic Disease, Surgery,
Behavioral Health; 4) End-of-Life Care, Rescue, Specialized Interventions;
5) Cost and Efficiency.
- There will
be 5 projects per cycle (9-11 measures per cycle, two cycles per year), each having a committee that evaluates, discusses, and
assigns ratings for measures under endorsement review - PQM will
bring in additional voices on an ad hoc basis. - Each
E&M Project committee will have about 60 members - Each
E&M Project Committee will be divided into:
- Advisory
Group: reviews and provide ratings and written recommendations on
measures prior to the Recommendations Group endorsement meeting.
Advisory Group members are encouraged to attend Recommendations
Group endorsement meeting to listen to the Recommendations Group
discussions and to revote on measures during the meeting. - Recommendations
Group: reviews and provides ratings and written recommendations on
measures prior to the Recommendations Group endorsement meeting. Areas of
disagreement (i.e., lack of consensus) identified from the initial
measure ratings from both groups will inform the Recommendations Group
discussions during the subsequent Endorsement Committee Review
meeting. Recommendations Group members will also revote on measures
during the meeting. - Essentially,
both groups provide recommendations and any differences are brought
forward at endorsement meeting. In the end, the entire group will
vote on an endorsement decision, but the clinical vetting would have
already taken place
-
This discussion was modified 1 year, 9 months ago by
Cathey Wise.
eur02.safelinks.protection.outlook.com
Home | Partnership for Quality Measurement
Home | Partnership for Quality Measurement
- The PQM
-
2 Replies
-
- Every year, the PQM will randomly assign E&M Project Committee members to the Advisory Group or Recommendations Group.
- Appointments will generally be for 3 years, but for the very first term (Fall 2023 Cycle), they will stagger terms- some will get 1, 2, or 2 year terms, but after that everyone will be appointed for 3 years.
- For E&M Committee, only individuals can submit a nomination, not organizations. For PRMR (formerly MAP process) and MSR (measure removal process), organizational nominations will be permitted.
- To nominate, you must be PQM member, but membership is free and encouraged (see: https://www.p4qm.org/)
- Here’s a visual of the E&M Committee process:
- There will also be opportunities for ad hoc, off-cycle reviews (e.g., if there’s an update to a measure or something that draws attention to unintended consequences)
- PQM is currently completing the two active cycles, Fall 2022 and Spring 2023, using the NQF process and committee structure. The next cycle, which will adhere to the new process, is Fall 2023.
- As noted above, the PQM announced its first Call for Nominations for PQM Committees today (deadline is July 30). Once selected, rosters will be open for public comment.
Here is a more specific overview of the E&M Process:
- Intent to submit measure: available in October 1st and April 1st: submit basic measure info to prepare PQM for proper review
- Full measure submission a month later: Nov 1 and May 1. Submit through PQM STAR system.
- Internal measure review/staff assessment. Bring forward issues that standing committees should be aware of. 5 domains: importance, feasibility, scientific acceptability (reliability and validity), equity, and use/usability. This process takes 4-5 weeks. Assessment shared with developers and stewards for factual review before they go to committees.
- Public Comment Period: 30 days, starts once measure is submitted. Full record of public comments will be available on PQM website. PQM will also provide a summary of these comments
- Endorsement Committee Review: independent review of measures. Will look at staff preliminary recommendations and PQM measure evaluation Rubric to see if consensus is lacking anywhere. This will take 3 weeks.
- E&M Committee Review and Endorsement: E/M Committee meets to review measures that lack consensus, based on aggregated independent committee-member reviews. Vote is only on the endorsement decision, not all the domains used to assess measure. Just non-consensus.
- How do they determine consensus? Consensus is determined to be 75% or higher agreement among members (presented evidence to support this level). Under NQF it was greater than 66%
- This takes place in Jan/Feb (Fall) or July/August (Spring)
- Decision Outcomes. Endorsed, endorsed with conditions, not endorsed or endorsement removed.
- Appeals Period:
- 3 weeks.
- Just submitting an appeal doesn’t mean an appeals committee will be convened. There are criteria that must be met (reviewed by E&M Team). If eligible, ad hoc committee is convened (committees consists of all co-chairs from across the E&M Committees plus Battelle staff and any subject experts needed). Can submit an appeal for measure that was endorsed or not endorsed (under NQF was just the former).
i. If measure was endorsed, appeal must cite evidence that the appellant’s interests are directly and materially affected by the measure and that the CBE’s endorsement of measure has had, or will have, an adverse effect on those interests; AND cite existence of a CBE procedural error or information that was available by Intent ot Submit deadline but was not considered by E&M Committee.
ii. If measure was not endorsed, appeal must be based on 1) the CBE’s measure evaluation criteria were not applied appropriately; or 2) CBE’s E&M process was not followed. Appellant must specify how these items were not fulfilled
- Feb/March (Fall) and Aug/Sept (Spring)
- Importantly, PQM noted that there will be no changes to the measure endorsement criteria at this time, just the forms themselves.
Q/A
- Will PQM preserve the current NQF Scientific Methods Committee. The PQM has done outreach to NQF Scientific Methods Committee and will try to maintain as many of them as possible.
- How to identify ad hoc experts? May look to other existing committees and then to PQM members.
- Measure developers role in process: developers will be aware of any public comments (publicly posted) and committee concerns. Developers have opportunity to do a factual review of the staff assessments before they go to committee for independent review (formerly, “preliminary analyses”). After they got to committee and if there’s not consensus, developers will have access to that information (and they can attend meeting and provide overview of the measure). Developers (and any stakeholder) can also submit an appeal. PQM will also be working with developers on “readiness” for each cycle and if it’s not ready for current cycle, how to ready it for next cycle (particularly for first Fall 2023 cycle). Battelle will reach out to developers who have measures up for maintenance in Fall 2023 (as well as new developers). There will also be a webinar in the fall regarding the submission of measures.
- PQM fully recognizes that they will need to pull in additional expertise in addition to the members of the broader 5 E&M Committees. They would do this after the developer’s intent to submit deadline.
eur02.safelinks.protection.outlook.com
Home | Partnership for Quality Measurement
Home | Partnership for Quality Measurement
-
Hey Quality Measures Subcommittee!
How would you like to review and recommend comments to the E&M Guidebook?
Perhaps review and add comments here and then a possible meeting to confer?
Log in to reply.