Are the measures used to assess GP clinical performance valid?

GPFUQ #207 How do you assess the validity of a clinical performance measure?

In "Time Out — Charting a Path for Improving Performance Measurement" http://www.nejm.org/doi/full/10.1056/NEJMp1802595, 

Catherine H. MacLean 

and Amir Qaseem assessed and reviewed the appropriateness of measures used to assess a doctors performance. They found that only about a third of measures met  the  

criteria developed by the Performance Measurement Committee (PMC) of the American College of Physicians (ACP) to assess the validity of performance measures.  For each measure, the committee rated validity with respect to five domains: importance, appropriateness, clinical evidence, specifications, and feasibility and applicability.

DOMAIN 1. IMPORTANCE

·       Meaningful clinical impact: Implementation of the measure will lead to a measurable and meaningful improvement in clinical outcomes.

·       High impact: Measure addresses a clinical condition that is high-impact (e.g., high prevalence, high morbidity or mortality, high severity of illness, and major patient or societal consequences).

·       Performance gap: Current performance does not meet best practices, and there is opportunity for improvement.

DOMAIN 2. APPROPRIATE CARE

·       Overuse: Measure will promote stopping use of a test or treatment in general population or individuals where the potential harms outweigh the potential benefits.

·       Underuse: Measure will encourage use of a test or treatment in general population or individuals in whom the potential benefits outweigh the potential harms.

·       Time interval: Time interval to measure the intervention is evidence-based.

DOMAIN 3. CLINICAL EVIDENCE BASE

·       Source: Evidence forming the basis of the measure is clearly defined with appropriate references.

·       Evidence: Evidence is high-quality, high-quantity, and consistent and represents current clinical knowledge.

DOMAIN 4. MEASURE SPECIFICATIONS

·       Clarity — numerator and denominator clearly defined: • For process measures, numerator includes a specific action that will benefit the patient, and denominator includes well-specified exclusions. For outcome measures, numerators detail an outcome that is meaningful to the patient and under the influence of medical care. Denominator includes well-specified and clinically appropriate exceptions to eligibility for the measure.

·       Clarity — all components necessary to implement measure clearly defined

·       Validity: The measure is correctly assessing what it is designed to measure, adequately distinguishing good and poor quality.

·       Reliability: Measurement is repeatable and precise, including when data are extracted by different people.

·       Risk adjustment: Risk adjustment is adequately specified for outcome measures.

DOMAIN 5. MEASURE FEASIBILITY AND APPLICABILITY

·       Attribution: Level of attribution specified in the measure is appropriate (measure ties the outcomes to the appropriate unit of analysis) and is clearly stated.

·       Physician’s control: Performance measure addresses an intervention that is under the influence of the physician being assessed.

·       Usability: Results of the measure provide information that will help the physician to improve care.

·       Burden: Data collection is feasible and burden is acceptable (low, moderate, or high)