Examining Maintenance of Certification Failure Rates

The American Board of Medical Specialties' (ABMS)  Maintenance of Certification® (MOC) program is marketed as "an ongoing process of education and assessment for certified physicians to improve practice performance."  After reviewing the recent failure rates of this program over the past five years, I worry the process is not about assuring physician quality, patient outcomes, or practice improvement, but rather as a means to ration the number of doctors eligible to earn Center for Medicare and Medicaid Services (CMS) payment incentives.

Background

As I've previously reviewed, the ABMS MOC program and the physician registry it creates has been written into our new health care law. To date, the MOC program serves as the sole measure of physician "quality" for the upcoming CMS value-based physician payment model that is to replace the current fee-for-service model in 2015.  To create an incentive for physicians to participate in the MOC program, CMS offered a 0.5% payment incentive to the Physician Quality Reporting System for physicians participating in the MOC program in 2014.  While this does not sound like much money, if we consider that physicians were paid $77 billion from CMS in 2012, this 0.5% represents approximately  $385 million dollars paid to doctors (or their employers).  Imagine the cost savings to the government if physicians were not eligible for such a payment.

Methods

I decided to evaluate the failures of "first time" MOC certification for all board certifications issued by the American Board of Internal Medicine from the pass rate data published online by the ABIM.  The number of failing physicians was calculated by subtracting the total number of doctors taking each examination from the number of doctors passing the test to arrive at the number who failed.  Non-integer values were rounded.  Next, I added up the total number of doctors who took the various ABIM MOC examinations each year and the total number who failed each year to generate an annual MOC percentage failure rate.  I calculated these values for 2009, 2010, 2011, 2012, and 2013.  I then applied a linear regression line comparing the total test takers and the total number of doctors failing the examination by year.  I then calculated the number of "certified" physicians each year as a "difference" of the total and failed physicians each year and applied a 2-period moving average trend line to these values.

Results

Here are the raw data assembled in a chart for your review: 

Year	2009	2010	2011	2012	2013
Total Test Takers (n)	8744	9574	10889	11524	12201
Number Failed (n)	861	1030	1357	1610	2138
Percent Failed	9.85%	10.76%	12.86%	13.97%	17.52%
Difference (n)	7883	8544	9532	9914	10063

Plotting these data shows the following trends:

MOC Failure Rate Trends (click to enlarge)

Discussion 

While the number of first-time MOC test takers grew each year studied, the failure rate also grew significantly. Is this because physicians were significantly less intelligent in 2013 than 2009?  Does this mean that board review courses run by each of our professional specialty societies are less relevant now than they were  despite their growing price?  Or might such a failure rate really be a way to "bend the cost curve" for health care delivery by covertly rationing the monies CMS pays physicians?  Each of these are fair questions that need to be answered honestly by CMS, the ABMS, the ABIM, and our professional specialty societies that collude with the ABMS as they run their various MOC board review courses. 

It goes without saying that test scoring methods and the raw responses of questions performed as part of the MOC process are shrouded in secrecy and serve no retrospective educational learning opportunity for doctors taking these examinations.  Doctors who take the MOC testing must also sign a statement that they understand that  divulging content in the examination will be met with harsh penalties including, but not limited to, possible revocation of hospital privileges or reporting to state medical licensing boards.  In return for this promise of secrecy, the ABMS and ABIM appears to operate in an environment that violates the trust of the public and those they test.  They do not explain their consistently higher failure rates seen year over year.  They do not mention the relationship they have to physician payments from government sources when physicians enroll in their MOC program.  Instead, they describe their process as "voluntary."  They espouse  the ethic of  "the need for public accountability and transparency," yet deliver none of these things themselves.  As such, it is clear that physician quality assurance or practice improvement is not the ABMS or ABIM's real mission for public good.

Rather, it appears from the MOC program failure trends above that the real reason for the ABMS MOC program is not only for self-enrichment, but to provide government cost savings without regard to the professional consequences to the many physicians they test and the patients ultimately affected by the loss of eligible care providers from their insurance panels. 


-Wes