Rebuttal: A Bit More on MOC Failure Rates

Recently, David H. Johnson, MD, Chair of the American Board of Internal Medicine (ABIM) Board of Directors and Internal Medicine Chairman at UT Southwestern Medical Center in Dallas, and Rebecca Lipner, PhD, Senior Vice President of Evaluation, Research, and Development at the ABIM published a "guest blog post" over at MedPageToday regarding Maintenance of Certification (MOC) pass rates, entitled "Debunking the MOC Pass Rate Myths." I do not know if they were directing their comments directly to me, but since I was the one who brought the issue of rising MOC failure rates to the public's attention some time ago using the ABIM's own published data of first-time MOC takers, I thought I should respond.

I appreciate Drs. Johnson and Lipner providing more insight into how the MOC examinations are constructed and scored.  The ABIM's arbitrarily-defined "65%" marker based on "expert consensus" and undisclosed "psychometrics" is an interesting one, and at least now we understand how this benchmark was arbitrarily created. They also acknowledge that "significant changes in the practice of the discipline may require that the passing standard be reset."  Unfortunately, they do not disclose what those changes might be, which way their 'standard' might be adjusted, or what internal or external force might mandate such an "reset."

Drs. Johnson and Lipner also confirmed the rising failure rates of board examinations over the past five years.  Oddly, they then seem to imply that this "decline in pass rates" is due to a misunderstanding by "some social media observers" of their testing and scoring process:

"Recently, based on a slow decline in pass rates over the past 5 years for the American Board of Internal Medicine Maintenance of Certification exam, some social media observers have made erroneous assertions about how the exams are constructed and scored.

Without a full understanding of the exam process, they have nevertheless claimed that the exam is being made more difficult, that the standard for passing the exam changes every time an exam is given, or that the exam is 'graded on a curve.' These assertions are all incorrect."

Rather than attack what was said or what wasn't by those unruly social media types, I will merely say this: my assessment of the rising failure just focused on simple math.  Because there are higher numbers of doctors taking this examination year after year due to the American Board of Medical Specialties' recently-imposed MOC requirement, the stable "65%" pass rate means there is necessarily an INCREASE in the number of total physicians who are FAILING the examination.  It goes without saying that a requirement for repeated testing or adding more people to the testing pool handsomely profits the ABIM and similar subspecialty organizations that provide proprietary subspecialty MOC-preparation courses taught by ABIM test question contributors (this relationship is also conflicted, by the way). But the downside of more testing is multi-pronged: it also redirects physicians away from direct patient care to formal classrooms, expensive hotel meeting rooms in cities often outside the physician's hometown, and ultimately to expensive corporate test taking centers.  The ABIM never considers these high costs nor the negative effects absence from one's practice has on patients or a physician's limited family time.

Drs. Johnson and Lipner also remind us that "pass rates are higher among first-time test takers." While there are likely many reasons for this, we must conclude that significant AGE BIAS against older, more experienced physicians exists.  Since this proprietary MOC process was cleverly made a "quality measure" imbedded in our new health care law, the logic of this biased process contradicts common sense.  Are patients not our best teachers? Are we to assume that physician experience after years of direct patient care is of less value than ABIM-selected facts and principles culled from an exponentially-growing body of medical literature regurgitated on a computer screen in an expensive testing center?  Furthermore, how the test is constructed or scored is of little relevance to more procedural based specialties of internal medicine like mine because these skills are never assessed.

Drs. Johnson and Lipner also mention (without supporting data)  that physicians who take their examination three times (at their expense) ultimately achieve a 95% pass rate. As if this is the point. Instead, we should ask what patients and physicians lose each time they must repeat this onerous, time-consuming and unproven process.  Can the personal, financial and professional losses of repeated testing ever be regained? How, exactly, does repeated testing of a flawed process help our patients?  What marginal utility to our health care system does this whole test-taking exercise really provide?  Does this MOC process actually separate the good physician from the bad physician or the good test-taker from the bad one?  Since ABIM never knows a doctor's scope of practice before their assessment, they make some heady assumptions about what they should test.  In reality, the ABIM only tests what THEY think should be tested, rather than what matters to the particular physician's patient population.  Worse still, since the ABIM remains completely unaccountable to doctors or patients in regard to their MOC process, their assessment may have absolutely no relevance to a particular physician's practice.  Shouldn't we be discussing these issues before arguing how the test is constructed and scored?

No physician I know argues with the need for continuing, life-long education in medicine, especially when its performed in a relevant, transparent, informative, collegial and feedback-oriented process without a ulterior financial motive.  If fact, most of us do this every week as part of our state licensure requirements and have plenty of continuing medical education credits to prove it.  Allowing the ABIM to monopolize this process as we have through artificial MOC point acquisition exercises that are often irrelevant to a physician's scope of practice not only circumvents more healthy and sustainable learning environment, but also might cause more harm than good if that physician loses their practice privileges as a result. 

In summary, I believe the ABIM's lucrative MOC process is no more valid than what doctors have already been doing for years before the American Board of Medical Specialties decided to make their MOC program an annual exercise.  The MOC process remains unproven in its ability to improve patient care and (as we've now confirmed) discriminates against more senior physicians. Those of us who have endured this process multiple times have witnessed the growth of "board certification" from a self-imposed professional milestone to a money-making scheme filled with clinically-irrelevant busywork that often detracts, rather than augments, patient care (especially those non-validated "practice improvement modules").

It is time that doctors of all ages insist that these self-appointed non-profits organizations cease their attempt to become expensive continuing education providers and stick to what they do best: assessing a doctor's ability to reach a level of exceptionalism of their practice during their career, rather than pretending that they can assure some arbitrary level of ongoing practice adequacy for the business community.  Until the ABIM comes to grips with their increasingly bipolar agenda, their ability to remain credible and relevant to practicing physicians, the public, and the mysterious "stake holders" they claim to serve will continue to be challenged, both on social media and in the court of law, as "significant consequences from losing certification" occur with increasing frequency to experienced and fully-capable physicians.

-Wes