I first submitted a manuscript with these data to the New England Journal of Medicine and it was rejected immediately before the manuscript even reached the peer review process. (I even gave the editor access to a password protected webpage containing the infographic you will see below). They weren't interested. (Perhaps because this is a retrospective review?)
I then submitted a slightly revised manuscript to JAMA. The after a preliminary review, the editor of JAMA would not submit my manuscript for peer review, but thought the work might warrant review as a 600-word "Research Letter to the Editor" that allowed only one figure and one table and no more than 6 references. Needless to say, nearly a month later, I just learned that my "research letter" was rejected by both JAMA and JAMA Internal Medicine. The reviewers comments were helpful should I ever decide to re-submit these data to a peer-reviewed journal again in a different format. However, given the many changes occuring to the ABMS MOC program recently, I felt time is of the essence and elected to proceed with publishing my data here on this blog. I feel these data are too important to sequester behind a medical journal paywall and encourage physicians to use these data to question the credibility and reproducibility of the American Board of Medical Specialties' (ABMS) MOC secure examination and the remainder of the MOC program as a requirement for maintaining hospital practice credentials. By publishing my data on this blog, I can publish ALL of the data I collected and collated, not just a tiny fraction of it. Also, I have found that this blog's reach is equal to or larger than many medical journals, particularly when a post contains important and credible information.
METHODS: The ABIM's first-time MOC pass rate data were gathered from current and prior archived ABIM web pages for each subspecialty between 2000 and 2014 using the Internet Archive Wayback Machine (https://archive.org/index.php). (Only the past 5-years of pass rate data were available for Hospital Medicine and 10-years of pass rate data were available for Interventional Cardiology because they were "newer" internal medicine subspecialties). The number of physicians failing their examination each year was then calculated by subtracting the product of the total number of physicians who took the examination by the published pass rate from the total number of physicians who took each subspecialty MOC examination. Linear regression trend lines of annual pass rates with their correlation coefficients over time were calculated for each subspecialty. Historical volatility of pass rates was calculated as the standard deviation of the data range of year-to-year percent change of pass rate. For example, published pass rates for General Internal Medicine from 2000-2014 were 89%, 92%, 91%, 85%, 86%, 84%, 79%, 83%, 92%, 90%, 88%, 87%, 84%, 78% and 80% respectively. The percent pass rate change from 2000 to 2001 was calculated as 0.92/0.89 - 1 = 3.37%. The 14-value dataset of percent pass rate annual changes for the years 2000-2014 therefore was 3.37%, -1.09%, -6.59%, 1.18%, -2.33%, -5.95%, 5.06%, 10.84%, -2.17%, 2.22%, -1.14%, -3.45%, -7.14%, 2.56%. The historical volatility of pass rate percent changes for General Internal Medicine equaled the standard deviation of this dataset, or 4.93.
RESULTS: First-time ABIM MOC pass rate trends (and the raw data for each internal medicine subspecialty's first time MOC pass rate) with their linear regression trend lines can be displayed using the interactive infographic below (just click the specialty circle to display the detailed annual pass/fail data):
The total number of physicians who took the test over 15 years and the percentage of physicians who failed their MOC exam on the first try are shown by subspecialty in the table below. Year-to-year historical volatility of pass rates by subspecialty also shown (bolded historical volatility values exceed one standard deviation from the mean of all subspecialty volatilities):
|Subspecialty||Physicians Undergoing MOC Examination (n)||Number of Physicians that failed MOC exam on their first attempt (%)||Pass Rate Historical Volatility (Minimum/Maximum annual percent pass rate change)|
|General Medicine||61,050||9,212 (15.3%)||4.93 (-7.14 / 10.84)|
|Cardiology||10,486||1,386 (13.2%)||3.23 (-3.53 / 6.10)|
|Cardiac Electrophysiology||1,398||113 (8.1%)||4.33 (-7.45 / 5.49)|
|Critical Care||5,596||552 (9.9%)||4.62 (-7.69 / 8.33)|
|Endocrine||2,308||310 (13.5%)||7.48 (-10.4 / 13.16)|
|Gastroenterology||6,255||770 (12.3%)||4.35 (-6.45 /4.71)|
|Geriatrics||6,559||539 (8.2%)||7.64 (-14.13 / 20.51)|
|Hematology||2,427||338 (13.9%)||7.11 (-9.76 / 13.51)|
|Hospital Medicine*||829||113 (13.6%)||1.09 (-2.27 / 0.00)|
|infectious Disease||3,520||312 (8.9%)||5.00 (-10.53 / 7.32)|
|Interventional Cardiology||3,182||244 (7.7%)||2.24 (-5.32 / 2.33)|
|Nephrology||4,129||466 (11.3%)||3.81 (-8.42 / 5.56)|
|Oncology||4,568||456 (10.0%)||3.87 (-8.79 / 6.90)|
|Pulmonary||5,792||803 (13.9%)||6.57 (-11.24 /13.92)|
|Rheumatology||2,143||217 (10.1%)||3.56 (-6.59 / 5.81)|
- A very significant 15,832 physicians (13.2%) have failed their MOC secure examination on the first try. This has a significant impact on physician morale without justification and has affected patient access to their physician as a result.
- A large variation in year-to-year pass rates exists for many subspecialties, particularly Endocrinology, Geriatrics, Hematology, and Pulmonary suggesting inconsistent content, irrelevant content, and/or inconsistent setting of pass rate cut-offs year to year using the ABIM's modified Angoff method of determining pass rate cut-offs.
- First-time MOC failure rates vary by as much as 51% between subspecialties, with General Internal Medicine having the highest failure rate (15.3%) and Interventional Cardiology having the lowest (7.7%)
- Fourteen of 15 subspecialties had declining pass rate trends in this 15-year review of ABIM MOC pass rates (see infographic). Was this because the tested material is increasingly irrelevant to patient care? Or might there be another ulterior (financial?) motive for the decline? Or are most physicians simply unable to cram larger and larger amounts of information into their heads and regurgitate the proper answer in a limited time period thanks to the exponential growth of health care information over the last 15 years?
- Year-to-year first-time MOC pass rates could vary by as much as 20.5% (Geriatrics).
- The steepest pass rate decline was interventional cardiology (1.62% decline in pass rate per year), though low numbers of physicians took the exam the first several years it was offered.
- Since MOC participation is increasingly tied to hospital credentials as "board certification" became "time-limited" in 1990, what responsibility does the ABIM assume to patients when they fail a physician? How many physicians of each internal medicine subspecialty had to retake their examination (and how many times did each have to repeat)? What is the total cost to the doctor and the health care system for this unproven "quality" metric in terms of real dollars, patient access, and care delivery?
- Given the large number of physicians that have failed their MOC examination, why has the ABIM not studied the psychological, social, professional, and clinical impact their failure of physicians? Or don't they care?
A word of caution: the American Gastroenterological Association (AGA) has recently posted a video promising to bring an end to the MOC secure examination for gastroenterologists. Instead of ending the MOC program, however, (and after reviewing the full proposal about to be published in November) the AGA promises to replace MOC with an even more complicated "Continuous Professional Develppment" program "tailored to your needs" called "GAPP", the "Gastroenterologists Accountable Professionalism Pathway." This pathway promises to be an even more complicated 13-step program rather than MOC's 4-step program. It is a spin-off from the ABIM's similar "Continuous Professional Development" program used before by the ABIM that still has no proof of its value to patient care and without any disclosure of the cost this program for working physicians. It maintains the need to participate in their program for physicians to remain credentialed to practice medicine in their hospital. Before leaping for joy that the MOC examination is being phased out, realize that our specialty boards are reeling from the exposure of the financial reality of their "programs" to our health care programs and are creating new "programs" to assure ongoing financial and time-commitments from working physicians without proof that recertification is of any value to patient care over traditional self-directed Contining Medical Education. Worse yet: the AGA uses the term "Professionalism" in the title of their new re-certification proposal - a word defined via a "Task Force" of the corrupt ABIM Foundation.
Please feel free to share this post with interested journalists, friends, and colleagues. If you have further questions or ideas about these data, feel free to leave a comment or send me a note at wes - at - medtees dot com.