Friday, May 08, 2015

What Happens to Doctors Who Fail Their Maintenance of Certification Examination?

What is it like for physicians to fail their ABMS Maintenance of Certification® (MOC) program examination? How does the largest member board of the ABMS, the American Board of Internal Medicine (ABIM), respond to doctors who fail their secure examination?

As I continue to confidentially collect information from physicians who have failed their MOC examination, I thought it would be important to publish an example of a physician letter I received. The psychological, social, and financial consequences of failure of Maintenance of Certification are real yet have never been studied by the ABMS and their member boards, including the ABIM. To me, this is both highly unethical and inexcusable. This lack of concern for the negative ramifications of the high-stakes ABMS MOC examination that is increasingly tied to a physician's ability to practice medicine is alarming, especially when judgment is rendered by an unaccountable organization led by non-practicing physicians and scientists that benefit so handsomely from this program.  It is also very concerning in light of the high suicide rate among physicians.

I have also included an example of how the ABIM finally responded. The lack of timely response, transparency, and obfuscation of methodology and facts reflects very poorly on the ethics, scientific credibility, and legitimacy of the MOC program.

(While some physicians have granted me permission to use their name, I have elected to redact their personal information for this post.)

My name is *******. I practice Internal Medicine in ******. I've been in private practice for 10 years.

I was devastated when I opened an e-mail from ABIM on Monday June 30, 2014. I knew it was the strangest, most difficult test I had ever taken, but I had no idea that I would be failed. I wanted to curl up in a ball and cry, but I knew I had to be strong for all the patients that needed me that busy Monday. I was in a state of functional depression for weeks. Crying whenever I was alone. The thought of studying all over again for a crapshoot exam was more than I could stand. Fortunately, I connected on Sermo and learned what was going on and that I was not alone.

I wrote to ABIM 4 different times, once a week for 4 weeks. I want clear answers about the delivery of the test questions and scoring. I have not yet received a reply.

I spent many, many, lost, precious hours away from my husband, children and grandchildren on weekends and evenings studying for my exam. I started studying hard several months prior to the exam.

Cost: Loss of time with my family - Priceless, MKSAP books, ?$650, Test $750, MOC module $1600, Loss of revenue $ 3000+.

Yes, I'm willing to co-author and fight for this cause, and yes my name can be used.

I announced my failure at the quarterly Int Med meeting a few weeks ago at ****** Medical Center where I admit. I'm lucky because they are going to grandfather me in, so I won't lose my privileges.

This is lame I know, but it can't bring myself to tell my parents.

In August, 2014, the physician above finally received this response from Richard Baron, MD, the President and CEO of the ABIM:

Dear Dr. *******:

Thank you for your e-mails to the American Board of Internal Medicine (ABIM). I apologize for our delayed response. I understand your disappointment in learning that you were unsuccessful on the Spring 2014 Internal Medicine Maintenance of Certification (MOC) examination. I appreciate the time you dedicated to prepare for the exam and that you were not expecting this result. I understand your concerns and I’d like to respond to your questions about the examination and how it is scored. I’d also like to let you know of a recent policy change we have made for physicians who, like yourself, were unsuccessful in their first exam attempt.

With regard to your inquiry about the specific questions each examinee sees, though ABIM uses multiple versions of the exam, we take steps to ensure all takers are on a “level playing field.” We do this by constructing equivalent versions of the exam in content and level of difficulty so that, regardless of the version taken, examinees faced the same challenge. In addition, scores for all examinees were converted to a standardized score in the scoring process. The scoring process, along with the use of equivalent examination versions, ensures the comparability of scores regardless of the examination version taken. We follow standard testing industry best practice throughout our processes.

Concerning your question about whether ABIM exams are scored on a curve, the answer is that they are not: ABIM uses an absolute standard – a specific level of performance one must achieve in order to pass our examinations. Rather than scoring on a curve where a fixed percent of physicians will pass and fail, the absolute standard is a more fair and equitable process in that each test taker must meet a threshold of performance that makes their performance totally independent of other test takers. Because of this, the pass rates for an exam can and do fluctuate naturally.

About your question related to pass rates dropping, the differences in the percent of test takers who pass the exam (pass rate) from one administration to another is not due to changes in the exam content or difficulty but to the natural fluctuation that occurs with using a consistent absolute standard with different groups of exam takers who may have different motivation, ability and training.

ABIM has made a recent policy change that I hope will be good news for you– because your Internal Medicine certificate expires this year and you were not successful in this exam attempt, ABIM will grant you an extra year to pass your exam, providing you complete the Self-Evaluation of Practice Assessment requirement by December 31, 2014. During the extra year, you will continue to be reported as “Certified, Meeting MOC Requirements,” provided all your other MOC requirements are being met. You will need to pass your exam by December 31, 2015 and once passed, your next MOC exam will be due 10 years from the last pass.

We hope that you find this information helpful. Again, I completely understand your disappointment and wish you success in your next attempt. If you need further assistance, you may reply to this e-mail or call us at 1-(800)-441-ABIM (2246) Monday through Friday, 8:30 a.m. to 8:00 p.m., and Saturday, 9:00 a.m. to 12:00 p.m. EST.


Richard J. Baron, MD, MACP
President and CEO
American Board of Internal Medicine

This letter is important for several reasons.

First, there is no excuse why the ABIM should have had such a "delayed response" to any physician diplomat's inquiry, given the high cost of their "program" and the millions of dollars spent to support this program.

Secondly, we now find doctors who sit for the MOC examination are not all given the same examination, but rather different ones deemed to be "on a level playing field."  How is "equivalency" between examinations determined exactly?  Wouldn't we all like to know?  Might there be bias created by such a process?  And what are "standard testing industry best practices?"  Shouldn't doctors be enlightened on these?

Thirdly, there's the issue of determining an "absolute standard" for physician knowledge.  Who decides this "absolute standard?"  How can an "absolute standard" be determined for a physician when a central authority is completely blind to a physician's scope of practice? Is there some "industry best practice" that doctors don't know about?  Is the "absolute standard" of a quality physician only someone who can cram facts and "pass" a computerized test based on an arbitrarily-determined knowledge base?

The ABIM leadership are masters at skirting these very real concerns of the MOC program.

Finally, the doctor's question regarding the rising failure rates on MOC examinations was not acknowledged nor  answered by Dr. Baron.  Rather, Dr. Baron blames the rising failure rates on experienced physicians' "different motivation, ability, and training." That's right: it seems the higher failure rates are the doctors' fault.

Not surprising, though.

The American Board of Medical Specialties' MOC® program, which has been exhaustively marketed and sold to physicians and legislators as a valid mechanism for assuring quality physicians, appears hopelessly mired in scientific uncertainty and conflicts of interest.  The process is also having profound emotional and psychological consequences to those adversely affected.  This is not a training or teaching exercise, but rather a punitive assessment technique based on an undisclosed set of centrally-predicated and non-transparent benchmarks that appear unevenly distributed.  And sadly, our own professional societies continue to support the program primarily because they profit handsomely from it.

As the veil is lifted on this Draconian process, it is becoming increasingly clear that the ABMS's MOC program is deeply flawed. Yet for reasons that only its well-paid creators understand, the program is no longer a voluntary exercise for physicians, but increasingly tied to government physician payment formulas and credentialing.   

I have no doubt that the members of the ABMS and ABIM feel in their heart of hearts that such a centralized, top-down secretive system of physician testing carries nothing but the most enduring benefit to society.  McCarthyism was hatched from a similar line of reasoning. 

It is clear that the inconsistencies and self-serving nature of the ABMS MOC program is becoming increasingly destructive to our colleagues and profession.  Now we must ask ourselves, what should working physicians do to fix this mess?

At the present time, it seems the only viable alternative is mass non-compliance with the ABIM MOC program or legal action. For not only is the MOC program scientifically indefensible, it is now clear that it is potentially very destructive to many physicians.


PS: Physicians who failed a MOC examination are still encouraged to confidentially submit their stories as I continue my efforts to expose all sides of the Maintenance of Certification debate.


W. Bond said...

Excellent piece, as always.

One nitpick: In the antepenultimate paragraph I don't think you're being fair to Sen. McCarthy.

Anonymous said...

Karma's a bitch - its just a matter of time for Richard and Nora to reap what they sowed.

x said...

as long as nbpas accreditation is not apprved by a few hospitals, abim will continue taking things for granted. the real problem is that AHA(american hospital association) which has enormous power in hospital matters - including making bylaws/ and physician credentialing process - is pretending to be a dead spectator. the lack of interest of govt in not helping communities in distress which are political soft targets (eg physicains) is not a hidden secret.. ABIM and ABMS are just the faces of a bigger problem, who are pinching us bitterly now... there seems to be a silent approval from govt/big hospitals to silence vocal physicians so that hospitals and govt can change medicine the way they want..!! more more of recognized physicians need to take their hospital boards to task in enforce a change.. as long the hospital administrators (AHA - baord members) dont realise that medicine is not run by their whims and fancies, but the hard work of physicians/nurses and their skills are irreplaceable we will be harassed.. and we deserve to be harassed... people who have no moral standing eg dr baron and dr humanun chaudhary are available at dirt cheap prices.. if these two go..more can be easily bought to continue the destruction of medical field..

Anonymous said...

After 20 years of fellowship in the American College of Physicians, I have decided not to renew my 2015-2016 membership this June in protest against MOC.

ACP will not care, but I will have the satisfaction of not giving them my money.

An orthopedic colleague of mine is similarly disgusted with their specialty board MOC, and has decided to withdraw from his orthopedic academy in protest.

If enough physicians withdraw from their specialty societies in protest to MOC, the external pressure on the ABMS will increase and hasten their demise.

jimbo26 said...

With regard to your inquiry about the specific questions each examinee sees, though ABIM uses multiple versions of the exam, we take steps to ensure all takers are on a “level playing field.” We do this by constructing equivalent versions of the exam in content and level of difficulty so that, regardless of the version taken, examinees faced the same challenge. In addition, scores for all examinees were converted to a standardized score in the scoring process. The scoring process, along with the use of equivalent examination versions, ensures the comparability of scores regardless of the examination version taken. We follow standard testing industry best practice throughout our processes.
Sounds like they use the wrong answer paper to the question paper , or they used the wrong conversion tables .

Unknown said...

Join the AAPS , American Assoc Physicians and Surgeons. They are the only group really fighting for us by sueing the ABMS . We now need to go after the individual "boards" for harming us but the AAPS needs more membership support. Put your society dues where it counts . No other society has fought back and this is what we need, court action.

W. Bond said...

To the anonymous FACP who is resigning from ACP over MOC/ABIMscandal: please let their leadership know by e-mail your intent. ACP needs to know that it cannot afford to remain "neutral" and avoid making a statement on this ABIM scandal.

x said...

dr walter, ACP is part of the scandal. it has direct conflict of interest. it is benefiting by selling the certification and re-certification education material. the more doctors fail, the more they are enforced to by ACP mandated educational material. even in the last acp meet at boston, dr baron has personally comented that ACP has conflict of interest in MOC as MOC has been designed in consultation with ACP..this explains their silence.. only the physicains boycotting/ leaving ACP enmasse will shake it.. not one person leaving it.. i understand we have compulsion of staying with abim as it is linked to licensure (for now) but i m surprised why there is no movement to leave/abandon ACP. even shaking one stone sometime can bring fall to an arrogant mountain.

W. Bond said...

I understand that argument, but still think you should let them know why you're ending your membership/fellowship. If they have x cancellations that all come with the same message, they won't need to do a study as to why
"the economy," etc. is causing a drop in membership. I think that the ACP has much to gain/lose by their response to this ABIM scandal. Sadly, they're choosing to condone ABIM behavior by remaining largely silent.

Anonymous said...

When I recertified in cardiology, I did not go to a course nor did I review any material. Since I keep current, I decided that if I could not pass the test it is not relevant to the practice of cardiology. Passing the cardiology general board was a breeze.

However, I also do bread and butter interventional cardiology. In a rural area of the state, I place perform PCI and feel very comfortable with AMIs.

I DO NOT complete complex procedures nor do I perform carotid stenting, PPMs, ASD closures, mitral valve clips, TAVR, alcohol septal ablation, septal punctures, LAA closure, etc. etc. Consistent with the recommendations from the ACC, I believe that these procedures should be concentrated in the hands of those who perform many to improve outcomes. Would any patient want to be the one ASD closure that I perform that year? I would not want my family member to go to a low volume operator. I perform around 150 interventions/year and send out the other patients to interventional cardiologists with more experience.

As a result, I am not out to date with all the details of the procedures and was subsequently punished on the interventional cardiology boards which I failed.

Leave it to the ABIM to punish me for my own good virtues.

Anonymous said...

The post above shows how bogus the ABIM's perception of "quality" is. This cardiologist is saving lives in a rural area by doing interventions in which the cardiologist is competent, improving quality of care in the rural area.

Anonymous said...

I also failed the MOC recertification in Spring 2014, and I refused to take it since, as the test had nothing to do with the practice of Internal medicine, it was clear how detached ABIM is from reality.

I spent many hours every night, after work, exhausted, reviewing materials for the exam, missed many hours to spend with my two little girls, and my wife, who resented me for a long time, as I didn't make more effort to spend with the family!! When I only had 3 hours after work, before I collapse sleeping.

The cost of purchasing the MKSAP, and Med study, and the fees for the exam. was over $ 2500

The emotional and psychological stress prior and after the test, and the incredible trauma seeing how the exam was not related to our practice..... and I thought I was well prepared. And of course the shock to receive the failure result.

That is still not the issue, the issue is that if I don't pass this exam, I am out of my job!!! When did it happen, that after becoming a board certified, that they take your certification away from you, if you don't pass a flawed test??

I am currently working overseas in a very prestigious job, and my certificate expires at the end of 2015, and the hospital asked me to provide them with an updated certificate, or I am out!!!!

Does the ABIM realise what it is doing to us, how it has affected our lives in so many levels? My job security, my time and relation with my family, and not to mention the financial cost.

I am a very popular doctor here, and have over 2500 patients in my practice, which reflects the level of trust they have put in me. An unrealistic exam, is going to literally put me out of business. What does ABIM think about that??

Anonymous said...

ABIM is just profiting heavily from unnecessary,impractical, and expensive tests. We should always remember the tests are not a reflection of how we care for patients. We should remember they are just a buisness - a buisness to make a lot of money on tests that do not reflect real medicine