On Pediatricians' Golden Pond

Satellite view of the
American Board of Pediatrics' 2.8-acre Pond

Pediatricians in the United States can thank the American Board of Pediatrics (ABP) and their time-limited "Maintenance of Certification" (MOC) program for the purchase of their very own "Golden Pond."

Surely, US pediatricians need a 2.8 acre pond. Where else would they be able to sit and contemplate their past careers as academics as the sun sets slowly over the horizon? Everyone knows that when properly maintained, ponds provide a peaceful venue for a leisurely stroll or for the opportunity to toss a hook and bobber in search of a bluegill or large-mouth bass. No doubt these activities are critical pediatric health care quality and safety skills.  Thanks to the leadership and "medical professionalism" portrayed by the ABP leadership over the past 30 years or so, all US pediatricians can feel proud to know that every bit of their $1200 "continuous MOC" fee is going to the support of this worthy landmark.

Surely pediatricians worldwide know about the pond they purchased, don't they?

Perhaps not.

Ponds, you see, don't typically make the internet websites of the American Board of Medical Specialties member boards or IRS Form 990 tax forms (except as "land and buildings"). Few would think of lands and buildings as including a pond, but for the American Board of Pediatrics, it does. In fact, their pond (picture) rests on a consortium of properties adjacent to 111 Silver Cedar Ct, Chapel Hill, NC that span over 9.3 acres and comprises two separate buildings (201 Silver Cedar Ct is also owned by the ABP). Unfortunately, it's hard to imagine how a pond adds to the safety or quality of pediatric care in the United States. Instead, we can only surmise that a pond is an important asset to the American Board of Pediatrics because it improves their property value and provides important tax revenues and political influence to the state of North Carolina and beyond.

And who better to pay for that pond and annual taxes than unsuspecting US pediatricians increasingly forced to comply with Maintenance of Certification?

No one was supposed to know that the first recorded deed of these properties coincided in time with the creation of Maintenance of Certification, 1986, according to North Carolina real estate records. No one was supposed to know that as President and Advisor to the American Board of Pediatrics, James A. Stockman, III, MD, earned $1,307,415 in Fiscal year 2012 before his retirement package of another $2.4 million kicked in the next year, either.

And let's not forget the many cars that pediatric diplomates helped Dr. Stockman purchase during his tenure at the ABP.

Here are Dr. Stockman's own words, published in Pediatric Annals:

Dr. Shulman: If you had not chosen medicine, which field would you have chosen?

Dr. Stockman: I’d be a car salesman.

Dr. Shulman: I suppose that’s not a surprise. You are quite the collector of classic cars. Your wife once told me that during a 5-year period in Chicago you bought and sold 40 or 50 separate autos. And that was before eBay!

Dr. Stockman: Yes, my collection is very diverse. Among them are a ’56 Cadillac, a Sting Ray, a Triumph, and a twin turbo engine Porsche 911. I love driving them on all the country back roads here in North Carolina. There is life beyond academe!…

Life beyond academe indeed.

Especially when one considers that Maintenance of Certification (MOC) was Stockman's "most far-reaching  achievement" of his 20-year career with the American Board of Pediatrics (according to the well-funded Illinois-based American Academy of Pediatrics that has an extensive lobbying presence on Capitol Hill).

-Wes

Staying Prepared at Christmas

Good Samaritan cardiologists never know when they (or someone else) might need 81 mg of aspirin  this holiday season.  Here are some fancy cufflinks (seen in clinic today) that every cardiologist should consider:

-Wes

PS: I have no commercial interest in this gift idea. :)

Nickle and Dimed

The American Board of Anesthesia's heavily-promoted "MOCA Minute" had some not-too-flattering details exposed recently.

First, was this letter sent to a diplomate who had not purchased their MOC offering:

Second, was the cost and the requirement to renewed requirement for physicians to participate in previously-suspended Part IV of the ABMS MOC program:

Here's the math: $210 per anesthesia diplomate annually x 20,000 anesthesiology diplomates = $4.2 million EVERY YEAR.

That's $42 Million every 10 years.

That's $42 million to the ABA for a program that threatens to compromise an anesthesiologist's ability to practice.

That's $42 million to the ABA for a program that threatens an anesthesiologist's ability to earn revenues from insurance companies.

It doesn't matter if doctors pay one lump sum every 10 years (as internists can do), or break their fees into bits. Forcing them to pay to keep practicing is the problem. Such a program in the egalitarian United States will never fly. Why, then, do the American specialty boards continue to insist that physicians participate when there are simply no proof it adds value to patient care or safety?

Because we're letting them get away with this.

It's time for all practicing physicians to stop supporting the ABMS MOC program, especially when there's at least one better option.

-Wes

More Insights to ABIM Board Certification Tests and Scoring

From a recent early-career physician who failed their certification board examination and is trying to understand why:*

Dr. Fisher,

I contacted ABIM today asking many questions about the initial certification exam and the scoring. 

As I've seen you mention, they claim to use the (modified) Angoff Method to set the cut score for passing. It involves a bunch of "experts" reading questions and estimating what percentage of "minimally competent" test-takers would answer correctly. They then average all experts' percentages for each question, and average the percentage for all questions, to achieve the minimum passing score. Apparently. No info on who the experts are. Also no definition of what a "minimally competent" test taker is.

That's a weird system, but it wasn't what I found most egregious. Without going into too much detail, there are 240 questions on test. By my answer report, based on number of missed questions, I got 74% correct. They have interviews with their president and examples online stating you need to get about 65% correct to pass. I blew that away and still failed. So, I called them. To start, they told me that at least 35 questions are "test questions" that end up being thrown out and neither count for or against you. So we are paying to be research subjects. And paying handsomely. Even if you throw out 35 questions, I still got 70% correct, so I asked how I could've failed if I was still well above the 65% threshold. They told me that not all tests are equivalent and essentially I may have had an easier version and needed to get more correct. So apparently someone could've missed more than I did but still passed if their test was arbitrarily deemed more difficult? 

I asked many questions about how this could even be considered an equitable way to grade a high stakes test, but got little response. They did offer to have a psychometric statistician call me. In 7-14 days.

Thanks for your time and for reading. What an arbitrary process. Also, they will not let me see my test or answers, nor can they show me what questions were thrown out. This being clouded in such secrecy just adds to the mistrust from physicians. How can we trust a test that's so arbitrary? Also, if I continue to fail (or am failed, depending on how you look at it), is the ABIM going to pay off my student loans? I've proven competence in residency and now fellowship training. I would argue I've proven competence on this very test too.

A few notes about this physician's observations.

ABIM has published several "abstracts" on their webpage about methods of psychometric testing. The first abstract, published in February 2011 an co-authored with a representative from the computerized testing firm PearsonVue, discusses "transitioning the board from linear computer-based test to an adaptive, multistage testlet-based examination" and their "experience 'selling' this change to leadership." In other words, it appears the method of scoring ABIM examinations made a change before February 2011 to IRT scoring. Secondly, it appears a member of PearsonVue and an ABIM non-physician helped "convince" the ABIM leadership why they should change methods. (On a side note for those interested, here's a not-so-"simple guide" to Item Response Theory.) The second abstract, published in April 2011, discusses a "transition from classical test theory (CTT) to item response theory (IRT) scoring." What prompted the change in scoring technique is uncertain, but it appears the method for scoring examinations for ABIM board examinations changed before early 2011. Importantly with this new technique, the pass rate cut-off appears to be determined AFTER the test is taken and may vary from individual to individual, as the doctor suggests in the email above.

For those interested, the 2005-2015 pass rates for initial ABIM board certification were as follows: 92% (2005), 91% (2006), 94% (2007), 91% (2008), 88% (2009), 87% (2010), 84% (2011), 85% (2012), 86% (2013), 87% (2014), 89% (2015). (source: the Internet archive and ABIM websites).

Was this change in examination methods responsible for the declining pass rates of physicians taking the ABIM certification examination noticed by program directors across the country in 2013? Or was this the way PearsonVue required the test be scored if PearsonVue's computerized test centers from  were used for scoring (making it a "win-win" for both organizations)? We are left to wonder. The  falling pass rate hypotheses considered even included the possibility that study methods of millennial physicians were less rigorous.  The New England Journal of Medicine's summary of the controversy tried to quell the outcry from millennials who were quick to respond. Even the American Board of Family Medicine's leadership felt compelled to explain their falling pass rates about the same time.

Isn't it interesting that no one ever entertained the possibility that a change in scoring method at the ABIM had occurred that might have caused the drop in pass rates? If true, the process of passing a physician's initial board certification is inconsistent between test takers since tests contain questions that are "weighted" differently for each diplomat and the process of determining pass rate cut-offs remain shrouded in secrecy.

It's time for the ABIM to stop hiding their "secret sauce" of test development, changes in test scoring, and test "security" processes and address this young physician's questions directly and honestly. Test scoring and the setting of pass rate cut-offs should be transparent and reproducible for any high stakes examination administered to US physicians, especially when this self-proclaimed "voluntary" testing and re-testing now affects all US physicians' ability to gain (and retain) employment.

Or is that too much to ask of folks that use physicians as research subjects without their consent, engage in illegal lobbying of Congress and "stakeholders" for their benefit, and delight in moving our testing fees offshore to the Caymans for their own benefit?

-Wes

* Portions of the letter were edited to protect the physician's anonymity.

ABIM Needs to Prove MOC Useful

From Medical Economics:

(Click to enlarge)

- Wes

P.S.: Here's my rebuttal to the same article that Medical Economics refused to print.

Teirstein vs. Nora

Lois Margaret Nora, MD, JD of the American Board of Medical Specialties vs. Paul Teirstein, MD of the National Board of Medical Specialties debate at the California Medical Association meeting 16 October 2016:

Nora was given  20 minutes to speak.

Teirstein was given 10 minutes (but California Medical Association members booed until he got another 5 minutes).

Definitely worth a 14-minute viewing for it shows the revolving door of non-profit interests that maintain the MOC monopoly as only Paul Teirstein, MD can:


-Wes

Dr. Teirstein's slides can be downloaded here or a a pdf of his slides can be viewed here.