Thursday, July 28, 2011

Rebuttal: The NEJM's Justification of Medical "Mystery Shoppers"

The proposed examination of access to primary care according to insurance status in nine representative states was largely derailed by physicians and other critics concerned about the potential for government invasion of physicians' privacy. They argued that less controversial survey methods would suffice or that additional studies of the well-known primary care shortage are a waste of public resources. I think these arguments are misguided.

- Karin Rhodes, MD NEJM, July 27, 2011 (10.1056/NEJMp1107779)

Fair enough, Dr. Rhodes. You certainly are entitled to your opinion.

But before I take on my rebuttal to your piece, let's both be clear on a separate issue: what is most misguided about your perspective piece in the New England Journal of Medicine was that comments were not allowed. If they were, the "physicians and other critics" could explain their aversion to these tactics.

So, let me be the first to state my position.

Covert, subversive tactics in research in an attempt to avoid bias carries the risk of introducing additional forms of bias. For instance, when a phone call is made to a doctor's office for a new patient appointment and the problem sufficienctly urgent that other real live patients are rescheduled to accommodate a mystery patient's needs, how, exactly are the affects and costs to the established patients compensated? How will those data be "counted" in your statistics when your one new patient's access if offset by the loss of two follow-up patient's access. Who will explain to those affected by these tactics why they were rescheduled? Will you?

You claim that "the study was intended to generate valid national estimates of primary care capacity before the anticipated expansion of private and public insurance to as many as 38 million currently uninsured Americans."

First, recall the problems with that "38 million" number. Wasn't that number "47 million uninsured not too long ago?" Right off the bat, we see how numbers can be spun in policy circles, Dr. Rhodes. Which leads to the most important question that remains unanswered regarding a study that uses these covert tactics: how will the data be used? Will the data (which most certainly are going to "discover" problems with access) be used to justify mandates to shorten office visits from 7.5 minutes per patient to 7 minutes per patient to improve access? Or might doctors be directed to see more patients that are not insured? Seriously. What policy directives can we expect from these data?

You justify the use of this deceptive practice saying "the use of masking and concealed allocation, widely endorsed for randomized, double-blind clinical trials, lends confidence to the interpretation of results." And yet in the circumstance of randomized, controlled trials, patients must sign informed consent to take part in such a study before they are randomized. Hardly a "mystery" process. Should patients and doctors of prospective clinics not be afforded the same respect who might be asked to take part in your study?

You also seem to feel that a sampling 18% of states (9 of 50) is adequate to formulate conclusions. I find this concerning. National policy development should have representation from all states affected, not a minority. To suggest that the concerns of states with relatively high congestion mirror those with more rural populations is certain to bias policy decisions going forward and, more likely than not, exclude the perspective of less populous states.

So these are just a few of my concerns. There are others. Please note that none of them even begin to address the privacy issues raised by "others." But given the flaws I've outlined, paired with the obvious shortage of physicians that we will encounter in 2014 when the full brunt of the Patient Protection and Affordable Care Act kicks in (not to mention our limited research funding these days), this study certainly does appear to squander our limited public resources. Must we spend our resources to become Masters of the Obvious?

No doubt others would like to share their views, so unlike the New England Journal of Medicine, I'll leave my comments open.



rhythm doc said...

Touché, Dr. Wes. This 'survey' is, in my mind, as an Institutional Review Bd member and former chair, constitutes a study requiring oversight for human protection. Funny that the govt itself regulates human subject studies which determines criteria for approval and oversight.

rhythm doc said...

Funny how the govt itself regulates the oversight of human subject studies, which this 'survey' falls under. As an Institutional Review Board Chair I was appalled at the lack of consideration of this issue. You raise concerns for which am IRB would need to approve this 'survey'. I do not think Dr Rhodes ever took a required FDA human subject exam that all investigators are mandated to.

Keith said...

Having no access to the study protocol, there seems like there are legitimate ways to get around these various concerns.

First off, it is not clear that the study actually intended to make an appt. for the sham patient; simply to confirm whether appts were availible and in what time frame.

Secondly, if the above is the case, then who is potentially being adversely affected by this protocol? Use of office staff time? I do not see any impact on patients per se.

It seems necessary for our goverment to obtain data on physician supply and demand in order to properly fund those positions needed for the future, and since Medicare is the main funding source for residency training, it seems reasonable for them to conduct these real world studies in order to make decisions based on fact rather than guess work. Seems like alot of concern for reasons that are not apparent to me. Having said this, subject the study to IRB oversight and get on with it so that all you "goverment is peering into our lives" folks can feel satisfied.

Anonymous said...

Sounds very familiar to me. The totally false propaganda that keeps circulating about a nursing shortage. There is NO nursing shortage, and there will not likely ever be. Many thousands cannot find employment. Why is this? Corporate pays-off congress and soon most of the nurses who were educated at the very top universities here, in the United States will be unable to gain employment in their own country.

MDs now have similar worries in a way. You guys are just a little newer to it.