ProPublica: Great story, but how much is too much?
I know I shouldn’t post on an article I haven’t read in its entirety–but I couldn’t get through ProPublica‘s latest investigative project.
The project was a great undertaking, by all accounts–a detailed analysis showing that millions of dollars are flowing from drug companies to doctor-marketers, some of whom have blemished records and limited expertise in the drugs they are promoting.
It is the result of a massive collaboration involving ProPublica and five other big news organizations: NPR, the Chicago Tribune, the Boston Globe, Consumer Reports, PBS’s Nightly Business Report. The links go to the stories that each of those news organizations wrote about the docs. I didn’t read those stories, either. And ProPublica’s story, “Docs on Pharma Payroll Have Blemished Records, Limited Credentials,” by Charles Ornstein, Tracy Weber, and Dan Nguyen, is only one of multiple stories that it is producing as part of this project. This single story alone weighs in at about 3,600 words, if I copied and pasted correctly into Word.
Megan Garber at Nieman Journalism Lab calls it a “big, important piece — the kind of anger-inducing, broadly affective narrative that is the bread and butter of investigative journalism.” And without having read it, I agree. This is exactly the kind of thing we should all be doing–shining light in dark financial corners.
The ProPublica story begins with three anecdotes concerning doctors who had been subject to investigations or disciplinary proceedings and who were subsequently paid tens or hundreds of thousands of dollars afterwards. It goes on to explain how the data for the story were collected. It then says it “found sanctions against more than 250 speakers, including some of the highest paid.”
That’s 250 out of how many? Only later are we told that the database contained 17,700 individuals. Does that mean 250 out of 17,700 had been subject to sanctions? If so–and please check my math on this–then about 1.4 percent of the individuals in the database had been subject to sanctions.
That’s, uh–not too many. I have no data–and have done no investigation–to support the claim I am about to make, but here it is anyway: If you compiled a database of newspaper reporters showing how many had been subject to sanctions, I bet it would be more than 1.4%. I’m just sayin’…
Including the three anecdotes at the top of the story, I counted about a dozen anecdotes altogether. They followed this form: “Dr A, an expert in B, received C dollars from X Pharmaceutical Company, despite having a record of Y or Z in his past.” Reminds me of an old story in The Onion–It was the 8th subscription card that fell out of the magazine that finally made me subscribe! I don’t need a dozen anecdotes. After the first few, I get it.
Further, I don’t find any of this terribly surprising. Docs who appear to be independent experts, even if they’re paid by a company, are effective advocates. As one authority said in ProPublica’s story, referring to drug makers, “Are any of us surprised they’re trying to maximize their markets in almost any way they can?” No, we’re not.
I’m sorry to be burlesquing this admirable effort. I’m in favor of investigative reporting. Along with many others, I lament its decline. I believe that the press has a critical role to play in keeping an eye on government and corporations.
I hope ProPublica does many more such stories. I hope the stories lead to legal and congressional investigations and help to curb abuses. But, please, don’t expect me to read any of them all the way through.
- Paul Raeburn
October 20th, 2010 at 4:42 pm
Paul, I hope you take this with the good intentions it’s meant with, and with a reminder of my warm regard for you. But I’m a little flummoxed by your dismissal of the story. I’ve not read the entire ProPublica story in question either. But I’m guessing the problem they’re pointing to is not that 1.4% of docs have blemished records, but that pharma is not doing anything to avoid hiring such doctors, despite that they present them as opinion leaders and especially sharp experts – models to follow.
Beyond that, of course, many a study has shown that this sort of pay tends to warp the docs’ opinions, and that the practice effectively turns doctors posing as experts into salespeople, with detrimental effect on treatment, health, and the cost of healthcare. This is simply a huge problem.
Perhaps you’ve failed to convey the real nature of your dismay, but even if the anecdotes/case studies are a few too numerous, I don’t see anything here that makes me understand why you so thoroughly dismiss it. And, having just now skimmed over the story, I actually thought the case studies in the story you mention — which, being bulleted, were easy to skip if you wanted to — were valuable, for they gave different examples of how pharma put its money to work and described a variety and level of malfeasance that was new to me, even though I’ve read plenty of these stories. It IS important that we know doctors are literally being paid not just to lecture and influence other doctors with tales about jiggered studies — a sordid enough practice — but to directly write a lot of prescriptions.
I can accept that you may have strong feelings about the subject. But in a venue like this one, aimed at evaluating science journalism, I’d prefer to see a more substantive argument for dismissing an example of precisely the sort of scientific investigative journal that seems threatened by recent changes in media. Conceivably there’s an argument that such stories don’t really make a difference; after all, we’ve had quite a few, and too little has changed. (On the other hand, there have been some changes.) That sort of response too would seem to offer more than this dismissal.
Again — offered in good spirit, and hope you can take it so, even if this is a bit frank.
David Dobbs
http://daviddobbs.net
http://www.wired.com/wiredscience/neuronculture
October 20th, 2010 at 7:21 pm
David,
No apologies necessary; all criticism is welcome. This is a conversation, and it wouldn’t be very interesting if we didn’t challenge one another.
Perhaps I was too flip. I agree that this kind of story is important; I meant that when I said it. The ProPublica folks are talented and a model for the rest of us. I simply thought this story misfired.
The story led with the notion that many of the doctors getting money from pharma shouldn’t be getting it, because they’ve been disciplined or are not qualified. You make the larger point: that money influences doctors’ judgment. Even if the best and most qualified doctors take money from pharma, it’s still a problem. But the ProPublica story seemed to suggest, implicitly, that giving the money to only the best docs would be an improvement; it wouldn’t.
Further, trumpeting the news that 250 doctors were suspect seemed, again, to be beside the point, especially if that represents only 1.4 percent of the docs in the database. I can’t be sure that’s the right math, because ProPublica didn’t do the math itself–another flaw. I’m guessing that 17,700 is the denominator, but I don’t know for sure.
It may be unwise to criticize folks who are doing the kind of reporting that we all want to see more of. But just because it’s ProPublica, and just because it’s solid investigative reporting, doesn’t mean every story is going to be a gem.
Hope that helps. And thanks, again, for the thoughtful comment.
October 20th, 2010 at 7:44 pm
A few points:
Putting the actual stories aside for a moment, ProPublica has performed a very important public service by creating and making accessible the database of payments to doctors. I’ve tried to navigate through the individual documents posted by the companies and have found it nearly impossible to do any kind of serious research. Clearly this was the intent of the companies, and ProPublica has now provided a very useful research tool. So this is just the start of a project, not the final product.
About the actual stories. Paul’s post helped crystallize a thought that had been bothering me since reading the main story (and yes I did read the entire story). To be blunt, I think Ornstein et al buried their lede. The real story isn’t that industry paid money to some bad doctors. It’s that they paid all this money to all sorts of doctors, good and bad. Would it really have been less of a scandal if they had filtered out the bad docs? I don’t think so.
But c’mon Paul, that’s no reason not to read the whole story…
Larry Husten
http://www.cardiobrief.org
October 21st, 2010 at 5:54 am
Larry,
OK, OK–I’ll read the story! And fair point about the valuable database that ProPublica has put together. They did the same thing earlier this year for disciplinary actions against nurses, a move I praised in a post entitled ProPublica: Giving away the treasure.
ProPublica is a very important model for us, and I’m delighted that it is covering things that interest science writers. It should not, however, be immune from criticism, and I know, Larry, you’re not suggesting that.
October 21st, 2010 at 10:18 am
Lot of good points made here. I’d like to see ProPublica go the next step and attempt to figure out the denominator, and also whether the numerator might be much larger if access weren’t limited to the sources they had. Push the math!
October 21st, 2010 at 4:23 pm
Paul,
I agree wholeheartedly that ProPublica should not be immune from criticism. But I find it insulting and unhelpful that you chose to criticize the series without even reading it. Suggesting that it’s too long to get through or contained too much detail certainly does not move the conversation forward.
Perhaps my standards are too high in this age of instant feedback, but I believe the Tracker has an obligation to its readers and to those it critiques to carefully and closely read stories before punching holes in them.
I rely on the Tracker to give me an overview of the best (and worst) science journalism being done today. I don’t have time to read every story, that’s what I rely on you for–to point me to the pieces that are worth my time.
For this particular series, I would have liked to have known which parts were strongest, which pieces contained surprises or bits of information that are especially important, and how the series fit together (or didn’t) as a coherent whole. What did the series do well? What worked and what didn’t? A project of this scope deserves a thoughtful critique.
Please do criticize where warranted, but do so with respect.
Sincerely,
Christie Aschwanden
http://www.christieaschwanden.com
October 21st, 2010 at 4:47 pm
Christie,
Saying I couldn’t get through the story was, I confess, a gimmick to make a point. I thought it would attract attention–and it did. I also thought, and hoped, that people would find some humor in it.
My criticism was not, however, taken lightly, nor was it meant to be insulting. I made some specific points about the writing and reporting (hint: I counted all the anecdotes in the story, so maybe I read more than I confessed to). My recommendation? Read the top of a few of them–pick any you like–and you’ll get the gist.
If you’re going to be at the meeting in New Haven, we can pick up the conversation there…and thanks for your comment!