High cost of food-borne illness: Says who?
From the Los Angeles Times comes news that the health-related cost of food-borne illnesses is an impressive $152 billion a year. That estimate is “more than four times” the figures calculated by the Department of Agriculture, the story says. Washing the grapes a little more thoroughly could apparently save us bushels of money.
So: Do we believe the new estimate?
Here’s what the Times gives us to go on:
The director of the Pew Charitable Trusts’ Food Safety Campaign–which helped support the study–said the report was solid, as we’d expect she would. The economist who wrote the report told the Times reporters he looked at “27 pathogens,” more than the Agriculture Department did in its studies. The FDA said it hadn’t yet seen the report. A food industry lobbyist said most problems are caused by consumers. A congresswoman called the figure “shockingly high.”
So, let me ask again: Do we believe the report?
The Times reporters, Andrew Zajac and P.J. Huffstutter, give us no help whatsoever. Did they believe it? If so, why?
Here’s a wild idea: Why not call up somebody who knows about this stuff and ask whether the report is legit?
I don’t need a congresswoman to tell me the figure is shockingly high; I can see that. If the report looked at more pathogens than the Agriculture Department did, then we can’t say this figure is four times what the government found; the two studies looked at different things.
Most important, however, is that Zajac and Huffstutter don’t give us any reason to believe this report. It could be a solid and important piece of research; or it could be puffed up environmental advocacy. The Times reporters did nothing to distinguish between the two.
Elizabeth Weise at USA Today gives us the same non-response from the FDA, and quotes an environmental activist who says, more or less, that if we washed the grapes we could save money. No help here.
Bryan Walsh at Time.com–same problem. No outside expert quoted. He quotes the congresswoman.
The AP, too, disappointed me. Reporter Shannon Dininny didn’t ask anybody whether this report was important or reliable.
The story by Steven Reinberg of HealthDay on the Bloomberg BusinessWeek site quotes a Dr. David Katz, director of the Prevention Research Center at Yale University School of Medicine. Is this, finally, the outside comment I’ve been looking for? I don’t know. Katz doesn’t address the methodology or reliability of the report, and I have a sneaking suspicion he’s connected with the report’s sponsors. Maybe not, but…I have a hunch.
Can I find what I’m looking for in the magisterial Wall Street Journal? Nope. How about in the blog of Scott Hensley, late of the Journal, blogging at NPR? Uh, no.
I give up. If you spot a story that tried to assess this report, please let me know. I’m still hoping there is something out there that will redeem the science press and ease my disappointment.
Grist for the mill: Pew Charitable Trusts press release. Report (pdf).
- Paul Raeburn
March 4th, 2010 at 5:03 am
Do you believe the report?
I looked it up here
http://www.producesafetyproject.org/reports?id=0008
and read the study to see what the methods they used were.
The author, an Ohio State University economist, computes an average cost per incident of foodborne illness at $1,851. This factors in the different average cost per case of a number of different pathogens; Listeria and botulism are expensive, and norwalk-like viruses are cheap. The calculation includes a “quality of life” number to monetize pain and suffering and discomfort; the pie charts show that is a substantial (45-60%) chunk of the total. Nothing implausible so far.
Table 2 shows cost per case * number of cases, and totals things up. Here is where the math gets interesting. The total cost is as advertised ($151 billion); but the table also includes a “confidence interval” for each number, with a resulting range at 5% of $39 billion and at 95% of $264 billion. The confidence interval math says that it’s based on a “Monte Carlo simulation to account for uncertainty in estimates”. I’m not precisely sure whether the 5% low estimate gives a simulation of the whole system, or whether it’s the low estimate for each of the events all summed together.
Table 3 breaks down numbers specifically on the different measures of non-monetary suffering accounting, and comes up with an even wider range of possible numbers.
The report’s conclusion says:
“In this report, I have demonstrated that, using what
I conclude is the best currently-available measure,
the mean economic cost of foodborne illness is
approximately $152 billion (95% CI $39-$265
billion), of which almost $39 billion can be attrib-
uted to produce. ”
that’s a mighty wide confidence interval, which the news media did not manage to notice.
March 4th, 2010 at 8:44 am
Edward,
Fascinating about the confidence interval; thanks for adding that detail, and you’re right–the media should have reported it.
Do I believe the report? Not on the basis of what I’ve read yet. And your analysis makes me even more skeptical.
March 4th, 2010 at 4:07 pm
I sent an email to the author, who replied this morning. In his response he noted a reason for the large variance in estimates:
“For example, the value of statistical life values come from a meta-analysis by Viscusi and Aldi (2004). The studies they examined had a mean value of $6.7 million with a standard deviation of $5.6 million. ”
So some of this numerical calculation depends on the existential question “what is the value of a human life”? To the extent that food-borne illnesses kill people, and to the extent that you as an economist want to reduce everything to numbers, then some of the variance can be attributed to that.
The other question is how current the number of illnesses is; here again is the author’s reply:
“On the other side of things, the data on number of illnesses is based on the 1999 Mead report. This is the most up-to-date CDC report and I did try to adjust the numbers in a reasonable way, but there probably have been some changes that I was not able to incorporate in the study. CDC has been promising an update for the last 3 years, but they have not released it yet.”
Based on those two observations, I’d say that the challenge is coming up with a number that doesn’t have such an enormous range that is so dependent on variables that are very difficult to pin down.
Having said that, some of the analysis is really good – trying to determine which of the various disease vectors causes the most harm is helped by this kind of analysis. Do we focus efforts on staph, or botulism, or norovirus? Do you aim at cases that kill a very small number of people very publicly, or at problems that are endemic and cause discomfort to lots and lots of people? If you ignore the sweeping conclusions, the details are something you could work from.
I’m just hard pressed to add up so many numbers and come up with any result other than something like
$152 billion, plus or minus $113 billion
or more succinctly “very, very, very bad”.
March 4th, 2010 at 8:24 pm
Again, Edward, thanks for the illuminating legwork.
March 5th, 2010 at 9:22 am
This conversation following the post is most impressive. The cost is likely to be somewhere between $30-some billion, and a quarter of a trillion, dollars? Unclear to me how much of this seems due to fatalities (and vulnerable to the fuzzy value of human life) or to non-fatal illness, loss of work, medical bills, etc. One conclusion is that the study’s authors and the press performed a service by publishing, and then publicizing, the conclusions. Another is that there is much more work to be done. Reporters did drop the ball by not saying so, and by giving readers little clue to how soft the numbers are.
And finally, reporters or somebody ought to look at the high end of that estimate. I don’t know about everybody else, but few of my medical bills, or those among people I know, stemmed from bad food. I know a lot of people who died, too, but none off the top of my head who checked out because of something they ate (not counting years of too MUCH they ate). Kaiser Family Foundation puts total health expenditures (2007 figures) at $2.2 trillion. Food borne illness at this study’s top end is about ten percent of that bigger sum (which doesn’t count, I presume, ancillary losses to the economy from deaths and disability). Any way you cut it, this has tainted food a potentially major driver of US health cost, a topic much in the news.
Reporting authoritatively and deeply on such things is a lot to expect of daily, breaking news reporting. But the topics ought to at least come up in news stories of any length. One expects that longer pieces are in the works – and more studies from academic and gov’t sources.