“Researchers report that a spinal fluid test can be 100 percent accurate in identifying patients with significant memory loss who are on their way to developing Alzheimer’s disease,” Gina Kolata writes on the front page of today’s New York Times.

One hundred percent accurate. That’s a startling figure in medicine and medical research. Can it possibly be correct? [Elsewhere in the same issue of the Times, as my colleague Charlie Petit points out in his Science Times review, a DNA test for colon cancer is also described as 100 percent accurate.]

Lauran Neergaard of the Associated Press, in this version of the story, didn’t even put the test in the lede of her story. In graph four, she writes, “Belgian researchers reported in Archives of Neurology that they found the pattern in 90 percent of the Alzheimer’s patients and 72 percent of the MCI patients in their study — but how to get accurate spinal tap measurements remains a big controversy.”

Reuters: “Measuring certain proteins in spinal fluid can accurately diagnose Alzheimer’s and predict which patients with memory problems will develop the fatal brain-wasting disease, Belgian researchers said on Monday.” And here is what reporter Julie Steenhuysen wrote in graphs four and five:

They said measuring traces of beta amyloid and tau — two proteins associated with the telltale plaques and tangles that form in the brains of patients with Alzheimer’s — accurately detected Alzheimer’s in 90 percent of patients with the disease.

They were also able to detect 100 percent of people with memory impairments who would progress to Alzheimer’s disease within five years. And they detected Alzheimer’s proteins in 36 percent of people with normal brain function.

From the press release: “A ‘signature’ consisting of three biomarkers in the cerebrospinal fluid was present in 90 percent of patients who had been diagnosed with Alzheimer’s disease but also was found in more than one-third of cognitively normal older adults, according to a report in the August issue of Archives of Neurology, one of the JAMA/Archives journals.”

The press release also says, further down, that in one of the studies reviewed by the authors, for “patients (n=57) with mild cognitive impairment followed up for five years, the model showed a sensitivity of 100 percent in patients progressing to Alzheimer’s disease.”

What we conclude, I think, is that in one set of data, in certain people with mild cognitive impairment, the test picked up all those who would progress to Alzheimer’s disease. That’s what “sensitivity” means. It was sensitive enough to pick up all of those who would develop Alzheimer’s. The release does not say anything about the test’s specificity. That is, was it also so sensitive that it picked up people who did not go on to develop Alzheimer’s? Was it, in other words, sensitive, but not specific?

The best we can say is that the figure of 100 percent does not mean anything like what Kolata says, without qualification, in her lede. The test had a sensitivity of 100 percent in one group of patients; it was not reported to be 100 percent accurate anywhere except on the front page of the New York Times. Kolata’s competitors were correct to be much more circumspect than she was. “Sensitivity” and “accuracy” are not the same thing.

Kolata does hedge, though: She says the test “can be” 100 percent accurate. I am stumped by that one; anything “can be” 100 percent accurate. And many readers will see “test” and “100 percent accurate” and miss “can be,” as I did the first time I read it.

If you try to find the backup in Kolata’s story for her assertion in the lede, you will quickly see that the story is oddly constructed. The back-up for the lede, such as it is, is way down near the bottom.

She begins, after the lede, with several grafs explaining how important such a test could be. She quotes a couple of authorities who praise the new test. She then discusses the problem of diagnosing a fatal disease in people for whom doctors can offer little or no treatment. We digress to talk about the difficulty of obtaining spinal taps, and then we get to the new study that backs up the lede–in graf 18.

In her review of the findings, she does not use “100 percent.” Instead, she writes this:

Nearly every person with Alzheimer’s had the characteristic spinal fluid protein levels. Nearly three quarters of people with mild cognitive impairment, a memory impediment that can precede Alzheimer’s, had Alzheimer’s-like spinal fluid proteins. And every one of those patients with the proteins developed Alzheimer’s within five years. And about a third of people with normal memories had spinal fluid indicating Alzheimer’s. Researchers suspect that those people will develop memory problems.

That’s more nuanced. And that’s what the lede should have reflected–it’s a good test, and it’s progress, but the test is not “100 percent accurate.”

If you’re concerned about whether I’m getting this right, I offer the following guarantee: This post can be 100 percent accurate.

- Paul Raeburn

This entry was posted on Tuesday, August 10th, 2010 at 12:36 pm and is filed under Health & Medicine Stories. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.