On the importance of understanding false positives

[drwex]

I'm stealing this from docorion who stole it from the Nordic Cochrane Center

If 2,000 women are screened regularly for ten years, one will benefit from the screening, as she will avoid dying from breast cancer. At the same time, ten healthy women will...become cancer patients and will be treated unnecessarily. These women will have either a part of their breast or the whole breast removed, and they will often receive radiotherapy, and sometimes chemotherapy. Furthermore, about 200 healthy women will experience a false alarm. The psychological strain until one knows whether or not it was cancer, and even afterward, can be severe.

and

It has not been shown that women who undergo regular screening live longer than those who don’t.

Note that we're talking here about scheduled mammograms, done because the calendar shows a particular date, not a diagnostic test done because there are observed symptoms that need to be understood. We're also talking about the general population, not people whose family histories or other risk factors would lead them to be in a higher-risk pool.

For men, you can make a similar argument about PSA tests and prostate cancer though I don't have the numbers handy.

I realize that trying to get people to understand false positives/false negatives and the like is tilting at peculiar windmills, but I do it nonetheless.

Comments

[bkdelong.livejournal.com]

So is the lesson-learned here to get a retest elsewhere when there is a positive result to confirm? I guess if armed with the right study one could write an appeal to their insurance company to get them to cover a second test if they wouldn't provided their PCP would allow for the referral to a second doctor for the additional test.

I'm not sure

[drwex]

I don't know whether a retest is more likely to improve the accuracy. Since the result of a test is based on subjective interpretation of a visual image, the interpretations given to that image by people who have similar training are likely to be correlated.

Re: I'm not sure

[bkdelong.livejournal.com]

Aye, and there's the rub. Why wasn't there "Recommendations" coming out of this study? "Well' that's up to whomever sponsored it or folks with power reading it", one might say.

Well, educating folks about the dangers/risks of false positives and false negatives is only helpful if you provide recommendations on how to reduce both whether that be peer review of diagnoses that prove to be incorrect and some sort of follow-up monitoring or random sampling peer review of that doctor's anallysi going forward until they "improve" or something else.

IANAD by any means so I have no idea if this is even a plausible or workable solution, just trying to suggest possible reccommendations to solving the ongoing issue of false positives and improving them much like one would train AV or anti-malware software.

Use a peer/mentor network of more accomplished ...analysts (?) In the review process to better train those who seem to have one or more false positive (or worse, false negative) incidents.

Thoughts?

[dpolicar]

For what it's worth, I suspect your odds of getting people who don't already understand how false positives/negatives work in this context to listen to you would increase with a different example.

[feste-sylvain.livejournal.com]

Agreed. I generally use "mandatory workplace drug tests", where the false positives mean a family loses their livelihood, but justice can eventually prevail.

Part of the problem with the mammogram example is not just "false positives"; it's how the medical establishment responds to false positives.

Modern data-mining techniques can make use of all of the data from regular screenings. Without destroying some women's lives.

Really

[drwex]

My experience is that people are even less rational about drugs and piss testing than they are about cancer screening.

See also vaccination.

[ringrose.livejournal.com]

How accurate is accurate enough?

If it were 20,000 women screened, would that be OK?
Or is it the 1 person avoiding vs 10 unnecessary treatments? Are you looking for 1 person avoiding vs 1 unnecessary treatments?
Or 1 person avoiding cancer vs 200 false alarms, most of which don't result in treatment? Would it be OK with 20?

What is "good enough", and how do you measure it?

Hypothetically

[drwex]

We would not screen en masse under the current conditions which leads to more harm than good. Screening would be used where it is more likely to be effective. Were I given the magic health policy wand I would require something more like parity before encouraging a blanket public policy.

Why do you ask?

[sykotropic.livejournal.com]

So are you arguing that cautious watch and wait is what should happen? This is what is being promoted in the prostate cancer world. Basically, all men will die with prostate cancer. Not all men will die of prostate cancer. Will all women die with breast cancer? But, only some will die of?

Watch and wait isn't bad for a general group

[drwex]

What I'm arguing is that blanket screening does more harm than good. Screening for people who have known risk factors (family history, used to smoke, chemical exposure at work, etc) makes more sense.

I like a baseline

[sweetmmeblue.livejournal.com]

I'm for the blanket screening once at a particular age and then as needed based on symptoms and risk factors. Thus, if there might be a problem you have something to compare it against. The screening every year, or every years with no symptoms as well as all the other checks seems excessive.

Do other countries do this? Is this maybe part of America's obsession with breasts and controlling women's bodies?