Polygraph Follies
If you had a machine that could accurate predict whether or not people were lying, that would seemingly be a useful tool. Instead, we have the polygraph machine, which is like a lie-detector test, except that it doesn't detect lies accurate. Nevertheless, the CIA and FBI say it's a good placebo: "many of the tens of thousands of people who are subjected to it each year believe that it works -- and thus will frequently admit to things they might not otherwise acknowledge during an interview or interrogation." I guess that could be true, but it doesn't stack up well against the reality "that if polygraphs were administered to a group of 10,000 people that included 10 spies, nearly 1,600 innocent people would fail the test -- and two of the spies would pass" (Bayes' Theorem is relevant here).
Issues of fairness aside, the widespread use of these tests has got to have a serious adverse effect on the ability of the intelligence community to recruit and retain highly qualified, highly motivated personnel. I know one person who wound up not getting a CIA job based on a truly absurd polygraph result, and that obviously wasn't the only instance of such things happening.
So, using your figures, a failed result on a polygraph test changes a 99.9% chance that you're not a spy into a 99.5% chance? This is Not Impressive.
April 30, 2006 9:22 PM | Reply | Permalink
Only a 20% success rate in outwitting the polygraph test?
Mon dieu. Où sont les espions d'hier?
April 30, 2006 9:40 PM | Reply | Permalink
Okay, you're gonna have to give us the math on this one. "You're not a spy" is a result that 2 spies out of 10 received. "You're a spy" is a result that 1,600 non-spies out of 10,000 received. Yes, 99.9% of test takers were actually not spies, but the polygraph doesn't return 99.5% accurate results. I don't know where that number comes from.
What's curious here (if I used Bayes correctly) is that .02 percent of all people who test negative for being spies are spies, which isn't terrible. However, .02 percent of all test takers is not the same as .02 percent of spies. If a terrorist organization or rogue state ran dozens and dozens of spies at our system, we'd let several through the polygraph.
Given that fact, the false positives also have to be a worry. It's fine that false positives occur; it's better to have those than false negatives. It would also be fine if we eliminated 100% of spies. But since we don't, there is such a thing as too many. We do have to worry about something that eliminates 16% of participants for being spies when, in fact, they are not spies, and which doesn't eliminate 16% of spies (I assume it was 16% rounded that let 2 of 10 spies through).
May 1, 2006 8:54 AM | Reply | Permalink
Wow, that's a lot more accurate than I'd thought polygraphs were; Accurate enough to be somewhat useful, if the problems are kept in mind.
Obviously one doesn't search for spies by randomly taking people off the street and polygraphing them.
May 1, 2006 9:44 AM | Reply | Permalink