Slashdot Mirror
How Statistics Can Foul the Meaning of DNA Evidence
azoblue writes with a piece in New Scientist that might make you rethink the concept of "statistical certainty." As the article puts it, "even when analysts agree that someone could be a match for a piece of DNA evidence, the statistical weight assigned to that match can vary enormously, even by orders of magnitude." Azoblue writes: "For instance, in one man's trial the DNA evidence statistic ranged from 1/95,000 to 1/13, depending on the different weighing methods used by the defense and the prosecution."
Sadly true, but there's so much about DNA analysis that you don't get on an episode of CSI. On TV DNA analysis only takes a few minutes and matches are proudly announced by flashing messages on the DNA machine.
In real life good DNA matching takes days, cost a lot of money and, as the article points out, matching can be in the eye of the beholder. DNA samples are incredibly easy to contaminate, whole labs can become contaminated over time if they don't have and follow strict contamination protocols. And there has been more than one reported case of harried techs gun-decking DNA analysis when police and prosecutors were certain they had the right guy.
Well done DNA analysis can be an amazing crime fighting tool but the science is not perfect and it's okay to be skeptical. There is no magic identification test that's completely fool proof. And DNA tests are only as good as the fool running the test.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
... in a case I was on the jury for. (Sorry for the bait-and-switch title, couldn't resist.)
This was a case of armed home invasion. The victim was a big bruiser of a man, a multiple convicted drug addict. The defendant was a scrawny young Cape Verdean guy. (Cape Verdean drug gangs are common in the area: this is important later.) The victim testified that, after buying drugs from the defendant, he got a series of enraged voicemails demanding the return of the defendant's cell phone. A few hours later, the defendant allegedly shows up at the victim's house with a gun and barges in yelling. A struggle ensued, a shot was fired into the floor, and the guy with the gun fled.
Evidence against the defendant included eyewitness testimony from the defendant, matching ammunition found at the defendant's house, and crucially a do-rag found at the scene of the scuffle. DNA tests matched the do-rag to a mixture of at least 3 people, including the defendant. The DNA mixing was probably due to really awful police work: a paper bag borrowed from the defendant's cupboard is not a proper evidence collection container.
As in TFA, mixed DNA dramatically affected the "probability of exclusion" statistics: the state's expert testified there was a 1 in 50 chance that a random man on the street would match the DNA on the do-rag. The odds that a random *black* man on the street would match were much higher, like 1 in 20; the defense pointed out that the odds that a random *Cape Verdean* would match would be much higher.
We've grown used DNA evidence saying things like, "not one other person on the planet could match this DNA", but in this case, the odds were good that the DNA evidence would match at least one other person sitting in the *courtroom*. The defense also took the unusual tactic of introducing the defendant's sister, who testified that her *other* brother looks very much like the defendant, and she said it was *his* voice on the enraged voicemails. What are the odds that the DNA matches the *brother* instead? Damned good.
Between the fact that the eye witness seemed shifty and unreliable and was probably on crack at the time of the incident, and the fact that all the physical evidence could just as well implicate the brother as the defendant, we couldn't rule out the possibility that the cops got the wrong guy, so we found him not guilty. If I had to take a bet, I'd say he did it, but I wouldn't bet his life on it.
Anyway. Moral of the story is: on cop shows and in the public awareness, DNA evidence is rock solid and incontrovertible. But in the real world, the statistics of DNA mixtures make things a whole lot less cut-and-dried.