Slashdot Mirror
This Impenetrable Program Is Transforming How Courts Treat DNA Evidence (wired.com)
mirandakatz writes: Probabilistic genotyping is a type of DNA testing that's becoming increasingly popular in courtrooms: It uses complex mathematical formulas to examine the statistical likelihood that a certain genotype comes from one individual over another, and it can work with the subtlest traces of DNA. At Backchannel, Jessica Pishko looks at one company that's caught criminal justice advocates' attention: Cybergenetics, which sells a probabilistic genotyping program called TrueAllele -- and that refuses to reveal its source code. As Pishko notes, some legal experts are arguing that Trueallele revealing its source code 'is necessary in order to properly evaluate the technology. In fact, they say, justice from an unknown algorithm is no justice at all.'
I think it is very reasonable to ask access, covered by NDA, to a source code when such code is used to produce results for criminal prosecution. Unless they can show independent third-party validation of their tool.
We have seen issues with red light cameras, we have seen issues with labs doing drug testing on hair, we have seen child abuse panics from psychology "experts". Both methods and experts have to be open for independent, impartial validation. Otherwise they are no better than a duck test.
As Terry Pratchett wrote somewhere: "Evidence means 'that what is seen'". Nuff said.
Paaia
Jurors and judges need to know what the probabilities are. Remember, in a criminal trial, the standard for evidence is "beyond a reasonable doubt." Sending people to prison for life or even to death row based on flimsy evidence is unacceptable.
This isn't to say that it hasn't happened before -- Cameron Todd Willingham was executed in Texas on the testimony of an "arson expert" with no formal training in the field.
The code should be evaluated or the tool should be banned from court. The company doesn't like it? Too bad. They don't have to sell to the forensic lab/law enforcement market.
You have the right to face your accuser, which includes examining the evidence against you. This is secret evidence. It amounts to "because we say so", and should not be tolerated.
A software bug you're not permitted to look for could send you to jail. At least with a human expert witness you can cross-examine them.
One thing is having access to the source code and a completely different story is properly analysing it. When dealing with something as complex as (probabilistic!) DNA sequencing, it seems quite clear that the most sensible way to validate the program is actually using it. Set up a proper benchmark with a relevant number of samples and confirm whether this (+ any other) program works exactly as expected. This would also be an excellent way to objectively assess its accuracy.
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
Expert judgement can be countered by other experts. Here we are being presented with something as a "Fact". There is no way to dispute it and there is no way to verify it which is what people are having a problem with.
Questioning expert's qualifications is fair game in trials. If you can demonstrate that expert is not impartial, you can largely mitigate their testimony.
How do you question algorithm like if (1) = Guilty; other than code review?
Please point to this model and indicate where it shows your precise location will be under 10' of water by now. Either that or admit you are full of shit and a liar.
A lack of evidence of this "model" will indicate you are a liar whether you respond or not.
Well it shouldn't be accepted as fact. Ideally the courts would instruct the jury to treat the software's output as similar to a human being saying, "This is my expert opinion." You can submit your own software's "opinion" as evidence as much as you can get your own expert human to testify on your behalf.
It is true that you can't cross-examine it; but ideally, that should make the software less reliable. If you had an expert who, upon cross-examination, always responded, "I don't know, it just seems that way", then he wouldn't have much credibility. Ideally, software that can't justify its "opinion" should be treated the same way.
I have said "ideally" here several times, recognizing that it may well be the case that this isn't how people actually think. But I think a more constructive response to this misplaced trust is to help inform courts and defense lawyers more clearly (who should in turn inform the juries).
TCP: Why the Internet is full of SYN.
As a programmer I can assure you that I am infallible and perfect. My superiority is the reason I am a programmer and most people are not.
“Common sense is not so common.” — Voltaire