Slashdot Mirror
Source Code On Trial In DNA Matching Case (post-gazette.com)
An anonymous reader writes: While computer analysis by other programs was inconclusive in matching DNA evidence to a suspect, one program, TrueAllele, gave a match. As reported in the Pittsburgh Post-Gazette, an expert witness for the defense wants access to the 170,000 lines of source code to determine whether the match is scientifically valid. Not surprisingly, the software creator is resisting. From the article: "TrueAllele, created by Dr. Perlin and in its current version since 2009, is the only computer software system of its kind that interprets DNA evidence using a statistical model. It can single out individuals in a complex DNA mixture by determining how much more probable a match is versus mere coincidence. Complex mixtures can involve multiple people, as well as degraded or small DNA samples. ... Although the technology is patented, the source code itself is not disclosed by any patent and cannot be derived from any publicly disclosed source. The source code has never been revealed, he said, and it would cause irreparable harm to the company if it were. In his declaration, Dr. Perlin said that reading the source code is unnecessary to validate the program, and that a review could be done in his office or online."
Guess if he didn't want his code audited, this guy shouldn't have marketed his software to this particular industry. Dumbass.
Seriously at what point is the general public going to stop accepting that bullshit lie?
My cold fusion work only when I run the experiment in my shed with no one watch too ...
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
From the perspective of the burden of proof placed on the Prosecution, they have to disclose how they arrived at this derived 'evidence' of a match via TrueAllele. Criminal justice can't be served using a "Black-Box" as an input.
Disclose the software and its methods to a legally-sworn-to-secrecy-expert-witness, or toss the evidence as inadmissible.
Perhaps it's time for a more open process and open source code backing these types devices before their results are accepted as forensic evidence.
CSI is a lie:
http://www.theatlantic.com/politics/archive/2015/04/csi-is-a-lie/390897/
Crime lab major errors:
http://www.mprnews.org/story/2013/02/14/news/saint-paul-crime-lab-major-errors-found
Guess if we can't see the code that shows that the DNA sample is valid then thrown out the results, pretty simple.
People need to learn if they are creating software that needs to hold up in court that they can't hide it. Trowing out the evidence would be the first step to make sure no one wants to pay for software that can't be audited in a investigation. Then no one will buy his software anymore and he can be happy that it is still safe from prying eyes.
If his method is patented he has no need to keep his code secret.
Unless it's shit, of course.
Watch this Heartland Institute video
If your going to use a computer to generate evidence then yes you must allow the defence to look at the technique that means source code. You must never be required to merely observe at somebody else's lab especially when there companys continued business relies on the test succeeding. Realy anything that another lab that is not associated to the first can not do should not be admissible.
This gets even more important when the tests are destructive so it can only be done once. Validating the means used etc etc elsewise it becomes a black box to provide evidence against whoever they want.
No sir I dont like it.
I'm interested how this plays out in the judicial system across the Atlantic.
Here we had, at the core, a similar case. We used electronic voting and a group wanted to know how secure the machines were. In the end the judge ordered them release the source code. It's free to download now. Once there is a verdict I'd like to compare the arguments of the judges to get a feel of the difference between Anglo Saxon law and European continental law.
Have a third party selected by the court audit the code. There are tons of firms which have the expertise to do this, be it any of the big four or even security consultants, and a court selecting one would make it independent from both parties in the case.
As the other poster said, he's not saying it's a patent or copyright issue, he's effectively saying it's a trade secret.
So the issue is really pretty clear isn't it? If he refuses to show his code to an expert witness and explain it, then the evidence can't be used.
We'll see what the judge has to say.
You were mistaken. Which is odd, since memory shouldn't be a problem for you
Pretty eye opening. Worth the 5 minutes to scan through it.
You were mistaken. Which is odd, since memory shouldn't be a problem for you
You mean RAND() is not a perfectly valid methodology for determining guilt? Come on, he looks guilty.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
...I think the defense has the better argument. I have used software tools (both third party and ones I have developed personally) to do source code comparisons and analysis, but they only serve to point me to likely areas of investigation; I have never directly reported and relied upon the output from one of my custom tools in my expert reports.
A key aspect of expert testimony is that your analysis should, in theory, be repeatable by any other qualified expert using the same methodology (which needs to be spelled out in your report). If Perlin is relying directly upon his custom program for his conclusions, he needs to thoroughly expose his methodology -- which, in effect, means either allowing his source code to be reviewed or producing a detailed summary of his methodology that would allow someone else to reproduce it. Trying to claim trade secret status (which is what he's doing, in effect) for a expert methodology is an oxymoron.
Bruce F. Webster (brucefwebster.com)
Then it is not science, and it should not be admissible as evidence. It doesn't really matter that it can't be reproduced because the software vendor won't share their techniques as they believe the software to be a trade secret. If it is not possible to confirm results, it's not science! Yes, TrueAllele is a toy and not only am I skeptical of anyone using it as the basis of their scientific research, use of TrueAllele om court ought to throw any conviction into question as well.
“Common sense is not so common.” — Voltaire
Although the technology is patented, the source code itself is not disclosed by any patent and cannot be derived from any publicly disclosed source.
If the patent doesn't disclose the invention in sufficient detail for it to be reproduced by someone skilled in the arts, is the patent valid?
It is perfectly reasonable for an expert to suggest he uses proprietary techniques to arrive at an opinion, Courts unlike software types can rely on expert opinions not computer programs so the actual source code is irrelevant. Simple put whether the software is junk or not if an expert witness trestifies he believes its results it up to the judge and jury to access credibility. The prosecution is free to bring experts who can opine that the software has not been independently tested or validated and can not be relied upon without examining the source code. So this is just a typical defense strategy to delay or confuse the issues when you have an obviously guilty party. And I say thats fine.
I can understand why the manufacturers don't want to do that. It may well show that, oh dear, the best product is wrong 5% of the time (not good). Or that product X is head and shoulders above the rest.
But seriously, what industry-wide testing has been done? We're staking peoples' lives to the efficacy of this technology. How effective is it?
linquendum tondere
When you're talking about sending people to prison for the rest of their lives, or establishing family history, etc, you had better be prepared for a software audit. If he wants to stay in the industry, he's going to have to get his code audited.
This has already been ruled on for traffic cameras. I think that is a pretty good precedent.
Nothing to do with copyright law.
It has everything to do with copyright law. It's what the company is using in order to claim that they have a right to keep information from the court.
No, even if they would show the code, it wouldn't become magically free software or public domain. What they claim here is that they want to keep a trade secret.
Correct. It has nothing to do with copyright law. The intellectual property law here is trade secret law.
http://www.geoffreylandis.com
Essentially this guy is going to jail based on a secret algorithm that can't be verified.
I can't imagine how this could be legal.
I don't read your sig. Why are you reading mine?
The main problem is finding a disposable expert.
The people who originally clean-roomed the IBM BIOS for Compaq were split into two teams, with a Chinese wall between them: the analysis team, and the implementation team. The analysis team analyzed the IBM BIOS, wrote a specification, and then the implementation team implemented a BIOS to that specification. At which point the analysis team were effectively "burned", as in being forever barred from ever working on an implementation team in the future. They were highly paid for this, but they were disposable.
As with clean-room engineering, this expert would not be permitted to work on any software covered by the trade secret in the future. In an expert witness situation, you might be able to get away with disposing of the expert, if all they did was witnessing, rather than actually coding in the field of expertise themselves. However, how likely is it that you can find someone like that who also qualifies as an expert?
Further complication: Having testified (presumably in favor of the prosecution, in this case), would the expert witness be permitted to testify on similar goal programs in the future, given what the [now] knows about the process and techniques of the one they testify about today? Would exposure to multiple, competing trade secrets, damage their ability to perform an unbiased analysis, given what they knew from earlier experience? In general, I think you [as the defense] could argue that it, in fact, did damage their impartiality in their analysis.
One word (well brand): Volkswagen
If software is used to determine the culpability of a person (specially after two other analysis didn't), I think it should be totally scrutinized and reviewed for validity by MULTIPLE third parties. The binaries should then be built in front of an approval team (to avoid fraud by compiling with a different source base) before independently SQT and approved for release.
Not trying to defend criminals. I just think that fair is fair. Unless it is proven beyond reasonable doubt that the software is accurate (to an acceptable degree since nothing is ever perfect), no software or hardware product shouldn't be used in a legal process.
Essentially this guy is going to jail based on a secret algorithm that can't be verified. I can't imagine how this could be legal.
Actually, it's worse than that, he's facing a death sentence. It's inconceivable to me - but, sadly, unsurprising in this day and age - that someone might be legally executed based in part on the results of a proprietary algorithm that the defense is not allowed to examine. I can only hope the judge recognizes the seriousness of the situation. This case definitely bears watching
The typical reason to keep code secret from everybody is because it is of abysmally bad quality or there are other severe problems hidden in there. Reasonable-quality code gets inspected and audited by 3rd parties all the time under NDA. In this particular case, it may also well be that the code does not do what its creator claims and the patent is bogus. If the expert finds this, the code becomes worthless and the creator may even become a target for litigation.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
From the article
According to a court filing made by Dr. Perlin in the case, his company, Cybergenetics, “has invested millions of dollars over two decades to develop its TrueAllele system, the company’s flagship product. Although the technology is patented, the source code itself is not disclosed by any patent and cannot be derived from any publicly disclosed source.”
Other points 1) Dr Perlin's software does not have a cited "independent validation study" that is a study without the involvement of Dr Perlin. 2) "TrueAllele, created by Dr. Perlin, "is the only computer software system of its kind that interprets DNA evidence using a statistical model."
Here is the trouble I see, 1) Cybergenetics may have a problem defending it's patent, give that the software cannot be derived from "any publicly disclosed source" does the patent really disclose the real methods or it is likely this is another "software patent" with all the legal problems these currently have at the supreme court. 2) The sofware, Cybergenetics and software validates all seem to be tied to Dr Perlin, if this causes problems in courts Cybergenetics market with law enforcement will evaporate. 3) We know "software engineers" have been publicly accused in the VW matter, could they not do the same for Cybergenetics with out Dr Perlin's knowledge?
"cause irreparable harm to the company". Yeah, I know corporations are people too, but how about the part were it might "cause irreparable harm" to the defendant?
Really they should not be relying on him at all to prosecute.
If other analysis systems can't match him well, the defense should be partying and ready to call it a day, regardless of how one particular system responds. That is easily reasonable doubt.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
The typical reason to keep code secret from everybody is because it is of abysmally bad quality or there are other severe problems hidden in there.
That's not the reason.
A lot of code violates copyright, patents, and license agreements like the GPL. You would *not* believe what some of the ATI and nVidia code looks like, and you would *not* believe the number of USB keyboards running firmware that one manufacturer pretty much copied wholesale from another, and you would not believe the number of companies that sell "sanitized" open source software as proprietary code to third parties.
If he refuses to show his code to an expert witness and explain it, then the evidence can't be used.
Not true...
As I understand it, he should be able to get his program (or a modification of it) to produce as an output:
- The computation of the probabilities
- The data used to compute them, with annotation giving a trace back to its source.
- The assumptions behind the computation.
The issue of HOW IT IDENTIFIED this individual is separate from WHAT IT IDENTIFIED ABOUT HIM. The former is the "secret sauce" and would not be revealed. The latter is the evidence and can stand on its own. Further, it MUST be able to stand on its own - because if it can't, it's inadequate.
Now if part of his "expert testimony" is that his program did NOT find any other people who 'matched' and this is somehow relevant, THEN how it goes about doing the matching also becomes relevant and he's hosed.
Of course the defence is going to do their darndest to monkeywrench the prosecution, and threatening the tool builder with disclosure of his trade secrets is a good move tactically. It's up to the judge (and possibly the appeals judge) to call them on it if it's just an irrelevant thrash.
(I say this all as someone who personally believes that DNA evidence should only be used for defence, not prosecution, in criminal trials, because non-match is definitive while "match" is a difficult probability estimate based on assumptions about genetics, gene distribution, gene correlation, and on some very difficult to grasp probability computations. Hunting for matches in databases is, IMHO, subject to false positives and overestimation of the improbability of the match being false, based on underestimation of correlation and the genetic and familial mechanisms that might promote it.)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
Surely the starting point would be the Unit Tests and System Test results of the code - not reading 170k lines of code. The test results for Acceptance Tests could then be compared to the programs current fucntion and cross checked with other similar programs and test data.
It seems like the software is accusing the defendant, and the defendant should have the right to face and question his accuser, which would mean reading the source code.
Join the IParty!
Where's the test data that demonstrates this? Is it real-world data or a test suite?
Has the defense team implemented the algorithm themselves and received the same results?
Evidence samples with a match of 0.5% have been used in court as proof by not telling the jury about the large number of mismatches. What p-values were used to identify a match?
30 years ago, Reader's Digest, of all journals, wrote how easily DNA evidence can be planted or damaged. Knowing person X visited location Y in the last 30 days may not prove anything, when 10 other people had opportunity too.
Rather than providing source code, you would want this kind of software certified. So, construct some tests with a number of samples and a number of reference samples and check whether the software can find one or more "golden eggs", that is a priori known matches. Also, do it double blind, etc.
I think,
You should have the right to cross examine the witness against you in open court.
The witness should have to explain the tests that he used to reach his conclusion.
He should not have to provide the tools to reproduce those tests.
It seems to me the defense should want to use their own tools to cross check the prosecution's tests.
(Unless this costs too much to make it possible to mount a defense?)
The question is how well should he have to explain his tests and methods?
It seems to me the bar should be in sufficient clarity so the defense can understand and reproduce the results.
In this case, the prosecution witness says he did this, and the defense witness says he needs more.
If he defense expert can clearly explain what he plans to do and why he needs more, then he should be able to convince the judge of jury.
In a complicated statistical search for an answer, he should need at least the search algorithm, matching criteria, source data, and results (both matches and near misses).
I'm not sure if the expert should have to provide the code to do the search, or if the defense should have to do their own legwork.
Seems like asking for the source, may be more a fishing expedition.
BAC can only be measured by a blood sample, and even then, different people have different tolerance to alcohol.
A long time drinker will have a much stronger tolerance to alcohol than let's say a 16-year old.
I know that even at .21 I can still walk, talk and function properly, someone else at .15 will be kissing the sidewalk and be barely able to breathe.
And Breathalysers aren't measuring BAC. They make an assumption of BAC based on what's in the breath. What about ashtma? what about somoene who smokes?
... or because it violates the GPL. :-D