Scientists and Lawyers Argue For Open US DNA Database

chrb writes "New Scientist has an article questioning the uniqueness of DNA profiles. 41 scientists and lawyers recently published a high-profile Nature article (sub. required) arguing that the FBI should release its complete CODIS database. The request follows research on the already released Arizona state DNA database (a subset of CODIS) which showed a surprisingly large number of matches between the profiles of different individuals, including one between a white man and a black man. The group states that the assumption that a DNA profile represents a unique individual, with only a minuscule probability of a secondary match, has never been independently verified on a large sample of DNA profiles. The new requests follow the FBI's rejection of similar previous requests."

Comments

chimps have 97% of human DNA

[mangu]

Before DNA tests are accepted as conclusive much better studies should be done, particularly for false positives.

I believe DNA tests should be used for finding someone innocent rather than guilty. Negatives aren't that big a problem. If there are discrepancies then obviously it's not the same DNA.

Positives are another issue, how many common features there must be to accept two DNA samples as coming from the same individual?

Re:chimps have 97% of human DNA

[Hurricane78]

And that’s far from all. Imagine having a bone-marrow transplant. Now your blood has another DNA than your skin!
I remember reading about a person, who had three different types DNA in his body... at the same time!

DNA can be as easily faked as fingerprints. Hell, I could just “accidentially” cut a big politician, while getting his autograph. And then plant that DNA at a murder site. While I myself am completely sealed off in a virus-lab-style overall.
A overall that suffices will be below 50 bucks an a special store. And an autograph just is some travel costs. Everybody can do it.

Re:chimps have 97% of human DNA

[Anonymous Coward]

I am consistently horrified that juries offload their responsibility by blindly applying the judgement of expert witnesses (who are often paid to say the same thing over and over again), whether forensic scientists, psychologists or IT specialists. I take DNA evidence the same way as I take the contents of a third party /var/log: with a pound of salt, because I know it could have been planted.

When I was a juror I was interested in means, motive and opportunity as necessary but not sufficient conditions to vote guilty. I also made use of the defendant's inconsistencies in his testimony, details about the background of the defendant and victim to the extent that it was relevant to his alleged act, consistency of information from eye witnesses around the time of the event, known and unknown, doctors' reports, police officers, etc. I paid little attention to forensic details which might, according to the arguments of a scientist, help /confirm/ the prosecution's case, because I have more than reasonable doubt in my mind of any evidence which requires me to be an expert to interpret correctly - especially when I'm not that expert, instead deferring to some guy I just heard in a courtroom.

Re:chimps have 97% of human DNA

[Anonymous Coward]

DNA can be as easily faked as fingerprints. Hell, I could just "accidentially" cut a big politician, while getting his autograph. And then plant that DNA at a murder site. While I myself am completely sealed off in a virus-lab-style overall.
A overall that suffices will be below 50 bucks an a special store. And an autograph just is some travel costs. Everybody can do it.

Why "cut" the politician at all?
Hand them the pen when asking for an autograph, that way you can get skin cells AND a partial fingerprint.

Re:chimps have 97% of human DNA

[TubeSteak]

Before DNA tests are accepted as conclusive much better studies should be done, particularly for false positives.

I agree with you, but I'd add the caveat that the study shouldn't be done with a Federal database that was never intended for the purpose.

Mission creep is the kind of thing /.ers usually rail against, especially when there are privacy implications. If scientists want to study large datasets, they should go put together their own, or buy it from someone who has. What shouldn't be happening is a database meant for law enforcement to be opened up to the public.

Re:chimps have 97% of human DNA

[hedwards]

I disagree, mission creep is serious, but as it stands now there's a sizable database held by the FBI that isn't necessarily known to be totally reliable. Worse is that there isn't really any way of knowing how reliable it is without some sort of outside review. The fact that the FBI has a DNA database should be far more concerning than open access to it. Finger prints were once thought to be unassailable evidence in court, now it turns out that since they typically only require a small portion of the fingerprint to match that it's not really particularly accurate in many cases. Same thing could turn out to be the case here.

Re:chimps have 97% of human DNA

[Leo_07]

I agree with mangu that "DNA tests should be used for finding someone innocent rather than guilty." Paternity tests are done in a similar way even though the general public does not seem to know: genetic microsatellite tests can disprove paternity but not prove if it is in fact the father due to false positives. The question should be how many microsatellite sites (sites that are usually different in the human population) should be analyzed to arrive to a conclusion?

Re:chimps have 97% of human DNA

[ceoyoyo]

So basically you're saying you believe witness testimony is more reliable than scientific evidence?

Certainly forensics should be scientifically validated and need to be evaluated in the context off all the evidence, but "paying little attention to forensic details" just because you don't personally understand them in full detail....

How is the testimony of an expert witness, which is scientifically verifiable, any less reliable than the testimony of the defendant, eye witnesses, doctors (expert witnesses themselves), police officers (also expert witnesses), etc?

Re:chimps have 97% of human DNA

[Anonymous Coward]

So basically you're saying you believe witness testimony is more reliable than scientific evidence?

I simply can't work out how you read that message from my post. I was saying that the corroborating testimony of multiple witnesses is worth a lot, while the /interpretation/ of some scientific test by some guy (for all I know, as a non-forensic scientist) is worth very little.

The jury is not there as an expert in forensic science - indeed, in the United Kingdom, a forensics expert might be exempted from jury duty - and, as such, is not qualified to decide whether he is witnessing "scientific evidence". He must judge the reliability of the interpretation by the expert witness in the same way that you treat all other witnesses. You are making a very dangerous appeal to authority if you decide that, because one witness is announcing himself as having "scientifically verified" his data, he is more reliable.

A doctor is there in his capacity as a reporter of the injuries which he treated. A police officer is there to report on what he saw and what he was told, and we must judge his impartiality, his memory, etc. A forensic scientist employed by the prosecution, moreover, there is therefore no other reason than to say "I am here to assure you that this evidence shows this guy was guilty, and trust me because I'm a scientist". He should be viewed with at least as much doubt as the police officer, if not more, because the police officer does not play the "I know more than you - trust me" card.

Re:chimps have 97% of human DNA

[Sebilrazen]

Hey, if they want to cut a politician, let them, we all dream of it from time to time.

Re:chimps have 97% of human DNA

[honkycat]

If the purpose is to independently evaluate the rate of false matches in a DNA database to be used in criminal investigations, what better database is there than the one that will be used for that purpose?

Privacy issues can easily be worked around here---there's no need for personally identifiable information (i.e., name or location, not the dna data itself :-P ) to accompany the database for this purpose. You might also worry about statistical independence between the sample to be used for the analysis and that used for testing the results, but there are very well established methods for using subsamples of a data set in just this way.

Re:chimps have 97% of human DNA

[thetoadwarrior]

Good luck with stabbing a politican to get blood to put into a cup and having no one realise what happened when the blood shows up at a crime scene.

Re:chimps have 97% of human DNA

[hey!]

Now you raise an interesting question.

Suppose that DNA can *rule out* suspects, but not convict them. In the interest of preventing miscarriages of justice, DNA screening should *still* be routinely done. I have a feeling that if that were the case, it wouldn't be done very widely.

It'd be a litmus test of a law enforcement agency: how interested is the agency in getting the right answer, as opposed to *an* answer?

Re:chimps have 97% of human DNA

[Anonymous Coward]

This innocent man:
http://en.wikipedia.org/wiki/Cameron_Todd_Willingham

Was found guilty and executed by testimony from BOTH a mistaken expert witness and multiple eye witnesses.

Re:chimps have 97% of human DNA

[AliasMarlowe]

Before DNA tests are accepted as conclusive much better studies should be done, particularly for false positives.

I agree with you, but I'd add the caveat that the study shouldn't be done with a Federal database that was never intended for the purpose.

You are probably right, if the only conclusion were to be scientific knowledge, so that the database would exist only in the interests of science. Unfortunately, the principal purpose of the FBI database is the provision of strong/irrefutable evidence to secure convictions. Other purposes are to aid in selecting suspects or to eliminate individuals from suspicion. Its suitability for these purposes is what has been questioned, and has never been empirically assessed. Indeed, the cited studies of comparable databases and of a subset of the FBI dataset suggest that the "genetic matches" are not irrefutable, and may be considerably weaker evidence than presented in court.
The FBI database should be quantitatively assessed for suitability for its intended purpose.

Re:chimps have 97% of human DNA

[hey!]

I wouldn't call it a case of mission creep. Research is needed to confirm that the database is suitable for the purposes it was created for.

These issues were identified as early as 1969, in a landmark HEW report on computer records and the rights of citizens. It boils down to this: inferences drawn from data that affect the lives of people ought to be rationally justifiable. This means not using data until its suitability can be established. Mission creep can lead to data being used outside the context it is reliable in; but we can also run afoul of privacy and due process concerns by collecting data in the first place without establishing it means what he hope it means.

I've been concerned for years about the reasoning used in DNA screening. It entails a long chain of assumptions, and while all the assumptions *seem* plausible, the chance that one or more of them is wrong or has some unknown wrinkle is not negligible.

Re:chimps have 97% of human DNA

[FuckingNickName]

Unlike DNA evidence,

You are the juror, not the forensic scientist. You are not being presented with the opportunity to collect and test DNA yourself. Even if you were to ignore the evidence that separate people can appear to have matching DNA as tested, and the evidence that evidence is often planted, and evidence that accidental contamination occurs, you are still assuming that the forensic scientist is accurately reporting his results, i.e. is both competent and impartial.

there have been many, many studies proving the unreliability of eyewitnesses. In a stressful, quick situation, people's memory is often wrong.

This is precisely why you don't rely on a single eyewitness, nor even on a group of eyewitnesses alone to make your case.

So because you're a doofus who didn't pay attention in school, you ignore the facts?

I don't know about AC, but I have had enough years of schooling to have a good idea of what I do not know, and one of the things I do not know is forensic science. All I can do is judge the reliability of the expert witness's interpretation, just as I do for all the other witnesses. I have also had enough schooling to know that, the more complex a system, the more difficult it is to identify any tampering which has occurred.

Re:chimps have 97% of human DNA

[Vellmont]

Mod parent way down, as GP said nothing of the sort.

We must have read two different posts. The post I read was trying to put both categories of evidence on equal footing. Neither is as perfect and reliable as the side presenting it is trying to make it out to be. Eyewitnesses are notoriously unreliable, and expert witnesses and forensic evidence have their own problems. Ignoring one of them over the other because of your personal bias is a bad idea.

Re:chimps have 97% of human DNA

[Hurricane78]

Sorry, I meant “accidentially cut by the sheet of paper you hand him”.

But you are right, there are easier ways. It was just what I came up with first. :)

Re:chimps have 97% of human DNA

[Vellmont]

The jury is not there as an expert in forensic science

The jury is also not an expert in eyewitness testimony. You accuse those who believe in forensic and expert testimony of making an appeal to authority. I accuse you of making an appeal to the infallibility of individuals and memory. People lie, have bad memories, are influenced by what they heard elsewhere, and insulate themselves from anything contradicting what they think they saw. How is that not as equally inaccurate as forensic evidence or expert testimony?

You don't have to pay a lot of attention to realize this. Just read any news story the day after it happened, and then later on find out what actually occurred. One example that sticks out of my mind was when the DC sniper was running amok, eyewitnesses claimed that the shots came from a white van. Later of course we all learned that John Muhammad was driving a blue Chevy Caprice (which you may note looks nothing like a white van). It doesn't take a lot of effort to find wild inaccuracies in eye-witness accounts.

You're right that we shouldn't take forensic evidence as a gold standard above all else. It simply needs to be interpreted with the unreliability of ALL evidence.

Re:chimps have 97% of human DNA

[donaggie03]

Many people would agree that there should be some sort of error or reliability checking in the FBI's database, or that they shouldn't have that database to begin with. But that is another argument entirely. Scientists aren't asking for access to the database in order to systematically verify the data. They would probably use the data with the assumption that it is correct.

Re:chimps have 97% of human DNA

[FuckingNickName]

I agree with this, with the additional proviso that (as I indicated below) it's more difficult to identify tampering of a complex system, especially one you're not very familiar with.

We're all fairly familiar with the acts of recalling and forgetting events we've seen, and we will (as good jurors) expect multiple independent eyewitness accounts if we are going to rely on eyewitness testimony. But we're not very familiar with the intricacies of collecting and testing DNA, and (as successes of convictions on DNA evidence show) it doesn't seem that we're putting on our critical thinking hats on when presented with complex science[tm] by authority.

Re:chimps have 97% of human DNA

[noidentity]

He didn't say he took witness testimony at face value; he said he looks at inconsistencies, etc. And his suspicion of evidence requiring an expect to interpret wasn't because he couldn't understand it per se, it was because he couldn't fully trust those who claimed to understand it, and couldn't fully trust that it wasn't planted. In other words, he values things he can directly perceive over those he must take at face value.

Re:chimps have 97% of human DNA

[ceoyoyo]

More precisely, he believes his interpretation of witness testimony about events etc. over his interpretation of witness testimony about forensic findings. Never mind that the forensic findings are (or ought to be) independently verifiable.

Re:chimps have 97% of human DNA

[FuckingNickName]

Never mind that the forensic findings are (or ought to be) independently verifiable.

All worthwhile eyewitness accounts are independently verifiable, i.e. involve independent eyewitnesses. Don't let a commendable scientific spirit enter a pathologically obsessive state where you're happy to take a report of a complex scientific procedure as close to infallible but won't accept a dozen people in a park telling you that the grass is green and the snow is white because "eyewitnesses are notoriously unreliable" and "the grass and snow weren't observed under scientific conditions". Such disconnect has been parodied since Aristophanes, and for good reason.

Re:chimps have 97% of human DNA

[ascari]

And neanderthal men have dna that's different from ours by only a handful of features. This supports the widely accepted theory that a large portion of violent crimes are committed by time traveling neanderthals and their pet chimps.

Actually, I thought tfa explained quite well how they look at specific non-coding markers in order to increase the signal to noise ratio. The noise being dna that codes for proteins common to most living organisms.

Re:chimps have 97% of human DNA

[Sparky McGruff]

The jury is not there as an expert in forensic science

The jury is also not an expert in eyewitness testimony.

Expertise in any subject area that is likely to come up in a trial will almost certainly get you excluded from the Jury pool.

Re:chimps have 97% of human DNA

[noidentity]

They may be independently testable, but they have not been independently verified. Given the distrust people have these days for those in authority, it takes more than someone in authority stating something before one is apt to believe it. Hence, more trust is put in one's own observations.

Re:chimps have 97% of human DNA

[jc42]

You are probably right, if the only conclusion were to be scientific knowledge, so that the database would exist only in the interests of science. Unfortunately, the principal purpose of the FBI database is the provision of strong/irrefutable evidence to secure convictions.

Exactly. There are a number of phrases describing this situation, such as "vested interest", "interested party", etc. The FBI has a strong incentive to, uh, select for data and methods that will maximize the conviction rate. They should not be considered a disinterested party in questions about the accuracy of their data or methods. The only way their DNA database should be considered acceptable is if it has been (and is continuously) vetted by independent observers who aren't rewarded or punished for their reports.

The rate of convictions being overturned by study of DNA evidence, and the strong objections that law-enforcement authorities have shown to this use of DNA, should be all that we need to say that the "evidence" in this situation is highly suspect. Anyone who values their own freedom should be interested in seeing law-enforcement DNA databases studied and monitored by independent observers. Otherwise, you may be one of the victims of invalid DNA "evidence" in the future.

Re:chimps have 97% of human DNA

[Enter the Shoggoth]

Also a child can have totally different DNA from the parent in rare cases because some women have two wombs where one is not used.

Whoosh!

WTF?

Re:chimps have 97% of human DNA

[TubeSteak]

Then put together a group, have everyone sign non-disclosure forms, and do the research.
I'm sure there's a Foundation or Agency that'd be more than happy to put up the grant money.

I'm not saying "don't do the research," just that opening up the database to every Tom, Dick, and Jane with a PhD would be a gross violation of the public trust, whether or not privacy issues can be "worked around".

Re:chimps have 97% of human DNA

[icebraining]

"The law is clear: anyone who voluntarily kills someone will have his head cut off"
http://www2c.ac-lille.fr/bts-lettres/images/FranquinInoires.jpg

Re:chimps have 97% of human DNA

[sjames]

The problem with independent verification, etc is that it's all in a sort of echo chamber. The experts all swear blind that a DNA match is proof positive, even in the face of contrary events. You can independently verify that the 13 (or however many) matches do indeed match, but how do you (as a juror) now verify that 13 matches is as good as proof beyond a reasonable doubt?

DNA testing as performed for forensics should be treated as exclusionary. That is, no match = not guilty, but match = perhaps.

Re:chimps have 97% of human DNA

[honkycat]

I'm not saying "don't do the research," just that opening up the database to every Tom, Dick, and Jane with a PhD would be a gross violation of the public trust, whether or not privacy issues can be "worked around".

Why? I don't understand the logic if the data cannot be tied to individuals. The FBI and law enforcement is one of the groups I personally worry MOST about having access and they've already got it. I'd feel relatively more comfortable if the data were public so we don't have to take their word for its quality, statistics, etc.

Re:chimps have 97% of human DNA

[jellyfrog]

Scientists aren't asking for access to the database in order to systematically verify the data. They would probably use the data with the assumption that it is correct.

Uh, what? I think TFA would disagree with you there.

Misuse Of Statistics

[Jane Q. Public]

I have been concerned for years about this, because you often hear prosecutors and "expert" witness testimony to the effect that "the odds are billions to one against this being someone else".

Among other possible statistical mistakes, these unrealistically large numbers are based on the idea that each genetic location being compared is statistically independent. But in fact we know that to not be so. What we definitely do not know is how, or how often, many of these may actually depend on each other.

Let me give you a purely hypothetical example: what are the odds that a genetic profile from a random person contains a gene determining curly hair. What are the odds of finding this gene in a random sample?

You can answer this approximately by simply observing what percentage of the population has curly hair. Let's say 1/4 just for argument. So your odds are 1 in 4.

But here's the kicker question: what are the odds that a genetic profile includes a gene for curly hair, given that it also contains a gene for sicle cell anemia?

The odds are going to change drastically.

This is not a real example, of course, just illustrative. But one can easily see that the contents of genetic locations are NOT necessarily statistically independent, even if one of them does not directly cause the other.

We simply do not know enough to say that any two genetic locations are truly independent. Therefore these huge probabilities ("billions to one" for example) being spouted by prosecutors are completely specious.

Re:Misuse Of Statistics

[Anonymous Coward]

DNA also contains metagenes that control other genes, which means that DNA itself can and does change, so treating it like it's absolutely immutable is ridiculous.

Re:Misuse Of Statistics

[misof]

Another misuse of statistics: Many people expect that FBI uses the DNA database in the following way:
1. get DNA sample from the crime scene
2. match DNA sample against all samples in the database
3. if you got a match, you got the killer.

This is not how it works. Say the real odds of a false positive are ten million to one. In a country of say 300 million people this still gives an expected 30 people who match the sample from the scene. Is each of them the criminal? Clearly not.

How it really works? Imagine that you already identified several suspects. If you take DNA samples of these few people and one of them matches the DNA from the hair from the scene, you can still conclude that given your knowledge, with a very high probability the person in question was present at the crime scene.

In other words, using DNA tests is perfectly reasonable as long as you know what you are doing, even if the probability of a false positive is several orders of magnitude larger than one to a billion.

Re:Misuse Of Statistics

[Nick Ives]

"the odds are billions to one against this being someone else".

I've heard odds of 1,000,000:1 for DNA testing here in the UK. That means that there are about 61 other people in this country who will match me on a DNA test.

DNA testing should only be used in conjunction with other evidence. When fingerprint evidence started being used everyone thought that fingerprints were unique and so therefore a match means you must've done it. We now know that getting a good print and matching it to records isn't an exact process so often the defence will have their own fingerprint experts try and refute the prosecution's interpretation.

What this means is that if there's fingerprints, DNA, a motive and a witness seeing someone of your description fleeing the scene then you did it. Just one of those, however, and it's not beyond reasonable doubt.

Re:Misuse Of Statistics

[Jane Q. Public]

As another poster stated, however, there are other factors that can skew even those odds.

For example, depending on the circumstances surrounding the crime, the probability of someone having planted the DNA evidence is often much greater than a billion to one, or even millions to one.

High probabilities are one thing, but grossly distorted "statistics" in the courtroom do not serve justice.

Re:Misuse Of Statistics

[Cassini2]

How it really works? Imagine that you already identified several suspects. If you take DNA samples of these few people and one of them matches the DNA from the hair from the scene, you can still conclude that given your knowledge, with a very high probability the person in question was present at the crime scene.

While true, this statement is yet another example from the trap of misleading statistics. Individually, your statement is likely true. However, collectively, for all the tests the FBI lab is likely doing, then it is likely false.

Look at it this way: "The probability of me, as a random individual, winning the lottery today is near zero." From this, it is tempting to conclude that: "no random individual in North America will win the lottery today." However, this is clearly not true. Multiple random strangers will almost certainly win the lottery today.

The statement "no random individual will win the lottery today" is false, because a huge number change occurred. There are millions of people in North America. A similar problem happens with the FBI genetic testing. They do a great deal of testing. Proving an individual test is likely correct is very different than proving large numbers of tests are all correct.

From a statistical analysis point of view, you would be better matching any given DNA profile against everyone else's in North America. Then you would know exactly how many random matches occurred, and if lab contamination occurred, because the sample would match the lab techs and the police officers DNA too. This is the test the FBI is arguing against. Nevertheless, this is the validation test that needs to be done, because modern PCR DNA techniques should detect significant numbers of people connected with the location and/or sample access path, over significant periods of time.

Re:Misuse Of Statistics

[Anonymous Coward]

The thing is, that 1,000,000:1 statistic is an *average* -- the standard deviation is quite large. There are some clusters where you could match over 1,000 other people on a DNA test, and some people might be the only match in the UK. The kicker is that unless the sequence being matched is very well known, they're unlikely to know which group the presented match falls into.

Re:Misuse Of Statistics

[honkycat]

If only that were the case. For example, from page 2 of http://articles.latimes.com/2008/jul/20/local/me-dna20:

In a typical criminal case, investigators look for matches to a specific profile. But the Arizona search looked for any matches among all the thousands of profiles in the database, greatly increasing the odds of finding them.

[emphasis added] What you say is how it usually works, and how it stands any chance of being statistically valid. In some cases I've read about (not sure if it's in the story I linked to or not), the raw "1 in 100 billion" type odds were presented, which is plainly and patently false when used in this manner, and I believe the defense was not allowed to correct this. Some states do not allow this sort of search, per the article, but some do.

However, this is not relevant to independent checking of statistics. I'm sure the FBI has done at least some good faith testing of their methods, but they are also far too interested in the outcomes of those tests to be trusted with that without some verification.

Re:Misuse Of Statistics

[bidule]

How it really works? Imagine that you already identified several suspects. If you take DNA samples of these few people and one of them matches the DNA from the hair from the scene, you can still conclude that given your knowledge, with a very high probability the person in question was present at the crime scene.

While true, this statement is yet another example from the trap of misleading statistics. Individually, your statement is likely true. However, collectively, for all the tests the FBI lab is likely doing, then it is likely false.

What is this nonsense?

Even if I roll a zillion dice I won't have a higher probability of rolling 6s. You are arguing that my cookie will have a salty taste because someone in Ouagadougou mistakenly used salt instead of sugar. I can still safely say that a random cookie has "a very high probability" of tasting fine, even if I may end up with one from Ouagadougou.

Now, if there's a 1/billion chance of being salty and I have a million random cookies, I can say *none* are salty with less confidence. There's about 1:1000 (.9995001671244674 : 1000 using bignums) that at least one cookie will be salty.

The statement "no random individual will win the lottery today" is false, because a huge number change occurred.

And even if tens of millions play the lottery every week, it still happens once in a while that no one wins.

If one was as likely of winning the lottery as being falsely sent to prison because of DNA error, then it'd take quite a few million trials before no one wins. That's what I call "a very high probability".

Re:Misuse Of Statistics

[noidentity]

How it really works? Imagine that you already identified several suspects. If you take DNA samples of these few people and one of them matches the DNA from the hair from the scene, you can still conclude that given your knowledge, with a very high probability the person in question was present at the crime scene.

No, you can conclude with high probability that the DNA sample you're identifying is that suspect's, not that the suspect was there. It could have been planted, after all.

Re:Misuse Of Statistics

[Cassini2]

If one was as likely of winning the lottery as being falsely sent to prison because of DNA error, then it'd take quite a few million trials before no one wins. That's what I call "a very high probability".

Firstly, if the odds of a false conviction happening are 1 in a million, then 50% of the time the first false conviction will happen within the first 700,000 people.

Secondly, you don't know the probability of being falsely sent to prison is in fact one in a million, because no one has done the experiment to measure it. In fact, most applied scientists wouldn't ever claim such a thing about a PCR DNA analysis, because of the contamination problem. With a practical experiment, you will measure the highest fault rate for the lab.

I special in statistical analysis of gauges. It is very very tough to build any device that repeats to one in a million given practical real-world problems. You hit major practical issues, like the probability of people misunderstanding a gauges output is greater than one in a million. Seriously, if you have 3 people repeat the same simple observational test, the odds of all three people making the exact same mistake are often greater than one in a million. We inadvertently replicated this test on a system I worked on. Humans are a massive source of defects in any gauging system, and a well-known source of error in PCR DNA testing.

And even if tens of millions play the lottery every week, it still happens once in a while that no one wins.

Think of two probabilities:
1. The probability of a double lottery winner occurring, ie: the probability that someone wins the lottery grand prize twice inside their lifetime.
2. The probability of no one winning the grand prize of any lottery in North America.
I think you will find the probability that in any given week, (1) occurs much more often than (2). Even if you do some work to compensate for massive numbers of lotteries in North America, and assume that everyone plays in the same lottery (which isn't true), and only once (which isn't true), the first probability is still greater than the second, often by many orders of magnitude.

Statistics when dealing with large sample sizes are strange. Lay people don't comprehend them well, and even statistics people frequently make mistakes. One mistaken assumption about the dominant statistical failure rate, and the analysis is thrown. Additionally, when dealing with very large numbers, the statistics result in surprising conclusions.

Re:Misuse Of Statistics

[westlake]

I have been concerned for years about this, because you often hear prosecutors and "expert" witness testimony to the effect that "the odds are billions to one against this being someone else".

Imagine a murder in Buffalo, NY, population about 290,000.

Imagine that you have three credible matches:

1 The priest in North Dakota.
2 The engineering officer from downstate now serving in Afghanistan.
3 The real estate agent and deer hunter in suburban Amherst NY who lives five miles from the densely wooded creek bed where the body was found.

The universe of possible - plausible - suspects in any case is often quite small. The forensic evidence may expose other significant connections.

Re:Misuse Of Statistics

[demigod]

I'm sure the FBI has done at least some good faith testing of their methods ...

Is that the same FBI as the one mentioned in this article?

Re:Misuse Of Statistics

[Jane Q. Public]

I believe parent was referring to the fact that it is not valid to apply sample statistics in individual cases.

As another kind of example of that, let's say (purely hypothetically) that 20% of convicted criminals in your state are Hispanic.

So you have just pulled over someone for a traffic violation, and the driver is Hispanic. Do the statistics say that this individual has a 20% chance of being a criminal? No, they do not. Such statistics are completely meaningless in specific cases.

This is one of the reasons that racial profiling (or other kinds of profiling, for that matter) is a bad idea. It just doesn't work the way most people think it should.

Re:Misuse Of Statistics

[russotto]

How it really works? Imagine that you already identified several suspects. If you take DNA samples of these few people and one of them matches the DNA from the hair from the scene, you can still conclude that given your knowledge, with a very high probability the person in question was present at the crime scene.

OK, we took DNA samples of the people. One of them had their DNA all over the scene; on the body, on the murder weapon, etc. No match for the others. Open and shut case, right?

But wait, that person had an airtight alibi; he was half a world away in a public place with many witnesses.

How? Yeah, he was there; he lived at the crime scene. His DNA was all over the place and the murder weapon was one of his knives. But he wasn't there at the time of the crime, which the DNA doesn't tell you.

I don't know how it works in the real world. But I do know on those "true crime" shows I often hear prosecutors using that exact sort of reasoning when a man is accused of killing a woman he lived with -- "His DNA is there, therefore he must be guilty" -- as if the DNA is magic and knows only to shed when a crime is taking place.

Re:Misuse Of Statistics

[bidule]

The statement "no random individual will win the lottery today" is false, because a huge number change occurred.

And even if tens of millions play the lottery every week, it still happens once in a while that no one wins.

Think of two probabilities:

1. The probability of a double lottery winner occurring, ie: the probability that someone wins the lottery grand prize twice inside their lifetime.

2. The probability of no one winning the grand prize of any lottery in North America.

I think you will find the probability that in any given week, (1) occurs much more often than (2). Even if you do some work to compensate for massive numbers of lotteries in North America, and assume that everyone plays in the same lottery (which isn't true), and only once (which isn't true), the first probability is still greater than the second, often by many orders of magnitude.

... and?

You are not answering the point. Contrary to what you stated "no random individual will win the lottery today" is true a few times every year. Maybe it's because you tried to use laymen terms that your explanations lack rigor, but they don't make sense to me.

I do agree that some single cases do not follow the norm. I do agree that you must validate every single case before applying the global statistics. It's just that your arguments to come to this conclusion are, well, not conclusive.

Re:Misuse Of Statistics

[zippthorne]

Suppose further that there is at least one other person who shares a DNA match with those three guys. (4) the actual murderer who is not in the database because his DNA was never collected (as DNA atm is only collected under certain conditions.)

The real estate agent will get the jail time, though.

The DNA database can be used to generate a list of persons of interest, but once used in that way, it becomes less useful as convicting evidence.

Re:Misuse Of Statistics

[jellyfrog]

That's more of a case of P(A | B) != P(B | A). That 20% of criminals are Hispanic doesn't mean that 20% of Hispanics are criminals.

Re:Misuse Of Statistics

[Jane Q. Public]

That's true. I mis-stated the case. I meant to refer to the percentage of an ethnic population that is incarcerated, not the percentage incarcerated who are part of a particular ethnic group. Mea culpa.

Re:Misuse Of Statistics

[Pollardito]

Google results for "metagene" seem to suggest that they're classifications attached to the way that genes express themselves. They're independent of the actual contents of the DNA and don't change the contents of the DNA (they only reflect how the genes operate). DNA should only change with mutations, copy-errors, and viri.
Maybe someone can drum up a good car analogy to make it clear

Independent Databases

[Jane Q. Public]

By the way, I should point out that there are at least several public and private DNA databases being developed in the U.S. alone. However, some of them are for special purposes (genealogy for example), and will test different locations than those used by forensics labs.

What exactly

[Anonymous Coward]

Are we talking about here? If this is a catalog of DNA of convicted criminals then it might be ok. But if its also DNA samples from other people who gave a sample to clear their name, then I don'yt think it should be made public.

Re:What exactly

[hedwards]

If they were given to clear their name, then the DNA shouldn't appear in the database at all. The knowledge from the DNA database is hardly something that you're average stalker is going to have much use for, the people that you don't want to have access are probably the FBI.

"Time for DNA Disclosure" letter can be read...

[bagofbeans]

Letter is at http://www.bioforensics.com/articles/Krane_Science_letter_2009.pdf/

Re:"Time for DNA Disclosure" letter can be read...

[ColdWetDog]

Except that the link has 404'd. Wandering around the website, I found this link which appears to be a number of articles on reviewing forensic DNA testing literature. More stuff to read....

It's not just the veracity of the DNA testing

[BetterSense]

Even more so than the issue of statistical independence or veracity of the DNA testing process itself (which SHOULD be investigated) is the simple possibility of corruption, incompetence, or simple mistake. If a DNA testing lab simply accepts a bribe to give their expert testimony, has a mistake and switches sample vials, etc, their expert court-testimonyer will still show up in court claiming "The chances are approximately eighty-three bazillion to one".

This giant number has the emotional effect of certainty, but that number is just the chances that the sample the DNA lab recieved corresponds to the DNA of the accused--IF NO MISTAKES WERE MADE and nobody is planting evidence or accepting bribes. It's not the chance that the accused is innocent. I'm sure this distinction is made in the verbal "fine print" but the jury will still be swayed. The giant odds numbers are nothing powerful emotional hooks. The real possibility that the DNA evidence does not finger the accused breaks down like this:

1:1billion the DNA matches someone else due to a flaw in the statistics of DNA testing
TIMES
1:$smallernumber the DNA lab has accepted a bribe, has a mole, made a mistake, etc
TIMES
1:$smallernumber the DNA lab has honestly received a sample from the accused but the sample was planted at the scene by police, the real criminal, or really bad luck.

The jury won't be considering these factors when they hear the "1:1billion" number. It's nothing but sensationalism.

Re:It's not just the veracity of the DNA testing

[Anonymous Coward]

The solution simple. Make it so that planting evidence is punishable by whatever the maximum the accused could have gotten for the prime is. So if a cop plants evidence in a capital murder case, he gets the death penalty. That should keep them honest.

Re:It's not just the veracity of the DNA testing

[techhead79]

Yeah I also think you'd find it hard to keep people in those jobs then. Who wants a job where they could possibly be put to death for just some lab tests...thanks to some mob wannabe taking the stand saying you took a bribe from them...bad idea.

Re:It's not just the veracity of the DNA testing

[NeutronCowboy]

Speaking of mistakes: last summer, there was a big furor in Europe over some mastermind criminal who was being implicated via DNA matching in all kinds of crimes: murders, car thefts, etc. The DNA popped up in France, in Eastern Europe in Germany - literally, all over the place. Newspapers started to talk about some sort of supervillain, able to commit crimes at will and escape undetected every time, and with the financial means and independence to constantly travel through Europe.

Turned out that the swabs used to collect the evidence from the crime scene all came from the same source, and had been contaminated by a manufacturing technician.

Now this is contamination during what is supposed to be a sterile manufacturing process. I can't imagine what the contamination risks are that come from regular use.

Don't forget the human epigenome

[junglebeast]

Scientists already know that the human genome (DNA) is not the complete blueprint for an organism. The human epigenome, which is far more complex, and contains more of the details about how to put those building blocks together, is no less important...and seems likely that it contains more of what separates us as individuals.

Privacy concerns?

[Vellmont]

Having the names of the people associated with each DNA analysis would be completely unnecessary. Just assign each person a unique, meaningless number in place of their name and the problem is solved. There's probably 6 other ways to solve the privacy problem and still make the data useful. If researchers find special cases where they need actual identities to better understand what's going on, make them sign NDAs and release the information to only them.

The FBI doesn't want to release this because they know there's a lot of partial or complete matches in the database. Suddenly having news stories about how there's 100 people in the FBI DNA database with the same 13 identifiers (flash to expert testimony claiming billions to one of such a match) would be a major disaster for the FBI. The FBI would then talk about how most of them are the same person using different names, and various other explanations, but the damage would be done (flash to news story about one side of a match being a 22 year old male from Alaska, and another a 76 year old female from Florida).

I understand why the FBI doesn't want to do this, but it's extremely important data about how valid this type of DNA testing is (especially within certain populations) (flash to news story about racism). Essentially the government holds evidence about the validity of DNA testing that's relevant to thousands of criminal cases that it refuses to release. That sounds like a strong constitutional issue to me.

FBI using outdated technology

[Animats]

The FBI's database only uses 15 markers, checking 15 sites in DNA. That's not good enough, and there are false matches. The problem is that they're using DNA technology from about 1990.

23andme, the commercial DNA analysis service, checks 580,000 sites in DNA. 23andme probably has enough data to validate the quality of the FBI's marker selection. That's a good way to check. Identical twins do match, even at the 23andme level of analysis.

Re:FBI using outdated technology

[Tisha_AH]

If it was only the FBI who had access to the genetic material than I would say that this is a major problem during a trial.

I would hope that my defense team would have access to a sample of the DNA taken at a crime scene and my DNA and they would run a more complete comparison. If they went into court with a more thorough DNA comparison that completely blew-away the CODIS matchup then this would turn the prosecutors case on it's ear.

If it is only 15 markers out of 580,000 that is used to determine your guilt or innocence then there is a major concern that the wrong person could be hanged for the crime.

I do believe that they should be using more than 15 markers, as you indicated, this is 1990's technology.

A hit on a 15 marker test should be a reason to narrow the pool of prospective folks down some. Further testing and evidence must be necessary to gain a conviction. We have seen where folks have been convicted off of partial evidence before.

Do they need to test for all 580,000? No, but a sampling greater than 15 is probably called for.

Re:Anonymized?

[Ironsides]

No, it wouldn't, for two reasons given in the article. First is that it is possible someone has two entries in the database. The only way to discover this is to find a matching pair of DNA sequences and then look at the personally identifiable information to figure out if you have a duplicate or not. Second, is the possibility the information in the database was entered wrong, and that someones profile does not match what their DNA is.

Similar to the Birthday Problem

[Anonymous Coward]

Its similar to the birthday problem. Given a class of 35 students the odds that one of them has the same birthday as yours are 35/365 = 9.5%. However, the probably that there are two students in the class who have the same birthday (not necc yours) is about 81% (check Birthday Problem on Wikipedia).

Its the same here. The probability of there being matches between different people in a large database of DNA is going to be a lot higher than the probability that there is a match to a given person or crime scene DNA.

Re:Anonymized?

[Anonymous Coward]

You do realize the DNA is PII, right?

Or did I just get wooshed?

p = (1/4)^(#base pairs)?

[multi io]

Forgive me that I'm a layperson who didn't RTFA, but this story makes me wonder how they actually arrive at these astronomically low probabilities that the DNA profiles of two different people are accidentally identical? They wouldn't just include some random base pairs in the profiles and then calculate the probability as p=(1/4)^(number of base pairs), which would not account for the fact that 99.xxx percent of all base pairs are identical in all humans... would they? I was always assuming that, given that scientists who know what they're doing should have invented this test, there was some sophisticated process that would ensure that they would somehow only choose base pairs from the subset that was actually different in different individuals (and, more specifically, where each of A,C,G and T would have a 0.25 probability of occuring). I'm still relatively confident that something like this takes place, but sometimes you can be just astonished at how stupid people can be...

Re:p = (1/4)^(#base pairs)?

[Vellmont]

Forgive me that I'm a layperson who didn't RTFA

I'd forgive you, but the article was written for lay people and it clearly answered your question.

I was always assuming that, given that scientists who know what they're doing should have invented this test, there was some sophisticated process that would ensure that they would somehow only choose base pairs from the subset that was actually different in different individuals

If you had read the article, you might have noticed that it says the test selected for non-coding DNA (that is it doesn't produce proteins) that commonly varies in humans.

How data would be misused

[Jeff1946]

Assume several thousand matches are found in the database. Defense lawyer will argue odds are in the thousands that the defendent was falsely matched. This is wrong. Much like the puzzle of how many people do you need to have at a party to have two with the same birthday (about 30, I believe). But the odds that two people have the same birthday are about 1 in 365 not 30/365 as would be falsely concluded using the same arguement as above.

Assume odds are 1 in 10,000,000 that two people have the same DNA profile. Then defense lawyers asks expert witness

"How many people would have to be in a stadium before the odds are greater than 50% that two have the same profile?

Witness "About 4400."

Of course the readers of slashdot would be excused from the jury by the defense as they would not fall for this.

Re:How data would be misused

[brian_tanner]

I don't understand what is to fall for. I guess it depends on if you are doing multiple comparisons or a single test.

If you independently identified a suspect and could put together a case against them, and *then* got a DNA match, slam dunk. In that case you're right, the jury should not fall for that argument.

However, what about the situation where DNA is found at a crime scene, and then a database search yields a match? Perhaps that person has no alibi or way to explain how what is apparently their DNA got into this rape victim. Then the defense should surely ask "How many people would have to be in the database before a DNA sample from the crime scene will match somebody?" If those odds are not infinitesimal and the case is built around DNA evidence, there is a big problem.

Re:Two measures?

[ChrisMaple]

There's this misconception that people lose their civil rights by becoming criminals. They don't.

A properly convicted criminal serving a jail sentence has lost a portion of his rights, the most obvious being the right to leave the jail.

Rights are what you have as a result of being human, i.e. a rational animal. When you act to hurt an innocent person (violating his rights), you have thrown away some portion of your rights immediately. If the violated rights are among those recognized by the government and you're caught and successfully prosecuted, then the government can punish or force restitution in proportion to the damaged rights of the hurt person. The government does this without violating your rights because you have forfeited them to the extent of the damage you've done. When the punishment or restitution is complete, the deficit in your rights is gone. Your rights are restored - whether the government recognizes it or not.

Rights in the sense of civil rights or political rights have a lot of similarity to the phrase "It is right that." If you are about to leave a grocery with a can of soup, "it is right that" you pay the grocer: he has a right to be paid.

-----

The protection of others is not the only reason governments jail people (and please don't confuse government with the fiction that is society). Punishment, political revenge, "protective custody", "crimes" that have no victims, are all reasons government use for imprisonment.

Subjective vs. objective evidence

[OeLeWaPpErKe]

I think the parent poster takes issue with your near 100% reliance on subjective evidence, and treatment of objective evidence (as reported to you by imperfect humans) as no better.

You could at least see that you would never convict a mobster (who'd have heaps of false witnesses standing by). Or anyone from a closely-enough-knit group for that matter.

It'd also be relatively easy to get you to convict an innocent. If you based yourself on witnesses alone, obviously, you can see why you'd be liable to convict someone of witchcraft, for example. Even in this day and age. There is never any shortage of witnesses for idiotic accusations.

Objective evidence must take precedence over subjective evidence. Yes, DNA can be planted/faked, but if that's the case, more research will find inconsistencies. The solution is not less reliance on objective evidence, imho, but more, and better methods. Perhaps some of this "stimulus" money could be put into creating an agency to improve the collection of objective evidence (like DNA evidence). Say paying a few people to plant fake evidence using a variety of methods without telling the researches and work that way on improved detection methods.

Re:Subjective vs. objective evidence

[FuckingNickName]

near 100% reliance on subjective evidence, and treatment of objective evidence (as reported to you by imperfect humans) as no better.

Not AC, but: precisely. See My post here. As a juror, you're not being given the opportunity to perform/commission tests, merely to determine the relative importance of various testimonies presented to you.

You could at least see that you would never convict a mobster (who'd have heaps of false witnesses standing by). Or anyone from a closely-enough-knit group for that matter.

If all eye witnesses, policemen, etc spoke in defence of the mobster, and the only evidence favourable to the prosecution was some DNA, it would be wrong to convict. Yeah, maybe the guy's a nasty piece of work, but I'm not convinced beyond reasonable doubt, and it's as likely that the exasperated LEO has planted something.

It'd also be relatively easy to get you to convict an innocent. If you based yourself on witnesses alone, obviously, you can see why you'd be liable to convict someone of witchcraft, for example.

No, because witnesses have to both provide evidence of guilt and remove reasonable doubt from my mind. 100 people saying that X is a witch may not technically have created any reasonable doubt, but it hasn't provided any evidence either.

Objective evidence must take precedence over subjective evidence.

A witness interpreting the collection and testing of DNA is subjective. A witness giving an account of what he saw is subjective. You attempt to move from "his truth" to "the truth" with peer review, and since you're neither a peer nor do you have the chance to perform much review, you should not treat the expert witness testimony as inherently more reliable.

Yes, DNA can be planted/faked, but if that's the case, more research will find inconsistencies.

Yes, eyewitnesses can be misled or their testimony be a lie, but etc.

Perhaps some of this "stimulus" money could be put into creating an agency to improve the collection of objective evidence

A government Department for the Elimination of Error and Corruption? I'm sure the USSR had something like that :-).

Re:Subjective vs. objective evidence

[OeLeWaPpErKe]

Your point is basically that it'd be impossible for you to convict an innocent, since you'd vote everyone innocent.

Granted, that would make it quite hard. I doubt it would provide much justice, though.

Btw, in the case of the mobster, you'd probably have 1 witness against him (the victim) and objective evidence. In addition to that there'd be masses of witnesses for the defense.

The solution to this, it seems to me, is to put more faith in objective evidence. Subjective justice is mob justice is not just at all.

Re:Subjective vs. objective evidence

[FuckingNickName]

Your point is basically that it'd be impossible for you to convict an innocent, since you'd vote everyone innocent.

How did you read that into my post?

Btw, in the case of the mobster, you'd probably have 1 witness against him (the victim) and objective evidence.

That's more than just "objective" (by which I assume you mean expert witness testimony) evidence, isn't it? Did the mobster plead the 5th, so there's absolutely no opportunity for him to slip up? Is there no evidence that he was involved in anything which might have led him to do what he supposedly did, i.e. no motive suggested by recordings of meetings he's attended, photographs of people he's been with, documents in the trash (virtual or physical)? Nothing suggesting he was strangely near the victim at the time of the incident, i.e. no opportunity?

You're going to have to do a lot better than merely "the victim Z said X did it and there's X's DNA on the weapon". Perhaps actual attacker Y hated both X and Z and wanted to deal with both of them at once. Maybe Y, having brutally attacked Z, has instructed Z to blame X, and Z goes along out of fear for his/his family's life. If you were a professional criminal brutally threatening someone, wouldn't you think to set up a stooge, and make sure your victim plays along? You might not even coach your victim until after the event: if you can get to other witnesses, you can get to the victim too. Maybe through his family.

In addition to that there'd be masses of witnesses for the defense.

And the prosecution would hopefully demonstrate the unreliability of the witnesses. If they're all saying the same thing, what we need is evidence to falsify the account. Since forensic evidence is much better for falsifying rather than proving - what with being the result of a scientific endeavour - that mission has more avenues open to it.

The solution to this, it seems to me, is to put more faith in objective evidence.

I'm not sure whether I like any sentence starting "the solution is to put more faith in..." Less so when I've already argued that there's no "objective" evidence in court, only lots of witnesses, expert and otherwise, giving their testimonies.

black or white

[EricKoh]

excuse me for being a noob, but if two beings have the same DNA, how did one turn out black and the other white? unless, the samples were taken from m.j. at two different stages of his life...

Re:black or white

[canajin56]

They don't have the same DNA. It takes months of work and a shit load of cash to fully sequence somebody's full genome. DNA matching looks at 15 markers from 13 loci and counts the number of repeats. None of the markers they look at are in coding regions, meaning none of them come from their actual genes, just the so-called "junk dna". So if they're not even part of a person's genotype, they certainly will be unrelated to his/her phenotype. In fact, they won't even identify their gender, that's why CODIS actually looks at both 15 markers, plus AMEL, because without AMEL, they can't even get the gender of the sample. Anyways, it's certainly possible that some counts at a given markers are more frequent in certain populations than others, since they are genetic, after all, and would be passed along. But I don't think there are any that are unique to particular population groups.

Fixed Link

[chrb]

Good find. The correct link is: Science Letters - Time for DNA Disclosure

The FBI has used bogus scienee before.

[Required Snark]

The FBI has a history of using completely unverified pseudo-science to convict people. For 40 years they used bullet lead analysis, where they compared fired slugs from crime scenes to unfired bullets in the possesion of suspects. They assumed that there was consistency from batch to batch of bullets and that all the slugs in a box came from the same batch. Neither assumption was true. http://www.cbsnews.com/stories/2007/11/16/60minutes/main3512453.shtml [cbsnews.com]

It was only when a retired FBI metallurgist did testing by himself that he proved that the technique was useless. Then the NSF did a study and found the same result, and the FBI stopped using this test. http://www.fbi.gov/pressrel/pressrel05/bullet_lead_analysis.htm [fbi.gov]

Now the FBI has a secret data base that they use to claim that people are guilty. They will not release the data for independent verification of their results. Do you really think that they can be trusted one more time?

fingerprints not "scientific" untill recently

[peter303]

The method of fingerprint identification was more of a learned-craft than based on rigorous scientific testing up to 30 years ago. What saved its butt was that identification was computerized in the 1970s. If the algorithms gave too many false matches, then the technique would collapsed like a house of cards. But the algorithms appear to work reliably. I recall some defense lawyers attacking the fingerprint method at that time, much like the early years of DNA matching.

Re:fingerprints not "scientific" untill recently

[FuckingNickName]

Methods of forging fingerprints are so widely known these days that I can't see any reason to use the mere discovery of the defendant's fingerprint in support of anything beyond a preliminary investigation. It's not as if someone would have to be discovered with a home forensics lab in order for a fingerprint forgery defence to be realistic.

I'd take good notice of fingerprint evidence which adds reasonable doubt, such as indications that a fingerprint was forged (biometric IDs are just another arms race). Otherwise you'd have to engage me with a lot of convincing evidence that a fingerprint was made by the finger's owner.

In the event someone was premeditating a naughty act, the first thing they would surely plan is to sway the known variables in their favour (i.e. against the stooge's favour), such as those resulting from well-known scientific tests. It'd be unpredictable and esoteric observations which catch the prepared criminal.

similar argument for government security cameras

[peter303]

The argument from an article in Wired years back suggested that government security camera feeds be made available for realtime public viewing. That could then check abusive uses of this system when you have "the watched watching the watchers".

Ditto for open source software, like for computer security or voting. More eyes can spot more flaws.