Fallible DNA evidence can mean prison or freedom

YOU are the juror: would you trust DNA evidence? Most people regard it as near infallible- it produces the right result or no result, exonerating the innocent and securing convictions where other evidence fails.

But DNA is not as objective as you might think. In the first of a two-part investigation, New Scientist reveals that much of the DNA analysis now conducted in crime labs can suffer from worrying subjectivity and bias. We asked forensic analysts to interpret a sample of real DNA evidence and found that they reached opposing conclusions about whether the suspect matched it or not. Our subsequent survey of labs around the world also shows that there are significant inconsistencies in the guidelines on how to interpret a sample. The findings suggest that the difference between prison and freedom could often rest on the opinions of a single individual.

The introduction of DNA evidence to the courtroom in the mid 1980s revolutionised forensic science, resulting in thousands of convictions and exonerating 255 wrongly convicted people so far in the US alone. The reason for more than 50 per cent of these wrongful convictions was unvalidated or improper forensic testing, including incorrect hair, blood or fingerprint analysis.

"It's not unreasonable to hold up DNA as a way that the rest of forensic science should be done," says William Thompson of the University of California, Irvine, and an occasional expert witness on DNA. "It is better validated, and often more carefully done and more rigorously interpreted than many areas of forensic science."

That's not the same as saying DNA is perfect, however. In a growing number of cases, DNA samples taken from crime scenes produce partial profiles, partly because smaller samples are collected. "Labs are trying to get more samples and they're trying to [get results from] lower and lower amounts of DNA," says John Butler, head of the US National Institute of Standards and Technology's genetics group, which aims to improve standards in DNA testing.

A standard DNA profile consists of a series of peaks relating to the number of repeating stretches of DNA found in certain genetic sequences, or alleles (see diagram). The repeats occur at specific locations on the chromosomes, called loci, and there are two alleles at each locus- one inherited from each parent. The number of repeats in each allele varies widely between individuals, allowing a person to be identified this way. Labs in the US typically look at 13 loci, while UK labs tend to look at 10.

Yet in partial profiles, alleles may fail to show up, a phenomenon called "drop-out". False peaks in the profile created by imperfections in the analysis machine may also be mistaken for alleles. This is called "drop-in".

It gets more complicated when several people's DNA is mixed (see "Mix and mismatch"). Butler has reviewed more than 5000 DNA samples from 14 US labs and found that mixing is a common occurrence: 34 per cent of the samples he studied included DNA from two people, while 11 per cent were three or four-person mixtures.

Interpreting alleles in a mixed or partial sample is where the subjective opinion of an analyst could play a part. To test this, New Scientist teamed up with Itiel Dror, a neuroscientist at University College London and head of Cognitive Consultants International, and Greg Hampikian of Boise State University in Idaho.

We took a mixed sample of DNA evidence from an actual crime scene- a gang rape committed in Georgia, US- which helped to convict a man called Kerry Robinson, who is currently in prison. We presented it, and Robinson's DNA profile, to 17 experienced analysts working in the same accredited government lab in the US, without any contextual information that might bias their judgement.

In the original case, two analysts from the Georgia Bureau of Investigation concluded that Robinson "could not be excluded" from the crime scene sample, based on his DNA profile. (A second man convicted of the same crime also testified that Robinson was an assailant, in return for a lesser jail term.) Each of our 17 analysts independently examined the profiles from the DNA mixture, the victim's profile and those of two other suspects and was asked to judge whether the suspects' profiles could be "excluded", "cannot be excluded" or whether the results were "inconclusive".

If DNA analysis were totally objective, then all 17 analysts should reach the same conclusion. However, we found that just one agreed with the original judgement that Robinson "cannot be excluded". Four analysts said the evidence was inconclusive and 12 said he could be excluded.

"Fingerprinting and other forensic disciplines have now accepted that subjectivity and context may affect their judgement and decisions," says Dror. "It is now time that DNA analysts accept that under certain conditions, subjectivity and even bias may affect their work." Dror presented the results at the Green Mountain DNA conference in Burlington, Vermont, last month.