Evaluating Legal Evidence

DNA evidence, where crime scene traces of blood, hair, and the like are matched with those of a suspect, has now been used many times both to help convict the guilty and to free the innocent. However it has also led to gross miscarriages of justice because judges or juries have misunderstood the evidence - not about DNA but about the probabilities of matches. A misunderstanding known as the Prosecutor's fallacy is known to have led to a series of innocent people spending years in jail. The Prosecutor's fallacy arises because of confusions about probabilities given some information is known. The probability that it is dark, given it is midnight (fairly certain), is different to the probability that it is midnight, given it is dark (possible but not certain). Similarly, the odds that someone is innocent (given forensics found a match) is completely different to the odds that forensics will find a match (given someone is innocent). If a jury confuse the two, they have fallen for the fallacy. The difference is very subtle but very important. It is the difference between jail and freedom for an innocent person.

The Prosecutor's fallacy arose in evidence given in the infamous O.J. Simpson murder trial though neither defence nor prosecution noticed it. It can occur not just for DNA evidence but in any situation where an expert gives evidence about the probability that an event occurred. One, now famous, case was that of Sally Clark, originally convicted of murdering her two babies. The jury are thought to have believed there was only a 1 in 73 million chance of her being innocent because an expert testified that that was the probability of two cot deaths occurring in a single family. In fact she was innocent despite the odds presented (which in fact included several other flaws of reasoning). The Prosecutor's fallacy here is that the jury should have focussed on the odds that she was innocent (given the likelihood of two deaths in a family) not the odds for two deaths occurring in a family (given she was innocent and it was not murder). It was the latter odds that were actually presented. Calculated in the same way the odds that two children in the same family were murdered is an even less likely 1 in 2 billion. Yet she was convicted of murder.

A mathematical area known as Bayesian Probability Theory gives an accurate way of calculating the correct odds. However the courts have ruled that such complex mathematics should not be presented to juries as it could lead to miscarriages of justice for other reasons. The unresolved problem is how to stop juries and lawyers making a range of errors of reasoning like the Prosecutor's fallacy if they do not understand the mathematics. Professor Norman Fenton and Dr Martin Neil of the RADAR group have a potential solution. They have developed models that can be used to present the implications of complex mathematical arguments about odds in legal evidence in a way that is understandable to the general public. These models are then run using tools and techniques developed in close collaboration with the company Agena Ltd for assessing risks generally. If their approach is adopted by the courts it could help prevent innocent people being jailed.

For more technical detail about this work read the paper:
Fenton NE and Neil M, ''The Jury Observation Fallacy and the use of Bayesian Networks to present Probabilistic Legal Arguments'', Mathematics Today (Bulletin of the IMA, 36(6)), 180-187, 2000. pdf

For more on the RADAR group visit the RADAR Group webpages.