Showing posts with label cognitive bias. Show all posts
Showing posts with label cognitive bias. Show all posts

Wednesday, January 19, 2011

Why I believe that Amanda Knox and Raffaele Sollecito are innocent

Part 25 in the Knox/Sollecito case

The molehill of evidence
It is the evidence that is not there that is the prosecution’s weakest point, as former FBI agent Steve Moore pointed out. Consider blood spatter, for instance: “t is inconceivable that the person stabbing Meredith was not contaminated by blood spatter. Guede was. Anybody holding Meredith (such as was alleged by the prosecution) would be within the spatter zone. Again; blood on clothes and skin.”

There is no evidence that Amanda or Raffaele had any contact with Rudi Guede on the night of the murder. There is no evidence that they took any drugs other than cannabis. There is no DNA of Amanda’s in the murder room, and the only evidence of Raffaele’s is the highly contested bra clasp. Forensic Engineer Ron Hendry refuted the arguments of the prosecution that the break-in was staged. The difference in the amount of evidence against Guede versus Knox and Sollecito can be likened to a strong versus a weak signal, and only by pretending that the strong evidence is no better than the weak evidence can one come to the conclusion that all three are culpable.

The interrogation on the night of the 5th of November
The police seemed to have prior knowledge of Amanda’s text message to Patrick on the night of the murder, and the police may have known that the two of them met on the afternoon of the 5th. We also know from both Amanda’s contemporaneous statements and those of others that she was tired and scared in the days leading up to this interrogation (Candace Dempsey, Murder in Italy, Chapters 5-8).

There are many instances of people making a combined false accusation and confession, and this is one of them. The interrogation started around 11 PM and produced two statements, one around 1:45 AM and the other around 5:45 AM. She asked whether she needed a lawyer and was told that would only make things worse. Her statement the next morning shows considerable confusion. Also, if she had been completely rational during the interrogation, she would never have accused Patrick, whether she were innocent or guilty. She believed that he was at the bar that night, which should give him a rock-solid alibi. Again, Steve Moore’s comments are extremely useful and thought-provoking: “Why would detectives schedule an interrogation overnight? ...the reason they interrogated Amanda all night was to break her. Not get the truth, not get answers, not make Perugia safer; but to break her so that she would say what they wanted her to say.”

An ordinary kitchen knife, not the murder weapon
The large knife from Raffaele's flat did not match at least one and probably not two of the three major wounds. It did not match the bloody outline of a knife in Meredith’s bedroom. The DNA on the handle from Amanda was probably deposited when she used it for cooking at Raffaele’s flat. Although the arguments are sometimes detailed, Meredith’s DNA on the knife is probably due to contamination in the laboratory itself, but it may have been contaminated during its transport.

Shoeprint and footprint evidence
The police tried to insinuate some of the luminol-positive footprints were due to Knox and Sollecito and that all were set in blood. This attempt was intellectually dishonest. All of the shoeprints matched Guede’s sneakers. The luminol-positive footprints in the hall do not appear to be Meredith’s blood, inasmuch as they did not have Meredith’s DNA. The one bloody footprint in the bathroom looks a little bit more like Guede’s foot than Sollecito’s foot, but attributing it to either person unequivocally is questionable at best.

Cognitive bias and tunnel vision
If they are innocent, then how did they get convicted? Amanda and Raffaele were detained and held without charge before the forensic evidence came back implicating Rudy Guede. The day of their arrest the police paraded them through the old town with lights on and horns blaring. This had only happened one time previously in Perugia, according to the memory of one citizen, when a mafia figure was arrested. By the time Guede was becoming a suspect, a major figure in the Rome police department had put Ms. Knox’s picture in the hallway, right next to the arrest of Bernardo Provenzano.

This was not a conspiracy in the sense of a bunch of people sitting around a big table. I think it is a case where the police and public minister (PM) Mignini had made a bold claim about Knox and Sollecito’s involvement and could not back down. It may have been a situation where the forensic police (especially) felt, “To get along, go along.” In addition, the forensic scientists might have really believed that Knox and Sollecito were guilty and subconsciously tilted their results in that direction. Koppl and Balko wrote, “To the extent that it's possible, evidence should be stripped of all context before being sent to the lab.” Given that Knox and Sollecito were already in custody in a high-profile case before some of the evidence was even collected, it is difficult to see how all cognitive bias could have been avoided.

Poor forensics and lack of discovery
The prosecution’s misrepresenting which tests were or were not done and their withholding of electronic data files underlying the DNA forensics suggests that they know how weak their case really is. But it is remarkable how much they did wrong or did not do at all, for no reason that I can understand. There is a possible semen stain on Meredith’s bed that was never followed up with a confirmatory test. There were three computer hard drives that the police so badly mishandled that recovery of the data has been difficult or not even attempted. The collection of the DNA evidence with dirty gloves that were seldom changed is just plain sloppy. There is evidence of Meredith’s blood mixed with Amanda’s DNA in certain places in the flat, but the prosecution misleadingly implied that the samples were from Knox’s blood. Since Amanda lived at the flat, finding her DNA there is not inculpatory at all.

Exculpatory evidence
Meredith is known to have returned home around 9 PM. Many small pieces of evidence point to an earlier time of death, before 10 PM, than the prosecution indicated, about 11:40 PM. Meredith had left a load of laundry in the washing machine, presumably when she left to visit her friends for dinner. Yet the laundry had not been removed. Meredith was probably still wearing her zippered sweatshirt when she was attacked, the garment she wore when she was walking home on a brisk evening. Meredith unsuccessfully tried to call her ailing mother around 8:56 PM but did not attempt to reach her again. Atypically, Meredith did not send any text messages that evening. There were two calls around 10 PM that the Massei motivation report ascribes dubiously to Meredith playing with her cell phone. These activities may be evidence of her attacker trying to turn the phones off. A different cell tower sent a MMS to her phone at 10:13 PM; this falls short of proof that the cell phone had left her flat by this time, but it is consistent with the phone’s being transported at this time.

Meredith’s stomach content and the lack of any material in the duodenum are difficult to reconcile with the time of her last meal (of pizza, then apple crumble), which started around 6:30 PM or earlier. Her friends watched a movie after eating pizza, and they stopped to eat apple crumble. Raffaele’s appeal argues that Meredith’s stomach contents indicate a much earlier time of death, by roughly two hours, to about 9:30 or so. Some argue that TOD cannot be accurately determined by stomach contents alone, but that argument can be taken too far: it is known that Meredith was alive until at least 9 PM. In lieu of a lengthy discussion of physiology, let us take a simple path. Patients are advised not to eat for six to eight hours prior to surgery involving intravenous anesthesia. This avoids the possibility that the patient will vomit and aspirate stomach contents into the lungs. Why would anesthesiology guidelines suggest six hours for a small meal if the stomach remained full after 5 to 5.5 hours?

If her death occurred before 10 PM, then it is somewhere between unlikely and impossible that Amanda and Raffaele are guilty. They were known to be acting normally as of about 8:45 PM, when Ms. Popovic came over. Raffaele’s appeal argues that there was some computer activity long after this time, and the police who examined his computer may have not been experienced with MacIntosh products enough to know where to look. Even if the undisputed computer activity ended around 9:10 PM, it is extremely difficult to see how the two of them had time to get so messed up that they lost control enough to murder someone before 10 PM.

Final thoughts
The appeals process has begun with an examination of the DNA forensics of the kitchen knife and the bra clasp. If the computer evidence is reexamined and the stomach contents are reevaluated, Amanda Knox and Raffaele Sollecito have a reasonable chance of exoneration as a result of their appeal.

Wednesday, June 16, 2010

DNA contamination and DNA cold hits

Part XVII in the Knox/Sollecito case

This post returns to the problem of forensic DNA contamination. It also addresses the problem of partial profiles and adventitious matches, especially with respect to cold cases. North Carolinians might want to take special note of the John Puckett and Raymond Easton cases, inasmuch as North Carolina may join other states in demanding DNA samples upon arrest. My prediction is that as the size of a DNA database grows, we will see a greater number of cold case convictions that turn out to be erroneous.

Murder of Jane Mixer
We have previously encountered the conviction of Gary Leiterman of the murder of Jane Mixer. The presence of both Mr. Leiterman’s DNA and that of John Ruelas. The prosecution was unable to come up with a reasonable explanation for Mr. Ruelas’ DNA; he was four years old at the time and lived in another city.

Dr. Theodore Kessis reexamined the DNA forensics of this case:
“Review of the electropherograms associated with this negative control sample (NEG 041902) reveals that it was contaminated, a fact that cannot be disputed since Dr. Milligan himself labeled it with a note indicating as much (Appendix 8 - Electropherogram sample NEG 041902).
Remarkably, Dr. Milligan stated in his 7/15/02 testimony that no contamination events had occurred during the course of his testing and that if any had, he would have documented them in his reports (pg. 141-21 and 142-4). Equally difficult to rectify here is the fact that when asked if he had ever committed an error, Dr. Milligan's replied he could never recall making one. (pg. 162, lines 1-5)”

While agreeing with others that most contamination events happen when two samples are handled closely in space and time, Dr. Kessis also offered a caution, “It must be noted however that contamination errors have been documented where no direct processing link between sample and contaminant have been established, raising the specter that a source of contamination can linger in a laboratory for some time.”

Dr. Kessis found another deficiency: “As previously mentioned, Corrective Action files and Error logs form an important part of a laboratory's quality control and assurance programs by centralizing the documentation of errors within the laboratory. Mandated by the DAB Standards (Appendix 2, section 14.1.1), such data allows laboratory management to quickly and efficiently identify and address systematic problems that affect the reliability of testing results.”

“Given the thousands of samples, hundreds of cases, and number of people handling casework within this laboratory, the lack of a centralize system to track errors and discrepancy indicates that the lab had no mechanism in place to identify and deal with systematic problems such as contamination events.”

There are several take-home messages for students of the Knox/Sollecito case here. Contamination by more than one individual’s DNA is possible. The laboratories in question will sometimes claim to be error-free, as did the Rome lab. In such an instance it would be helpful to check the laboratory’s corrective action files, which are more likely to be accurate than someone’s memory. Whether the Rome lab has such files is unclear. Finally, when DNA results force one to an absurd conclusion (as with the kitchen knife), contamination or misinterpretation should be suspected.

The case of Gregory Turner
Gregory Turner was indicted and jailed for a murder when the DNA from a murdered woman in Newfoundland was found on his wedding ring. The ring also contained DNA from one other contributor, thought to be his accomplice. However, the lawyer sought out the names of the technicians from the lab and determined that the technician working on his ring also worked on the victim’s fingernails. Moreover, the DNA from the second contributor to the ring turned out to be the profile of the technician herself. After many months, Mr. Turner was released from jail. This particular case of DNA contamination is an example of where both primary and secondary transfer of DNA occurred. As did the Mixer case, it shows that contamination can transfer the DNA from more than one person onto an item of evidence. This is worth bearing in mind given the multiple DNA contributors to the bra clasp in the Knox/Sollecito case.

Profile N
As was true in the Jaidyn Leskie case, the victim of one crime can become a suspect in other crimes on the basis of contaminated DNA evidence. Sometimes it takes a major foul-up to prod a laboratory into corrective action. In New Zealand, the Wellington police were investigating two murders, dubbed Rex and Pad, as well as N, the victim of an assault.

Michael Strutt reported, “One caught up in the testing was a mild mannered Christchurch man who had been assaulted outside his local, the Hagley Arms Hotel, on 23 April 1998. Police had taken blood in order to eliminate his DNA from samples taken from the scene of the assault and sent it to ESR for testing. The result of this test came to be dubbed 'Profile N' by the inquiries which were to follow… The Christchurch assault victim was subjected to 'extensive police inquiries' for more than three months and his financial records were seized. This established what he had claimed all along, that he had not left Christchurch around the time of the murders. In fact he hardly left Christchurch at all… Although they never discovered exactly how the mistakes had happened they did determine that the 'Rex' and 'Pad' samples had been accidentally contaminated with DNA from Profile N at an early stage of processing at ESR's Mt Albert laboratories. Extracts from them sent to other laboratories for testing also returned Profile N. Numerous recommendations for improving oversight, record keeping and even laboratory ventilation were made.”

“During the inquiry ESR [Environmental Science & Research Ltd] was asked to check the results of all testing done in July 1998 for any further instances of Profile N. Although they initially reported that none were found, later retests showed a partial profile consistent with N in another sample from 'Pad'. They also revealed 25 other 'unexplained' profiles in 14 of the 36 samples tested that month.”

A group investigating these cases asked a manager of the forensic unit of the ESR lab to comment, “The Wellington murders you refer to were not DNA failures. The problem was picked up and investigated. Five separate investigations failed to find scientific proof for the cause of the anomalies. No direct evidence of either accidental or deliberate contamination has come to light. Existing facilities and protocols were shown to meet independent quality audit requirements. The Ministerial Inquiry recognising the points above concluded that on the balance of probabilities the apparently anomalous results could be ascribed to accidental contamination of DNA extracts within the ESR laboratory. They made a number of recommendations which ESR has either implemented or is planning to implement. This sits comfortably with our continuous improvement management focus.”

The fact that the lab in question met the audit requirements is evidence that simply following accepted protocols does not guarantee that contamination will never happen. The lack of the determination of an exact mechanism of contamination is consistent with Dr. Kessis’s caution in the Mixer case above. People who argue that Amanda Knox and Raffaele Sollecito are guilty sometimes demand a mechanism of contamination; unfortunately, this cannot always be determined.

The first false cold hit case in the UK
Michael Strutt wrote, “A man [Raymond Easton] with advanced Parkinsons disease who could not drive an automobile or dress himself unaided was linked to a burglary which had occurred 200 miles from his home. In spite of protestations of innocence and alibi evidence police arrested him because the DNA profiles matched and 'so it had to be him'. It was several months before10-point DNA tests were done on samples from the suspect and the crime scene. The results exonerated him.

He gained his freedom and a brief note from the prosecutor saying that charges were being dropped because ‘there was not enough evidence to provide a realistic chance of conviction’. He still awaits an official apology. Or even an admission of error.”

The reason why Mr. Easton was initially suspected was an adventitious match, as opposed to contamination. This case shows that adventitious matches can happen especially for partial profiles. The first profile had only six loci, but the second profile had 10 loci. As with DNA contamination, when a cold hit match leads to an absurd result, the DNA evidence needs to be reconsidered.

The Sylvester murder, a cold hit combined with a partial profile
According to Dr. Donald Riley, “A partial profile is one in which not all of the loci targeted show up in the sample. For example, if 13 loci were targeted, and only 9 could be reported, that would be termed, a partial profile…Over-interpretation of partial profiles can probably lead to serious mistakes. Such mistakes could include false inclusions and false exclusions, alike.” Let us examine the rape and murder of a young nurse which ultimately yielded a partial profile of the assailant many years later.

Diana Sylvester was raped and murdered in San Francisco in 1972. After evidence in this case was subjected to forensic DNA analysis, a trawl through California’s DNA database identified a suspect over thirty years later, a cold hit. Of the investigations treated here, the Sylvester murder is the one in which the particulars of the investigation (a cold hit) and the differences in expert opinion on the correct statistical calculation have the least relevance to the Knox/Sollecito case. However, several more subtle points in common can be found.

In this case the suspect was identified on the basis of his DNA matching a partial profile from evidence taken from the victim. This suspect, John Puckett was in the DNA database because of his convictions for rape about five years after Ms. Sylvester was raped and murdered. However, the MO of the crime was consistent with another rapist, Robert Baker, who matched the physical description of the assailant better than Mr. Puckett and who was deceased by the time the case was reopened. The defense was barred from introducing information about Mr. Baker, the original suspect. Over the objections of the defense, the prosecution was allowed to bring in Mr. Puckett’s past convictions. How strong was the DNA evidence against Mr. Puckett?

The main issue in this case is a disagreement among experts over the correct statistical calculation to use in cold hit cases. Another key element in this case is that it made use of a partial, not full, DNA profile. In the United States a full DNA profile has thirteen separate markers (loci). A profile must have seven markers for it to be searched in California’s database. However, only 5 and a half markers were clearly found in the Sylvester murder. The jury was told that the chances of a random person’s matching the DNA found at the crime scene were 1.1 million to one. Yet when the odds were calculated with a different set of statistical assumptions the odds were only one in three, a statistic that the jury was barred from hearing. This is why the odds were only 1.1 million to one, and not substantially higher. However, when one uses a model that takes into account that there were 338,000 profiles in the database, one arrives at the 1-in-3 odds. The question of which model is better is a difficult one, yet it is odd that California courts have taken it upon themselves to decide which model is more appropriate. And the difference between one in 1.1 million and 1 in 3 is huge. It would have probably been enough to move at least one juror from guilty to not guilty beyond a reasonable doubt.

A second issue concerns the number of loci in profile itself. The peaks that were needed to flesh out the profile from 5.5 loci to 8 loci fell below the laboratory’s intensity threshold, but forensic technician Bonnie Cheng wished to interpret them anyway. DNA profiles contain artifact peaks, some of which are known as blobs and others as stutters. Deciding which peaks are true and which are artifacts may contain an element of subjectivity. The prosecution’s outside DNA witness, Dr. Chakraborty used Mr. Puckett’s full profile to interpret the partial profile, a practice that is open to serious question.

A third issue is that the sample was mixed, containing DNA markers from both the victim and assailant. The question of which markers belong to which profiles naturally arises. DNA forensic expert Peter Gill said, “If you show 10 colleagues a mixture, you will probably end up with 10 different answers.” With reference to this case Dan Krane made an important point, “There is a public perception that DNA profiles are black and white. The reality is that easily in half of all cases—namely, those where the samples are mixed or degraded—there is the potential for subjectivity.”

A fourth issue is that the lab used up so much of the sample that the defense could not do its own testing. This is a debatable practice in a DNA case; some guidelines indicate that this should not be done unless the defense agrees in advance.

One lesson for students of the Knox/Sollecito case is that the judge’s decisions might control the outcome of a case. The judge’s decision to exclude one statistical model in the Puckett case can be compared with Judge Massei’s decision not to order an independent review of the forensics of the Knox/Sollecito case. A second message is that DNA profiling is not immune to cognitive bias. In both the Puckett case and the Knox/Sollecito case, the forensic technicians lowered the bar with respect to the intensity threshold. A third fact to bear in mind is that Meredith’s DNA profile from the kitchen knife was a partial profile. However, how many loci should be counted for this piece of evidence depends entirely upon one’s choice of intensity threshold. Fourth, the bra clasp is a mixed sample, and interpreting mixtures contains an element of subjectivity.

Some possible reforms

Australian criminal defense barrister Peter Faris wrote in response to the Benjamin Forbes case, “Faced with a figure of 10 billion, a jury will always convict. The only way that it can be defended is for the defence to demonstrate a doubt arising from contamination, forensic or statistical procedures. In the real world, defence lawyers and legal aid bodies are just not equipped to investigate these matters. Many cases rely (to a greater or lesser extent) on DNA evidence. In some of them, such as Forbes, there is simply no other evidence. Consequently, if there is contamination at the scene, in police or laboratory storage or in the laboratory itself, the defence can never prove it.

Mr. Faris proposed some reforms:
“DNA should never be admitted unless there is corroboration: that is, there must be some other evidence of the identity of the offender. As for the rest of DNA cases, the solution is to put the burden on the prosecution. This can be done by amending the various Evidence Acts around the country to make DNA evidence inadmissible unless the trial judge is first satisfied, by the prosecution, that it should be admitted. The prosecution should apply to the trial judge, in a preliminary hearing, to admit DNA evidence. They must satisfy the judge, on the balance of probabilities, of various matters. The important matters are that the burden of admissibility is upon the crown. No DNA evidence could be led unless this procedure has been completed.
Conditions could be imposed in the legislation to protect the accused. For example, the prosecution would be required to produce evidence to disprove contamination, whether at the scene or in the laboratory. Strict proof of continuity of exhibits would be required. The prosecution would have to call evidence of the current international statistical procedures. Proof by the defence of international articles and learned writings could be relaxed.”

Final Thoughts
We have examined the problems with clerical errors, contamination, statistical manipulation, and outright fraud in this and preceding posts. It must also be borne in mind that the presence of DNA on an object rarely provides information on the time or means (primary, secondary, etc.) of its deposition. Paradoxically, DNA profiling is the branch of forensics with the firmest grounding in science. Much evidence exists to the effect that it is held in particularly high esteem by juries (a possible subject for a future post).

Taking all of these issues into account, I would not convict someone in a case where the sole piece of evidence comes from DNA forensics, but I would give it considerable weight, provided that the data were obtained legitimately and presented to the court in a transparent manner that respected the discovery rights of the defense. The worthy question of whether or not states should demand DNA samples upon felony arrest is outside the scope of this post.

Wednesday, June 2, 2010

How does the Patricia Stallings case shed light on the murder of Meredith Kercher?

Part XVI in the Knox/Sollecito case

Patricia Stallings was convicted, then exonerated of poisoning her infant son Ryan with ethylene glycol, the major component of antifreeze. When Ryan was unable to keep food down, Mrs. Stallings took Ryan to Cardinal Glennon Children’s hospital accidentally, which happened to be associated with a toxicology unit. Ryan had metabolic acidosis (the pH of his blood was too low) and the laboratory associated with St. Louis University reported 180 milligrams per liter of ethylene glycol.

Ryan was put into foster care, but Mr. and Mrs. Stallings were allowed weekly visits. Four days after one such unsupervised visit, during which Mrs. Stallings gave Ryan a bottle, Ryan became ill again, having metabolic acidosis and died three days later. The same lab reported finding 911 milligrams per liter of ethylene glycol in Ryan’s serum and also in his baby bottle, which had been put into a dishwashing machine.

What are the odds of the tests being wrong?
There were three separate determinations of ethylene glycol, the original serum, the serum after the second feeding, and the bottle itself. In addition, one or both of the serum samples was also retested by a commercial lab. How could this be just chance? The prosecutor told the jury not to worry about motive, but just to concentrate on the fact that there was no rational explanation for these results except deliberate poisoning. He implied in so many words how unlikely it was for the labs to be wrong.

An alternative explanation begins to emerge
Patricia Stallings was arrested after a half-empty bottle of antifreeze was found in the couple’s basement. Mrs. Stallings was pregnant with her second son at the time of her arrest; her son David was immediately placed in foster care. David began to exhibit the same symptoms as Ryan, but he was taken to a children’s hospital. The hospital diagnosed David with a genetic disease known as methylmalonic acidemia (MMA)*. In this disease, one enzyme in a particular biological pathway is defective, causing a metabolic traffic jam, in which other compounds build up in the bloodstream. Although the odds of any one infant having this disease are about 1 in 48,000, David’s diagnosis meant that there was a one in four chance that Ryan had the disease as well. This fact was not allowed to be presented at her trial however, possibly because the judge and the defense attorney misunderstood each other with respect to what the defense attorney needed to do in order to bring the issue up.

Professor of biochemistry William Sly was one of the citizen-scientists who became involved in the case. Molecular geneticist Piero Rinaldo also reexamined the evidence, with some surprising results. James Shoemaker was involved in gathering evidence prior to the trial and subsequently. Both William Sly and Robert Ritter, Mrs. Stallings’ second attorney, learned of the case from the television show "Unsolved Mysteries.”

How could the analyses of ethylene glycol have been done better?
The gas chromatography (GC) technique measures how much time it takes for substances to pass through a column of stationary liquid (a waxlike substance) over which inert gas flows. This period is the retention time of a substance, and this length of time depends on its chemical properties. Two compounds may have the same retention time but usually do not.

The first analysis of ethylene glycol showed that the unknown compound in Ryan’s serum had a retention time within 30 seconds of a standard of ethylene glycol. A difference of 30 seconds is larger than is typically seen for an identical compound run twice, but let us assume generously that 30 seconds is within experimental error between one GC run and the next. The investigators should not have concluded that they had identified ethylene glycol even under this assumption, rather they should have performed additional tests.

One way to distinguish between two compounds that have identical or almost identical retention times is to run each compound through a mass spectrometer as it emerges from the end of the column. The mass spectrometer measures how heavy a compound is and in some cases how heavy the fragments of the compound are when the compound disintegrates. The combined technique gas chromatography-mass spectrometry (GC/MS) is usually very discriminating between one compound and the next, but it was just coming into widespread use about the time of the Stallings case in 1989.

A second way to distinguish two compounds is to run the gas chromatography experiment three times with three chemically distinct stationary phases. If the retention times of the two compounds, the unknown and the standard, are the same in all three trials, then the two compounds are probably identical. A third way is to spike a portion of the sample with the unknown compound with the reference. This eliminates the factors that cause run-to-run variation in retention times. The labs in question apparently did not do any of these three additional tests.

Shoemaker, Sly, and their coworkers examined a stored sample of Ryan’s blood using gas chromatography/mass spectrometry and showed that the substance believed to be ethylene glycol was actually propionic acid, one of the substances that would be expected to be elevated if Ryan had the same genetic disorder as his brother David. They also spiked a sample of Ryan’s blood with ethylene glycol and showed that it did not have the same retention time as the unknown compound, now identified as propionic acid.

The analysis of Ryan’s blood after the unsupervised visit suggested a very high concentration of ethylene glycol. Yet ethylene glycol is highly water-soluble and clears the body with a half-life of 3.0 hours. The half-life is the time it takes for the concentration of a substance to go from some value x to x/2. Therefore, Ryan would have had to consume 300 liters of ethylene glycol during his unsupervised visit to account for his blood plasma concentration four days later, an absurd volume.

Dr. Rinaldo found evidence of methylmalonic acid in Ryan’s serum. Along with propionic acid, methylmalonic acid would be expected to be in elevated concentrations in a person suffering from MMA. He also reexamined the GC data from the two labs. He found that the commercial lab saw an unknown compound in Ryan’s serum and merely assumed it was ethylene glycol without even running a standard.

He was unable to find evidence of ethylene glycol in the tests that were run on the bottle, but the bottle itself had gone missing. The positive result for ethylene glycol on the baby bottle should have raised a red flag on its own. Ethylene glycol in the bottle would have been washed and rinsed away in the dishwasher. Drs. Shoemaker and Sly believed that the small amounts of ethylene glycol found in some of the samples were due to contamination.

Cognitive bias
Once some of these facts emerged, D.A. McElroy joined with Mrs. Stallings’ new lawyer in a motion of ineffective counsel. First Mrs. Stallings was granted a new trial, and later the charges were dropped.

The first analysis to show ethylene glycol was done in a toxicology laboratory, and they might have been trained to focus on poisonings, not inherited metabolic disorders. Dr. Bob Smith called this an “expectation effect,” and he later referred to the general forensic phenomenon as “investigator bias.” The commercial lab that retested on of the serum samples may have been guilty of confirmation bias. It seems likely but not a certainty that the second lab knew that ethylene glycol poisoning was suspected. In this context it is interesting that Dr. Shoemaker later sent normal plasma spiked with propionic acid to three laboratories and two of the three reported finding ethylene glycol.

The lab that first identified the unknown compound might or might not have been following the standard protocols of the day, but the protocol was insufficiently rigorous in the sense that the identification was only one piece of information, the retention time, as discussed above. However, the commercial lab was deficient by any measure. Clearly, neither lab did a thorough job of examining all information pertaining to Ryan’s serum.

Roger Koppl discussed many of the problems that beset forensic science in the United States. Some of the problems he illuminates, such as dependency bias, were not factors in the Stallings case but may have been at work in the Knox/Sollecito case. He wrote, “the typical forensic worker has psychological incentives to reach findings based on his
close psychological ties with the law enforcement community he serves.” He also treated confirmation and other cognitive biases (p. 13).

“Larry S. Miller demonstrates an excellent example of cognitive bias at play. He asked a group of 14 students trained in hair analysis, all of whom met the basic requirements for expert testimony on human hair identification in courts of law, to examine four cases each. For each student, two cases were presented the usual way: They were given two samples and told that one was from the crime scene and the other from the suspect. The other two cases were presented through a forensic lineup. The known sample from the imaginary crime scene was compared to five suspect-known hair samples. In all 56 cases, there were in reality no true matches. The first group of cases yielded an error rate of 30.8 percent; the second group an error rate of only 3.8 percent.12

Miller’s study illustrates how evidence is often presented to the forensic scientist in a bias-inducing manner. The samples are labeled as coming from the defendant or from the victim and are frequently accompanied by a synopsis of the investigation indicating the reasons that the investigators believe the suspect is guilty. This protocol cues the forensic worker to the expected or correct result.”

The Stallings case in the context of the Knox/Sollecito case
The Johnson/Hampikian open letter criticized both the DNA evidence surrounding the knife and the bra clasp. Some observers of the Knox/Sollecito case have asked what are the odds that both of the DNA results could be the result of contamination or secondary transfer. One might ask what are the odds of at least four identifications of ethylene glycol being wrong, and yet they were in Stallings case. It may be that assigning a probability to the chances of both the bra clasp and the knife being the result of contamination or secondary transfer by standard rules is misleading. When two events are independent, the probability of both occurring are the product of the probabilities of each. So if event A has a 10% probability, and event B has a 10% probability, then the probability of both is 1%. However, when some form of cognitive bias is in play, then this calculation does not apply; in other words, the two events are no longer independent.

Some other parallels can be found. In both cases the investigators seemed oblivious to the contradictory nature of their findings. The second determination of ethylene glycol in Ryan’s serum was so high as to rule out ethylene glycol poisoning four days earlier. The lack of blood on the kitchen knife should have told the forensic DNA analysts that Meredith’s DNA profile was the result of contamination. In both cases the motive for the crime was dubious. Patricia Stallings was thought to be guilty of Munchausen’s by proxy syndrome on the basis of scant evidence. Amanda Knox was accused of trying to get back at Meredith Kercher, because of tensions from rooming together.

References
Michelle Hoffman, “Scientific Sleuths Solve a Murder Mystery,” (1991) Science 253, 931.

Roger Koppl, “CSI for Real: How to Improve Forensics Science”

Dr. Bob Mead, “Accused and Convicted—a case study”

Journal of Pediatrics, The. 1992 Mar;120(3):417-21. Misidentification of propionic acid as ethylene glycol in a patient with methylmalonic acidemia. Shoemaker JD, Lynch RE, Hoffmann JW, Sly WS.

Bill Smith, “Not Guity! How the system failed Patricia Stallings”

Appendix: Methylmalonic acidemia* (MMA)
A metabolic pathway is a series of chemical conversions, in which an enzyme catalyzes each reaction. Enzymes are the products of genes, and an alteration of a gene from its normal form may produce an enzyme that does not function properly. The compound propionyl coenzyme A is the thioester of propionic acid conjugated with coenzyme A (CoA), a thiol. When propionyl coenzyme A is produced in the body, it is converted into succinyl coenzyme A in a pathway that is three steps long: Propionyl CoA carboxylase is the enzyme that changes propionyl CoA into D-methylmalonyl CoA, and this compound is converted into L-methylmalonyl CoA by methylmalonyl CoA epimerase. Finally, this compound is converted into succinyl CoA by the vitamin B12-dependent enzyme methylmalonyl CoA mutase. The gene for this last enzyme is defective in methylmalonic academia, and the inheritance pattern is autosomal recessive. When this happens, the previous compounds build up in concentration. At some point the bond linking propionic acid or methylmalonic acid and coenzyme A is hydrolyed, and the free acids are found in the bloodstream. Among the symptoms of this disease are lethargy, seizures, and vomiting. Metabolic acidosis and ketosis are also observed. Some forms respond to supplementation with vitamin B12.

References to Appendix
Healthline
Nelson, DL and Cox, MM, "Lehninger Principles of Biochemistry," 5th edition.

*update 6/3/10, 4:30 PM. Corrected several spelling errors/typos.