Part 24 in the Knox/Sollecito case
It is sometimes claimed that the defense teams of Ms. Knox and Mr. Sollecito failed to show up for the DNA forensic testing in this case. Therefore, they do not know what went on and do not have cause to complain about the testing methods or lack of disclosure. This argument is false on a number of fronts. Ms. Knox’s stepfather Chris Mellas said that the defense was given only a few hours of notification, yet Rome is a considerable drive from Perugia (in addition to the defense having other responsibilities). This argument also ignores the fact that several of the experts from the defense only came aboard long after the testing was over. It is absurd to contend that they should be denied free access to the data. However, another problem with this argument is that it assumes that observing the testing is really that critical in the first place.
I posed the question of whether it would be more useful to observe the testing or to have the electronic data files to Professor Dan Krane, one of the cosigners of the open letter on the DNA forensics of this case. He responded:
“Having the electronic data for review is enormously important. Having the opportunity to witness the testing of samples is of marginal utility at best. Reviews of the underlying data for DNA tests often reveals alternative interpretations of the evidence samples, especially in circumstances were small amounts of DNA are involved and it is difficult to distinguish between signal, noise, and technical artifacts. Observing testing rarely provides any more insights than what should be possible from a review of contemporaneous notes that should be part of a lab's case file. Witnessing testing is far from a cure-all. Problems such as contamination of samples can easily arise before a sample arrives in a laboratory yet could not be detected by an expert observing the testing process itself.”
The subject of the electronic data files has been a major theme of this blog’s coverage of the Knox/Sollecito case. Every expert whom I have contacted has spoken in support of full release, and some private DNA forensic companies even have a standard form for the defense to fill out to obtain them. Another benefit of full release of all case files is that serious clerical errors are occasionally found and corrected.
In addition to forensic experts, legal experts within the United States support complete disclosure of all pertinent information, including but not limited to the electronic data files. Nothing close to full disclosure happened in this case. Bob Graham wrote, “It has also emerged that the prosecution has failed to deliver to the defence all the paperwork and documentation related to the forensic testing.” He reported on the prosecution’s response: “Deputy prosecutor Manuela Comodi brushed off the request for all forensic documentation and added: ‘They have everything they need. That is enough.’”
The American Bar Association’s Criminal Justice Section Standards on DNA evidence discusses disclosure in section 4.1:
(a) The prosecutor should be required, within a specified and reasonable time prior to trial, to make available to the defense the following information and material relating to DNA evidence:
(iii) the laboratory case file and case notes;
(iv) a curriculum vitae for each testifying expert and for each person involved in the testing;
(viii) all raw electronic data produced during testing;
(ix) reports of laboratory contamination and other laboratory problems affecting testing procedures or results relevant to the evaluation of the procedures and test results obtained in the case and corrective actions taken in response; and
Obviously the Italian courts are not bound by the ABA’s guidelines. However, the ABA guidelines are additional documentation, as if any were required, that release of the electronic data files is a near-universal norm in most nations. Ms. Comodi indicating that she has the right to decide what documentation is enough is remarkable given the unanimity of opinion on the usefulness of full disclosure among DNA forensic experts.
Showing posts with label fsa files. Show all posts
Showing posts with label fsa files. Show all posts
Friday, October 22, 2010
Monday, April 5, 2010
The utility of electronic data files in DNA forensics cases
Part XII on the Knox/Sollecito case
“Full disclosure allows a thorough independent review of results and thereby helps assure that the underlying scientific methods are strong and appropriate, and that the laboratory’s interpretations are fair and accurate. Until the electronic data are made available, I cannot say that I have performed a complete review of the underlying evidence in this matter.”
-William Thompson, Victoria State Coroner’s Inquest into Death of Jaidyn Leskie
In a previous post (http://viewfromwilmington.blogspot.com/2010/03/interim-evaluation-of-evidence-against.html) I wrote:
Why are the .fsa files important? Technical bulletin 40-035 from Chromosomal Laboratories, Inc., is a checklist of what the laboratory is expected to provide, and it states, “Copies of all data files used and created in the course of performing tests and analyzing data in this case, including .fsa files, if applicable. These files should include all data necessary to independently reanalyze the raw data.” In response to a question on this subject, Professor Dan Krane wrote, “The biggest concern that I personally have regarding this case is the refusal of the prosecution to provide the defense with a copy of the electronic data that underlies the DNA test results -- that is virtually unheard of world-wide today and it would be especially important to review that data in a case such as this which seems to involve such low level samples.” According to a knowledgeable source, “Each tested sample has its own file. The file contains the full electropherogram trace information along with other information about the testing conditions (e.g., date, time, injection time, voltage, temperature, current, the RFU threshold used by the analyst). If you have the electronic data, you can use the DNA analysis software (GeneScan & Genotyper or GeneMapper ID) to independently analyze the electronic data. That allows you to examine the results as closely as possible (zoom in on the electropherogram to evaluate low-level results) and establish the RFU threshold of your choosing.” (emphasis added)
The knowledgable source I quoted above is Jason Gilder, and he and Dan Krane are cosigners of the open letter from a total of nine forensics scientists that is critical of the evidence in this case. I would like to reexamine this issue with respect to how these files are useful. I will have more to say about circumstances of their being withheld from the defense in the future.
Contents of fsa files
The fsa files are created by the DNA analysis instruments of Applied Biosystems, and the program GeneScan can read them. These files contain the times of and dates on which samples were (runhttp://www.nfstc.org/pdi/Subject06/pdi_s06_m01_02.htm). They also contain the peak detection threshold in relative fluorescence units. Perhaps most importantly, they contain the electronic information needed to recreate a DNA electropherogram with the parameters chosen by the analyst.
Electronic data files and the Leskie case
I would like to examine this case more extensively, both for what it tells us about contamination and what it tells us about electronic data files. Jaidyn Leskie was a young boy in Australia who was murdered n 1997. DNA from an alleged rape victim, Ms. P, was found on two items of his clothing, a bib and track pants. There is general agreement that this was the result of contamination (http://viewfromwilmington.blogspot.com/2010/01/forensic-dna-contamination.html), despite the fact that contamination did not show up in the negative control (reagent blank or no-template) samples. It is helpful to know that Jaidyn’s body was submerged in water for several months before its discover, and these conditions are not at all conducive to the stability of DNA. None of Jaidyn’s own DNA was found, only Ms. P’s.
Professor Thompson provided a report for the Leskie case (quoted above). Professor Thompson wrote (http://www.bioforensics.com/articles/index.html), “When I review the casework of forensic DNA laboratories in the United States, I typically ask for and receive copies of the electronic data collected by laboratory instruments during DNA testing. These data show the actual results of the DNA tests. By examining these data, an independent expert can check whether the forensic laboratory interpreted its results correctly and can detect a host of potential problems that would not be apparent from examining the laboratory notes. Electronic data would be particularly helpful in evaluating the present cases. Beyond allowing a cross check of the analysts’ interpretations, it would allow a detailed independent assessment of whether there is evidence of low- level contamination in any samples.”
Professor Krane provided two reports; the supplemental report was written after he was given access to the electronic data files. With respect to the negative controls not showing contamination his first report state, “there is no good reason to expect that contamination would uniformly affect all samples if and when it does occur. In much the same way, Salmonella-tainted meat placed on one part of a countertop at the beginning of food preparation does not mean that all food subsequently prepared for the same (or even a later) meal will be similarly tainted – just those items that also come in contact with the same portion of the countertop can be reasonably expected to have a chance of being tainted.” In previous posts, I have noted that the amount of DNA after it has been amplified by the polymerase chanin reaction (PCR) makes it a serious potential contaminant. However, professor Krane explained why the PCR process itself was not likely to be the source of the contamination in this instance. Professors Thompson and Krane both indicate that substrate controls would have been appropriate in this case.
It is Dr. Krane’s second report that is especially pertinent to the utility of access to the electronic data. He wrote, “My subsequent review of the underlying electronic data associated with the testing performed by the VFSC during their investigation of the death of Jaidyn Leskie in particular leaves me even more convinced that the STR-DNA profile match observed between the condom of the rape investigation and the bib in the Jaidyn Leskie abduction investigation is the result of contamination.” He went on to say, “Now that I have had an opportunity to review the electronic data from the Leskie investigation I, like Professor Thompson, do indeed find low levels of signal that are consistent with Ms. P’s DNA profile and lend substantially more credence to the already compelling proposition that she is the source of the DNA associated with the Leskie bib and track pants.” Professor Krane subsequently wrote in response to one of my inquiries that the observation of these peaks “helped strengthen the case that contamination had occurred by several orders of magnitude.” (emphasis added)
Professor Krane noted that there was a systematic difference in the peak heights depending on the software one used and the analysis options one chose: “However, I thought it might also be helpful to consider the output as it was seen by the testing laboratory primarily for two reasons: 1) no-smoothing with the Macintosh versions of the software systematically makes peak heights larger; and 2) the peak heights generated in this way should be more directly comparable with the threshold established by the testing laboratory in the course of their validation studies. As Professor Thompson points out, “Most laboratories establish a peak height threshold for reporting alleles and make it a practice to ignore peaks falling below that threshold.” I, like him, feel that this practice is appropriate but that low-level signals may also contain useful information in investigations such as this one.” It is possible that peaks could fall above the threshold in the hands of one analyst and below it in the hands of another. The sizes of the peaks in the two figures in this report vary from less than one hundred RFUs to many hundreds of RFUs, documenting the fact that profiles arising from contamination can easily be mistaken for samples arising from genuine contact.
Justification for the release of electronic files in another case
An anonymous DNA forensic expert unaffiliated with the present case had this to say in a case from several years ago:
“The laboratory has provided hard copy printouts of the data that do not allow the possibility of reanalysis by an independent expert. Certain parameters have irrevocably been applied to the data; as such it represents an interpretation by the laboratory rather than the original data. Of additional concern is the fact that the documents provided by the laboratory are represented as fully complying with the discovery request.” The expert gave several areas where independent review requires the electronic data files, including.
1. Determination of the threshold. “Not all qualified experts would necessarily apply the same threshold given identical data…The possibility of false inclusions as well as false exclusions exists if real data that happen to reside below an artificial threshold are ignored.”
2. Scale of viewing. “This ability is an important element in determining whether a particular peak represents and authentic DNA fragment.”
3. Proprietary concerns. “The data itself, as the work product of a government laboratory in a criminal matter, is clearly in the public purview.”
“Honest differences of opinion by qualified experts should be welcomed by the Court. This can only be accomplished if the independent expert is provided full and complete discovery by the government.”
Electronic data files and the Knox/Sollecito case
In response to a question, Dan Krane wrote me to say, “I would like to see that data primarily for two reasons: 1) to determine the actual heights and data collection points of all the peaks (for purposes of mixture resolution and the possibility that some are associated with pull up, respectively), and 2) to calculate a limit of detection from the data in the controls so as to determine if the lab was reliably distinguishing between signal and noise in the evidence samples.”
Professor William Thompson wrote (Tarnish on the 'gold standard:' Understanding recent problems in forensic DNA testing. The Champion. 30(1):10-16, January/February 2006), “A key aspect of discovery in DNA cases is the electronic data produced by the computer-controlled genetic analyzers that are currently used to “type” DNA samples. Analysis of the computer files can not only reveal undisclosed problems and support alternative interpretations of the findings, but also, as discussed above, these files can be crucial for detecting instances of scientific fraud, such as that committed by Jacqueline Blake and Sarah Blair.” Jacqueline Blake was an employee of the FBI whose two-year record of contamination-free work was the result of dumping a portion of the negative control down the sink. Sarah Blair repeatedly substituted a file of a contamination-free blank for the actual blank.
I can think of several things I would look for using the electronic data files. The first is what peak threshold was used for the 400+ runs and whether or not it was the same threshold as the knife blade profile. The second is to zoom in on the small peaks in the bra clasp, as Jason Gilder implied. It is conceivable that the identities of the three additional people who contributed DNA to the clasp could be ascertained. The third is to examine the negative control runs for evidence of fraud, as Professor Thompson noted could be done. The negative controls are the most common place for fraud to be detected.
State of Michigan
One exception to the general rule of complete disclosure of the electronic data files was the state of Michigan (http://www.bioforensics.com/articles/Thompson_Champion_Tarnish.pdf). In 2005 the Michigan State police Issued a statement saying that, “it is the position of the Michigan State Police Forensic Science Division that any release of this (sic) data for processing with non-validated parameters is tantamount to evidence tampering.” Thompson refutes this position: “By analogy, if the police disclosed digital photos of the crime scene, it would hardly be ‘data tampering’ for a defense expert to manipulate the digital images in an effort to enhance them or bring specific details into focus…The use of proper analysis parameters might well become an issue if and when the defense decided to offer into evidence the results of its analysis of the electronic data. But to deny access to digital data on grounds that the defense might analyze it improperly eviscerates the right to discovery.” (emphasis added) I will update this post if I am able to learn whether or not the State of Michigan has changed its policy.
“Full disclosure allows a thorough independent review of results and thereby helps assure that the underlying scientific methods are strong and appropriate, and that the laboratory’s interpretations are fair and accurate. Until the electronic data are made available, I cannot say that I have performed a complete review of the underlying evidence in this matter.”
-William Thompson, Victoria State Coroner’s Inquest into Death of Jaidyn Leskie
In a previous post (http://viewfromwilmington.blogspot.com/2010/03/interim-evaluation-of-evidence-against.html) I wrote:
Why are the .fsa files important? Technical bulletin 40-035 from Chromosomal Laboratories, Inc., is a checklist of what the laboratory is expected to provide, and it states, “Copies of all data files used and created in the course of performing tests and analyzing data in this case, including .fsa files, if applicable. These files should include all data necessary to independently reanalyze the raw data.” In response to a question on this subject, Professor Dan Krane wrote, “The biggest concern that I personally have regarding this case is the refusal of the prosecution to provide the defense with a copy of the electronic data that underlies the DNA test results -- that is virtually unheard of world-wide today and it would be especially important to review that data in a case such as this which seems to involve such low level samples.” According to a knowledgeable source, “Each tested sample has its own file. The file contains the full electropherogram trace information along with other information about the testing conditions (e.g., date, time, injection time, voltage, temperature, current, the RFU threshold used by the analyst). If you have the electronic data, you can use the DNA analysis software (GeneScan & Genotyper or GeneMapper ID) to independently analyze the electronic data. That allows you to examine the results as closely as possible (zoom in on the electropherogram to evaluate low-level results) and establish the RFU threshold of your choosing.” (emphasis added)
The knowledgable source I quoted above is Jason Gilder, and he and Dan Krane are cosigners of the open letter from a total of nine forensics scientists that is critical of the evidence in this case. I would like to reexamine this issue with respect to how these files are useful. I will have more to say about circumstances of their being withheld from the defense in the future.
Contents of fsa files
The fsa files are created by the DNA analysis instruments of Applied Biosystems, and the program GeneScan can read them. These files contain the times of and dates on which samples were (runhttp://www.nfstc.org/pdi/Subject06/pdi_s06_m01_02.htm). They also contain the peak detection threshold in relative fluorescence units. Perhaps most importantly, they contain the electronic information needed to recreate a DNA electropherogram with the parameters chosen by the analyst.
Electronic data files and the Leskie case
I would like to examine this case more extensively, both for what it tells us about contamination and what it tells us about electronic data files. Jaidyn Leskie was a young boy in Australia who was murdered n 1997. DNA from an alleged rape victim, Ms. P, was found on two items of his clothing, a bib and track pants. There is general agreement that this was the result of contamination (http://viewfromwilmington.blogspot.com/2010/01/forensic-dna-contamination.html), despite the fact that contamination did not show up in the negative control (reagent blank or no-template) samples. It is helpful to know that Jaidyn’s body was submerged in water for several months before its discover, and these conditions are not at all conducive to the stability of DNA. None of Jaidyn’s own DNA was found, only Ms. P’s.
Professor Thompson provided a report for the Leskie case (quoted above). Professor Thompson wrote (http://www.bioforensics.com/articles/index.html), “When I review the casework of forensic DNA laboratories in the United States, I typically ask for and receive copies of the electronic data collected by laboratory instruments during DNA testing. These data show the actual results of the DNA tests. By examining these data, an independent expert can check whether the forensic laboratory interpreted its results correctly and can detect a host of potential problems that would not be apparent from examining the laboratory notes. Electronic data would be particularly helpful in evaluating the present cases. Beyond allowing a cross check of the analysts’ interpretations, it would allow a detailed independent assessment of whether there is evidence of low- level contamination in any samples.”
Professor Krane provided two reports; the supplemental report was written after he was given access to the electronic data files. With respect to the negative controls not showing contamination his first report state, “there is no good reason to expect that contamination would uniformly affect all samples if and when it does occur. In much the same way, Salmonella-tainted meat placed on one part of a countertop at the beginning of food preparation does not mean that all food subsequently prepared for the same (or even a later) meal will be similarly tainted – just those items that also come in contact with the same portion of the countertop can be reasonably expected to have a chance of being tainted.” In previous posts, I have noted that the amount of DNA after it has been amplified by the polymerase chanin reaction (PCR) makes it a serious potential contaminant. However, professor Krane explained why the PCR process itself was not likely to be the source of the contamination in this instance. Professors Thompson and Krane both indicate that substrate controls would have been appropriate in this case.
It is Dr. Krane’s second report that is especially pertinent to the utility of access to the electronic data. He wrote, “My subsequent review of the underlying electronic data associated with the testing performed by the VFSC during their investigation of the death of Jaidyn Leskie in particular leaves me even more convinced that the STR-DNA profile match observed between the condom of the rape investigation and the bib in the Jaidyn Leskie abduction investigation is the result of contamination.” He went on to say, “Now that I have had an opportunity to review the electronic data from the Leskie investigation I, like Professor Thompson, do indeed find low levels of signal that are consistent with Ms. P’s DNA profile and lend substantially more credence to the already compelling proposition that she is the source of the DNA associated with the Leskie bib and track pants.” Professor Krane subsequently wrote in response to one of my inquiries that the observation of these peaks “helped strengthen the case that contamination had occurred by several orders of magnitude.” (emphasis added)
Professor Krane noted that there was a systematic difference in the peak heights depending on the software one used and the analysis options one chose: “However, I thought it might also be helpful to consider the output as it was seen by the testing laboratory primarily for two reasons: 1) no-smoothing with the Macintosh versions of the software systematically makes peak heights larger; and 2) the peak heights generated in this way should be more directly comparable with the threshold established by the testing laboratory in the course of their validation studies. As Professor Thompson points out, “Most laboratories establish a peak height threshold for reporting alleles and make it a practice to ignore peaks falling below that threshold.” I, like him, feel that this practice is appropriate but that low-level signals may also contain useful information in investigations such as this one.” It is possible that peaks could fall above the threshold in the hands of one analyst and below it in the hands of another. The sizes of the peaks in the two figures in this report vary from less than one hundred RFUs to many hundreds of RFUs, documenting the fact that profiles arising from contamination can easily be mistaken for samples arising from genuine contact.
Justification for the release of electronic files in another case
An anonymous DNA forensic expert unaffiliated with the present case had this to say in a case from several years ago:
“The laboratory has provided hard copy printouts of the data that do not allow the possibility of reanalysis by an independent expert. Certain parameters have irrevocably been applied to the data; as such it represents an interpretation by the laboratory rather than the original data. Of additional concern is the fact that the documents provided by the laboratory are represented as fully complying with the discovery request.” The expert gave several areas where independent review requires the electronic data files, including.
1. Determination of the threshold. “Not all qualified experts would necessarily apply the same threshold given identical data…The possibility of false inclusions as well as false exclusions exists if real data that happen to reside below an artificial threshold are ignored.”
2. Scale of viewing. “This ability is an important element in determining whether a particular peak represents and authentic DNA fragment.”
3. Proprietary concerns. “The data itself, as the work product of a government laboratory in a criminal matter, is clearly in the public purview.”
“Honest differences of opinion by qualified experts should be welcomed by the Court. This can only be accomplished if the independent expert is provided full and complete discovery by the government.”
Electronic data files and the Knox/Sollecito case
In response to a question, Dan Krane wrote me to say, “I would like to see that data primarily for two reasons: 1) to determine the actual heights and data collection points of all the peaks (for purposes of mixture resolution and the possibility that some are associated with pull up, respectively), and 2) to calculate a limit of detection from the data in the controls so as to determine if the lab was reliably distinguishing between signal and noise in the evidence samples.”
Professor William Thompson wrote (Tarnish on the 'gold standard:' Understanding recent problems in forensic DNA testing. The Champion. 30(1):10-16, January/February 2006), “A key aspect of discovery in DNA cases is the electronic data produced by the computer-controlled genetic analyzers that are currently used to “type” DNA samples. Analysis of the computer files can not only reveal undisclosed problems and support alternative interpretations of the findings, but also, as discussed above, these files can be crucial for detecting instances of scientific fraud, such as that committed by Jacqueline Blake and Sarah Blair.” Jacqueline Blake was an employee of the FBI whose two-year record of contamination-free work was the result of dumping a portion of the negative control down the sink. Sarah Blair repeatedly substituted a file of a contamination-free blank for the actual blank.
I can think of several things I would look for using the electronic data files. The first is what peak threshold was used for the 400+ runs and whether or not it was the same threshold as the knife blade profile. The second is to zoom in on the small peaks in the bra clasp, as Jason Gilder implied. It is conceivable that the identities of the three additional people who contributed DNA to the clasp could be ascertained. The third is to examine the negative control runs for evidence of fraud, as Professor Thompson noted could be done. The negative controls are the most common place for fraud to be detected.
State of Michigan
One exception to the general rule of complete disclosure of the electronic data files was the state of Michigan (http://www.bioforensics.com/articles/Thompson_Champion_Tarnish.pdf). In 2005 the Michigan State police Issued a statement saying that, “it is the position of the Michigan State Police Forensic Science Division that any release of this (sic) data for processing with non-validated parameters is tantamount to evidence tampering.” Thompson refutes this position: “By analogy, if the police disclosed digital photos of the crime scene, it would hardly be ‘data tampering’ for a defense expert to manipulate the digital images in an effort to enhance them or bring specific details into focus…The use of proper analysis parameters might well become an issue if and when the defense decided to offer into evidence the results of its analysis of the electronic data. But to deny access to digital data on grounds that the defense might analyze it improperly eviscerates the right to discovery.” (emphasis added) I will update this post if I am able to learn whether or not the State of Michigan has changed its policy.
Subscribe to:
Posts (Atom)
