Tuesday, September 29, 2015

Short shots on forensic error

Grits ran across a site with an excellent list of press clippings about crime lab problems and found several items I hadn't seen before.

Interviewing Brady Mills
Here's an interview with Brady Mills, the man in charge of all Texas DPS crime labs, from Forensic magazine in May on the topic of confronting flawed forensic science. Grits learned from the article that Mills is past president of ASCLD-LAB, a crime-lab accrediting body, the Association of State Crime Lab Directors and sits on their board. Note to Brady: You have a standing invitation to come do an interview/podcast on Grits. We could find a lot to talk about.

Rapid DNA and 'black box' software
In that interview, Mills discussed "rapid DNA," which refers to systems where a machine analyzes a swab and spits out a result without human interpretation. The other day, some of the nation's leading experts cautioned against using "black box" software systems alone to analyze DNA, insisting that human interpretation was required to understand where certain assumptions may lead to error. (Once assumptions become embedded in code, they become invisible and thus harder to detect and correct.) Rapid DNA sounds like it's headed in the black-box direction, perhaps problematically automating judgments that require more nuance than code can afford. That's certainly a concern on mixture cases, given what we now know about them. So far, no Rapid DNA system has been validated for use in forensic labs, but several companies are trying.

Indulging schadenfreude, or, we're screwed up but we're not Massachussetts
The Annie Dookhan fiasco in Massachussetts, where a crime lab worker intentionally framed defendants through faked forensic results, depicts the worst case scenario for a crime lab, with thousands of defendants potentially eligible to have their convictions overturned. Now, it turns out another forensic chemist in Massachussetts stole drugs and replaced them with counterfeit, potentially calling into question thousands more cases. These episodes call to mind the Jonathan Salvador mess in Texas, but that was on a much smaller scale. He handled about 5,000 cases and only a quarter to half of those were called into question. As that debacle demonstrated, though, just because hundreds or even thousands of people are eligible for relief doesn't mean that many will eventually get it. The systems for notifying defendants and getting them lawyers just aren't there.

Not West Virginia, either
In the New York Times Magazine this week, Emily Bazelon previewed a case at the West Virginia Supreme Court which will determine whether the state is obligated to turn over exculpatory DNA evidence after a defendant has entered a plea. Under Texas statutes and case law, the answer would clearly be "yes," there's an obligation to turn it over under the Michael Morton Act. Moreover, a guilty plea would not preclude a future innocence claim (Ex Parte Tuley). But Bazelon mentioned that, "The U.S. Court of Appeals for the Fifth Circuit [which includes Texas] has repeatedly ruled that 'a guilty plea waives the right' to claim that your right to exculpatory evidence has been violated." So maybe it's not perfect, but thank God for the Michael Morton Act, and in the case of Ex Parte Tuley, the Texas Court of Criminal Appeals!

Might flawed field tests contribute to Harris County drug exonerations?
Texas has witnessed dozens of cases out of Harris County where defendants are exonerated months or years after they plead guilty because a crime lab determines the substance they were charged with possessing wasn't drugs. Nobody knows for sure why this is happening, except that people plead guilty - often to time-served or probationary sentences - because they can't afford bail to get out of jail. But why are so many falsely accused in the first place? A theory presents itself from a couple of stories about erroneous field test results. One test can confuse soap or candy for drugs, while in Australia, a man was jailed when epsom salts were confused for meth. I wonder if faulty field test results help explain the rash of Harris County drug-case exonerations? They're up to more than 80 so far and insiders tell Grits to expect dozens or even hundreds more before the situation is resolved.


Anonymous said...

For the layman and Defense Attorneys...
"Forensics, Lawyers, and Method Validation — Surprising Knowledge Gaps"
By Dr. Kevin Schug, UT Arlington


The article is mostly about GC-MS (toxicology), but it applies to EVERY SINGLE TEST IN THE CRIME LAB! DNA analysis too (especially)!

If you want to freak out the Expert for the Prosecution during testimony, start asking about method validation, accuracy, precision, LOD, LOQ, etc. Most lab technicians don't know the definition differences between these words.

Anonymous said...

Hey Anny, you should really consider promoting yourself as an expert to the 3% +/- of real CDLs that actually take criminal cases all the way to trial (all the way). You could make bank poking holes in bullshit testimony. Think about it.

Anonymous said...

Grits, Rapid DNA and 'black box' software could be jacked with like VW did and those igits got away with it for a very long time. I'm having a hard time believing that out of the thousands of shops performing the so-called - Inspection, not one single mofo (tech or owner or customer) noticed that a smoking VW Passed the friggin Inspection.

In order for the scam to have worked for so long, VW had to have had a Bag-Man on call to intervene and grease a few greasy palms. Once the DPS gets the green light, all the prosecution has to do is blame false arrests and the convictions on software and poof humans hands are considered clean. The taxpayers become the Bag-Man on the back end and bribe the wronged to go away. With that, we need to know the names of everyone that participates in the creation, set up & maintenance of Black-Boxes. I hope you get the opportunity to ask him if he can assure the 'Public' of 100% transparency. For the sake of the PC & race-baiting crowd, let's just call them Computers to avoid any color wars. Cya.

Anonymous said...

Field tests for drugs are by their nature presumptive screening tests, with different false positive and false negative error rates than confirmatory laboratory tests. So, to talk about the field tests as "flawed" isn't correct. It would be fair to compare different field tests in their accuracy and specificity, and conclude that one was better than another. But the fact that a certain percentage of field tests are not confirmed in subsequent confirmatory lab tests is not particularly noteworthy. It is expected and predictable science. It is up to the legal system to determine if it wants to use that presumptive information, with its known limitations, for decision making purposes.

Anonymous said...

I also wouldn't expect a technician to be fluent in the theory of validation, LOD, LOQ, etc. Technicians are just that - technicians who run wet procedures and instruments, and process data using standard protocols They don't design and optimize processes and procedures. For that sort of information you would want a higher level scientist, probably a Ph.D.