Speaking of Error in Forensic Science

When Continental flight 3407 crashed on approach to Buffalo Niagara airport on February 12, 2009, all 49 persons on board, and one on the ground, were killed. The National Transportation Safety Board began its investigation within hours, and ultimately determined that the crash was caused in part by pilot error, with fatigue as a contributing factor. To prevent similar accidents from happening in the future, the Federal Aviation Administration issued new work rules mandating shorter shifts and longer rest periods for all commercial pilots. 

The aviation industry is well-known for its approach to managing errors, which involves investigating root causes and establishing new practices to address them. Many other industries, such as health care and energy, have emulated this approach. But the forensic science industry has been slow to adopt a proactive approach to error management, despite the fact that in the courtroom, as much as in the air or on the operating table, lives are often at stake.

But some forensic scientists are working to modernize the industry’s approach to managing error, and many of them were present at the International Forensic Science Error Management Symposium, held last month at the National Institute of Standards and Technology (NIST).

You can find video of the speakers, and view and download their presentations, at the Symposium page.

This is the only widely-attended conference in the United States dedicated to managing forensic science errors, and if you’ve been paying attention to the headlines, you likely agree that the conference addressed a critical need.

In recent years, high visibility errors have occurred at crime labs in almost every state. These have ranged from simple mistakes, such as mislabeling evidence, to testimony that overstates the scientific evidence, to criminal acts. The latter category includes dry-labbing, which is when an examiner fraudulently claims to have performed laboratory analyses which in fact were never done. Dry-labbing is not an error—it’s a crime—but if it goes undiscovered, there is a problem with error management.

The cost of these cases, whether the result of simple mistakes or criminal malfeasance, have been incalculable. Innocent people have spent years behind bars, countless criminal cases have been thrown out due to tainted evidence, and the cost of litigation has soared.

So why has the forensic science industry been slower than other industries to adopt best practices for managing errors?

Lynn Garcia, general counsel for the Texas Forensic Science Commission, addressed this question during her plenary presentation. Hospitals and airlines have very high stress work environments, she said, but they are not, by design, part of an adversarial system. Forensic science, because of its role in the courtroom, is. When an error is discovered, defense lawyers and prosecutors may exploit them, or minimize them, to advance their case.

“That culture of fighting and not giving an inch, of trying to discredit people, makes it really hard to do the proactive work that’s needed to advance and improve forensic science,” Garcia said. When it comes to responding to error, she urged the forensic and legal communities to focus on accountability rather than blame.

Focusing on accountability requires transparency, said conference organizer Mark Stolorow, who as director of the Organization of Scientific Area Committees for Forensic Science (OSAC) leads NIST’s effort to promote standards and best practices for the industry. “You cannot get to the root cause of an error without transparency and full disclosure,” he said.

Transparency was a theme that came up repeatedly at the conference. During his presentation, Peter Stout, CEO and President of the Houston Forensic Science Center, showed a section of his agency’s website where the public can search for online reports of all quality incidents and corrective actions that occur under his watch. Transparency, he said, “has made an enormous difference in regaining the public’s trust.” It also puts the focus where it belongs—not on individual evidence examiners, but on the entire system, including a laboratory’s quality control infrastructure. 

Focusing on accountability also requires a venue where forensic experts can openly discuss errors and lessons learned. That’s one way that NIST was able to make a valuable contribution. NIST is leading an effort to strengthen forensic science, but it is not a regulatory agency, and it is not involved in actual casework. Because of that, NIST is able to provide a neutral ground where stakeholders can have a conversation about error that one attendee described as, until recently, “taboo.”

Judging by the frank presentations at this conference, that conversation is well underway.