DNA is often called the “gold standard” of forensic science. But not all DNA evidence is the same. Like if there’s a pool of blood at a crime scene and you need to figure out whose blood it is. That’s a pretty simple DNA analysis. But when you’re dealing with very very small amounts of DNA, like just a few skin cells left behind, Or a mixture of a bunch of people’s DNA, then the analysis is more complicated. So, it used to be that if DNA found at a crime scene was too small or too mixed, labs would say the evidence was “inconclusive.” But the New York City Crime Lab invented high-tech methods to try to solve this problem. Let’s start with a scenario. John is driving in a car and he’s pulled over by a cop for a minor traffic violation. When the cop is writing up his ticket, he notices a gun in the backseat of the car. So, the cop ends up arresting John for gun possession. But John says the gun isn’t his. In fact, he says he didn’t even know the gun was in the car. So the police officer might send that gun to the lab for testing. Let’s say the lab found a small mixture of DNA on the gun and they want to see if John’s DNA is in there somewhere. For this situation, the lab created a software called the “Forensic Statistical Tool”—or FST—to analyze the evidence. FST compares the DNA on the gun and the DNA from John and it generates something called a likelihood ratio Now, it doesn’t tell us if John’s DNA is actually a match to the DNA on the gun. It’s complicated, but—basically, the likelihood ratio is a number that describes how probable it is that John’s DNA might be in the mix. So the likelihood ratio might be 1,000. Which means it is 1,000 times more likely. As you can imagine, this can have a huge impact on criminal cases. John might hear that big number and feel pressured to take a plea deal. A jury might hear that number, not really understand what it means, and think that John’s DNA has actually been found on the gun. And now, he’s guilty. But there are some serious problems with this “likelihood ratio.” It’s really hard to tell how many people’s DNA is actually in a mixture. What if the DNA from the gun in John’s case looks like it’s from 3 people, but it’s actually from 5? If you get that number wrong, you throw off the whole calculation. It’s also really hard to distinguish between close family members who share a lot of DNA So, what if John was driving a car he shares with his family? This could be John’s brother’s gun, But now it’s pinned to John. And here’s the other thing: No one knew how this software actually worked until recently when a federal judge in New York made the lab hand over the software’s code for the first time. Digging into that code, a defense expert found a lot of questionable math that might have had real world consequences. Meanwhile, the New York City crime lab no longer uses this software as of this year. Lawyers in New York City are now asking the state inspector general to investigate. So what will happen to the thousands of criminal cases that have already been decided with the help of this disputed evidence? I don’t know yet. But when I do, I’ll let you know. I’m Lauren Kirchner and I report on criminal justice and technology. Check out ProPublica.org for updates on this story and more of our series on machine bias.