The Analytics Engine: What’s the probability that a breathalyzer could be wrong?

By Kevin Silberberg

Some evidence may be weaker than you think

You’re in the passenger seat while your co-worker is driving you home from work on a sunny Tuesday afternoon. Suddenly, traffic slows to a crawl. After five minutes of stop-and-go you spot the problem: a police checkpoint is making every driver pass a breathalyzer test. You finally make your way to the front of the line. Your friend blows into the breathalyzer… they fail. They swear they haven’t touched any alcohol, but the next thing you know they’re in handcuffs.

What’s the probability that the breathalyzer could be wrong? In this situation, it’s much higher than you think.

You don’t know your co-worker that well but you’re skeptical that they would be drunk on a workday. Let’s assume that only one in a thousand drivers are drunk on a Tuesday afternoon.

However, as the police officer is arresting your co-worker, he informs you that the breathalyzer test incorrectly classifies a sober person as drunk only 3% of the time, and will always correctly classify a drunk person as drunk.

Instinctively, we tend to overvalue the impact of this new evidence. Due to the breathalyzer’s 3% false positive rate, many might give estimates as high as 97% for the probability that the co-worker is drunk. In reality, the probability is much lower.

Given what you know, we can assume that if 1,000 random vehicles are stopped, on average:

• One driver would be drunk while 999 would be sober, since one in 1,000 drivers are drunk in general
• The one drunk driver would fail the breathalyzer
• Thirty of the 999 sober drivers would falsely fail the breathalyzer due to the 3% false positive rate

Therefore, when randomly breathalyzing drivers, for every 31 drivers that fail, only one of them will actually be drunk. Thus, the actual probability that your co-worker is drunk, given that they failed the breathalyzer, is still only 1/31 or 3.2%. In other words, there is a 96.8% chance that the breathalyzer was wrong.

In this situation, we can say that despite the breathalyzer evidence, your co-worker was likely sober. We overvalue the breathalyzer test because we ignore how unlikely it was that a random driver was drunk in the first place on a Tuesday afternoon.

Virtually every company relies on statistics to make better decisions; this situation is just one example of how unintuitive statistics can be. At SATOV, we pride ourselves in our analytical processes, basing our insights at the intersection of deep statistical rigour and business strategy.

This is my second instalment in my statistically focused article series. Stay tuned for my next article as I explore how to better leverage data and analytical tools for decision making so you don’t fall into the same trap the breathalyzer example shows. If you missed my first instalment, you can find it here.