February 24, 2012 posted by Patrick DiCaprio

Ryan Braun and False Positives

Print Friendly

I am on record as saying I do not care about PED’s in baseball, and have no issue at all with athletes taking them. However, I am not going to defend Ryan Braun here, since he has not admitted taking them. The interesting thing to me is that the number of tests he and other players have taken almost guarantees a false positive by someone at some point.

According to his statement, Braun had 25 negative tests, or so I heard on MLB Network this morning. Let’s assume that there is a 99% accuracy rate, for now. Given a sample of 25 tests, how often would at least one come back as falsely positive?

The way to figure that out is to take the chances of any one test being accurate, namely 99% and multiplying it by itself for each test. Why? Because if we are trying to figure out the chances of any one being a false positive then the easiest way is to figure out the chance that every single one will be accurate.

So, in this case we take 0.99 and multiply it by itself 25 times, which is 0.99 to the 25th power. We get 77%. That is a one in three chance that Braun, or anyone else, would have a false positive over 25 tests. Over a sample of more than five or six players it is a 100% certainty that there have been false positives, at least based on the assumption that the test is 99% accurate. But this assumption way underestimates the accuracy, it was just for explanatory purposes.

At least one source has said the test is 99.99% accurate; to put it another way the false positive rate is 1 in 10,000. So what about with that probability? Well, obviously the rate is very, very low, and doing the math it is 99.97%, or a 0.3% chance of at least one false positive in Braun’s case.

But it is not so simple!

This raises the issue of Bayes’ Theorem. Bayes’ theorem deals with conditional probability; you compare the predicted results with actual results and calculate what might be called “after the fact” probability. This is a gross simplification.

One of the issues raised by Bayes’ Theorem is that when the rate of false positives is compared to the actual rates observed in the universe of samples, the probability of “accuracy” is not 99.99%. To put it another way, saying that Ryan Braun, or anyone else, had a positive test and that the test is 99.99% accurate and he is therefore 99.99% likely to be guilty is not correct.  Nor is it the case that, across the universe of players, the chances of an individual player being falsely accused is 99.99%, nor is it 99.97%.

And yet, this is exactly what MLB (and prosecutors relying on DNA evidence as another example) want the public to believe. They deliberately mislead the public about what the math actually means.

Forgetting Ryan Braun specifically, assuming that any individual player has a 0.3% chance of a false positive over 25 tests, and assuming that the number of veterans is such that the average veteran has taken at least 25 tests what are the chances of at least one false positive among the sample of players? To know that we would need to know information that we do not, such as the number of tests. But what we can say is that the chances are likely very close to 100% as long as the number of tests and veterans is high. And we can say with actual certainty that the chance of Ryan Braun being falsely accused is not one in 10,000.

Of course we don’t know about appeals, and how many test results are overturned. Confounding the problem is this secrecy. For one to figure out the actual chances of a player being innocent when testing positive, we would need to know the observed rate. But we will never know, and perhaps the player’s union would win more often in the court of public opinion if the results were not confidential. Or maybe not; that is a topic for another day.

You can tweak the numbers, and to be fair I am essentially in the dark as to the accuracy and the observed rate among players. But what I can say with relative certainty is that the odds of at least one individual player, no matter how accurate the test, getting a false positive is virtually 100%, even if the odds of a specific player being falsely positive is relatively low.


You must be logged in to post a comment.