Bayesian probability is a powerful framework for reasoning under uncertainty, offering a unique perspective on how we can update our beliefs in light of new information. At its core, it employs a simple yet elegant principle: the probability of an event is not set in stone, but rather a dynamic, evolving concept that adapts as we gain more knowledge. Traditional probability, like rolling a dice, provides fixed odds for each outcome. In contrast, Bayesian probability equips you with the tools to adjust your beliefs as you uncover new clues. As you gather evidence, you can rationally and incrementally refine your hypothesis, shifting the probabilities like pieces of a puzzle until the final, compelling picture emerges. This adaptive, data-driven approach to probability makes Bayesian reasoning a fascinating and indispensable tool in various fields, from machine learning to medical diagnosis, and even the realm of philosophical inquiry.

In 1763, Reverend Thomas Bayes released his seminal publication An Essay towards Resolving a Problem in the Theory of Probability. Within this work, he introduced his renowned theorem, known as Bayes’ theorem, which stated the following for two events and :

Let’s think about the recent pandemic – when it comes to COVID-19 testing, two crucial concepts that play a pivotal role in determining the accuracy and reliability of a test are specificity and sensitivity.

Specificity refers to a test’s ability to correctly identify individuals who do not have the virus. In other words, it measures the test’s capacity to produce a negative result for those who are genuinely not infected. A highly specific test will have a low rate of false positives, minimizing the chances of mistakenly categorizing healthy individuals as infected. This is especially important for public health measures, as an overly non-specific test could lead to unnecessary quarantines and economic disruptions.

On the other hand, sensitivity gauges a test’s capability to correctly detect individuals who are infected with the virus. It assesses the test’s ability to provide a positive result for those who are truly carrying the virus. A highly sensitive test will have a low rate of false negatives, ensuring that infected individuals are correctly identified and isolated, thus preventing the potential spread of the virus.

Balancing specificity and sensitivity in COVID-19 tests is a significant challenge. Increasing sensitivity might lead to a rise in false positives, while enhancing specificity may increase false negatives. Public health authorities, healthcare professionals, and researchers must strike a careful balance to choose the right tests based on the specific context, prevalence of the virus, and the goals of the testing program, whether it’s to screen for infection, diagnose symptomatic individuals, or trace contacts. Achieving an optimal balance between these two factors is vital for effective COVID-19 testing and managing the pandemic.

To better visualize this, we can draw a grid as shown below :

This table illustrates how a test result relates to whether you actually have the virus. It reveals areas where errors can occur, and addressing these errors is crucial. We can use specificity and sensitivity percentages to estimate the probability of having the virus (or not) based on the test results. Specificity deals with the likelihood of Type-I errors, while sensitivity is linked to Type-II errors. In simpler terms, specificity helps us understand the chance of a false positive (incorrectly saying you have the virus when you don’t), and sensitivity tells us the likelihood of a false negative (incorrectly saying you don’t have the virus when you do). Finding the right balance between these factors is essential for accurate testing.

For example, let’s assume that in a population of 1000, 700 residents tested positive – so intuitively, you might think that the actual amount of residents affected would be very close to 700. However, the test used has 99% specificity and 85% sensitivity – using this, we can approximate the actual proportion of residents infected :

Type-I Error :

Type-II Error :

Actual proportion of residents infected :

As we can see, the actual proportion of infected is more than what the test suggests (a increase). This may not seem significant at first, however can quickly pose problems in a large population. This is one of the problems why Bayesian probability is so confusing for the majority of the public – it is much counterintuitive and requires a conceptual understanding of how probability works for the following factors :

1. Subjectivity: Bayesian probability incorporates prior beliefs or subjective information when making predictions or inferences. This can be challenging for many to accept, as it deviates from the frequentist approach, which relies on purely objective data. For some, the idea that probabilities can be influenced by personal beliefs can seem perplexing.
2. Complexity: Bayesian probability often involves complex mathematical calculations, which can be daunting for those without a strong background in mathematics or statistics. It requires a deep understanding of conditional probability, Bayes’ theorem, and often necessitates the use of computer software for practical applications.
3. Updating Probabilities: The Bayesian approach involves continuously updating probabilities as new data becomes available. This dynamic aspect of Bayesian probability is powerful but can be difficult to grasp for individuals accustomed to fixed and unchanging probabilities.
4. Inverse Thinking: Bayesian probability often asks individuals to think inversely, considering the probability of a cause given an effect, rather than the more intuitive effect given a cause. This inversion of thought can be perplexing for those not familiar with the Bayesian framework.
5. Data Interpretation: Bayesian probability requires individuals to interpret and quantify uncertainty in a different way. Instead of providing a single point estimate, it often results in probability distributions, which can be challenging to interpret for those unfamiliar with the concept.

The fact that a seemingly unrelated population can affect your probability of a certain event would naturally seem unprecedented – one of the main reasons why Bayesian probability is so counterintuitive to the public. We shall illustrate this further with disease testing.

Suppose a test for a certain disease has 99% specificity, i.e. it has a 99% chance to correctly identify individuals who do not have the virus. However, assume that you take the test, and it comes back positive. What is the probability that you have the disease? Well, you might think that it’s 99% or close to that given the specificity – au contraire, you actually cannot calculate the probability as there is insufficient information. In fact, you also need the sensitivity – assume that this is 100%.

Does this mean that you are doomed to have said disease? Most people (including doctors¹) might intuitively think so, however you still don’t have enough information according to Bayseian probability – you need to know how common it is in the general public. Provided that it is , we can now apply Bayes’ theorem to calculate the actual probability.

First, we need to calculate , which can be given by , which equals . Now we use Bayes’ theorem : . Astonishingly, even though the test has a very high specificity and sensitivity, there is still a very high chance (99.9% in fact!) that you do not have the disease. This fallacy is often miscommunicated in the medical world in that a positive result from an accurate test should imply a high chance of infecting said disease (which we have shown is obviously false), and also echos how seemingly unrelated factors (in this case the rarity of the disease) can affect your chance of actually having the disease. However, all of this does not mean that such tests should not be taken seriously – if we do look at our chance of having the disease before and after the test, we can see that it has improved dramatically from a chance to , which is an increase by a factor of 100. This is also affected by the rarity of the disease as the more likely the population is to contract the disease, the more certain you can be when faced with a positive test result that you have the disease.

Ultimately, Bayesian probability stands as a fascinating and powerful approach to decision-making and prediction, often regarded as a conundrum for both the public and experts alike. Its capacity to integrate seemingly unrelated information into our probability assessments challenges traditional intuition and opens the door to a more flexible and dynamic understanding of uncertainty.

Through the lens of Bayesian probability, we see that the probabilities of events are not static but instead evolve as new information becomes available. This evolutionary perspective on probability allows us to continually refine our beliefs, leading to more informed decisions and a deeper comprehension of the world around us.

While Bayesian probability may remain a conundrum to some, it is also a valuable and adaptable tool that can help us navigate the complexity of our ever-changing reality. By embracing its principles, we move closer to making better decisions, whether in fields as diverse as science, finance, medicine, or everyday life, and by doing so, we unlock the secrets hidden within the numbers and uncertainties of our world.

1. The Diagnostic Importance of the Normal Finding, New England Journal of Medicine, 29th June 1978 ↩︎