Approximate Bayesian Computation: The Likelihood is Dead, Long Live Simulation!

Knowingly or unknowingly, a web of complex models underpins modern life. Models are used to predict traffic patterns (via mechanistic network models or interconnected dynamic queues), help environmental agencies control pest populations (such as invasive species), and underpin policy decisions regarding epidemiological restrictions for disease prevention (such as restrictions aimed at reducing the spread of COVID-19). To use such models, we must first perform inference on the unknown quantities that drive these models. However, in many interesting cases the cornerstone upon which inference is based, the exact Bayesian posterior, is intractable and cannot be readily accessed using standard techniques.

Approximate Bayesian methods provide a solution to conducting inference in challenging settings where the exact posterior is intractable. Using simulation from the model, and by replacing the unknown likelihood with a version estimated using simulated data, these methods circumvent the intractability of the original inference problem. However, this simplification is not without cost, and ultimately results in a posterior distribution that only approximately expresses our uncertainty about the true process assumed to have generated the data.

In this course, we introduce the ideas behind simulation-based approximate Bayesian methods, the practical application of these methods, their theoretical behavior, and discuss fields of application where these methods are commonly applied. A general outline of the course is as follows.

Lecture 1: Intractable Likelihoods, standard solutions, and the method of Approximate Bayesian Computation (ABC).

Lecture 2: The statistical behavior of basic ABC.

Lecture 3: Alternatives to basic ABC.

Lecture 4: Limitations of approximate Bayesian inference, and (some) potential solutions.

Labs: Application of ABC methods in R.

Pre-requisites

The course requires at least a basic understanding of several concepts in classical and Bayesian statistics. Pre-requisites include at least some familiarity with the following topics.

• Bayesian statistics
  o Posterior probability.
  o Posterior sampling methods.
  • Accept/Reject, Importance sampling,
  • MCMC.

• Classical statistics
  o Consistency
  o Law of Large numbers
  o Central Limit Theorem
  o Empirical distribution functions

All statistical computing will be carried out in R, and thus the course requires basic knowledge of R and working software. A package list of the programs that will be needed to complete the labs will be sent out before the course begins.

Pre-reading

1) Marin, JM., Pudlo, P., Robert, C.P. et al. Approximate Bayesian computational methods. Stat Comput 22, 1167–1180 (2012). https://doi.org/10.1007/s11222-011-9288-2.

2) Csilléry, Katalin, Michael GB Blum, Oscar E. Gaggiotti, and Olivier François. “Approximate Bayesian computation (ABC) in practice.” Trends in ecology & evolution 25, no. 7 (2010): 410-418.

3) K Csillery, L Lemaire, O Francois, MGB Blum. “Approximate Bayesian Computation (ABC) in R: A Vignette”. https://cran.rroject.org/web/packages/abc/vignettes/abcvignette.pdf

4) Drovandi, Christopher C., Clara Grazian, Kerrie Mengersen, and Christian Robert. “Approximating the Likelihood in Approximate Bayesian Computation.” arXiv preprint arXiv:1803.06645 (2018).