[Spring 2011] Intro to Bayesian data analysis

A very nice compact argument for Bayesian methods can be found in John Kruschke‘s Bayesian Data Analysis, WIREs Cognitive Science 1, 658-676. Here’s the very beginning of the article (followed by a section entitled “The road to NHST is paved with good intentions”):

This brief article assumes that you, dear reader, are a practitioner of null hypothesis significance testing, hereafter abbreviated as NHST. In collecting data, you take care to insulate the data from your intentions. For example, double-blind procedures in clinical trials insulate the data from experimenter intentions. […] After carefully collecting the data, you then go through the ritual invocation of p < 0.05. Did you know that the computation of the p-value depends crucially on the covert intentions of the analyst, or the analyst’s interpretations of the unknowable intentions of the data collector? This is true despite the emphasis by the data collector to make the data unaffected by his/her intentions, as will be shown below. Moreover, for any set of data, an intention can be found for which p is not less than 0.05. There is a better way to draw inferences from data. Bayesian data analysis is gaining acceptance in many fields as the best way to conduct data analysis, but many disciplines within cognitive science have been slow to re-tool. This brief article reviews a fundamental problem with NHST, and shows some of the advantages of Bayesian data analysis. […] This article emphasizes the fatal role of experimenter intention in NHST, and that this fault is inherent in confidence intervals too. The article highlights multiple comparisons of groups as an illustration of the advantages of Bayesian analysis. This article also presents two perspectives on Bayesian interpretation of null effects. Finally, this article describes Bayesian approaches to statistical power, more generally framed as the probability of achieving a research goal.

A good way to start thinking about the differences between Bayesian and frequentist approaches is to think about the two-envelope problem along the lines outlined in the very short article Bayesian Resolution of the “Exchange Paradox” (Ronald Christensen & Jessica Utts 1992; in The American Statistician 46:4, 274-276). Here’s the very beginning of the article:

One of the arguments in favor of using Bayesian methods is that prior assumptions are made explicit rather than being incorporated implicitly into the solution of a problem. In this article we discuss an interesting paradox and show how explicit use of prior assumptions leads to its resolution. The paradox is also resolved using a frequentist argument and illustrates the pitfalls of using symmetry incorrectly. It shows that careful thinking is important in solving probability problems and elucidates the difference between the Bayesian and frequentist approaches. […] A swami puts m dollars in one envelope, and 2m dollars in another. He hands one envelope to you and one to your opponent, so that the probability is 1/2 that you get either envelope. You open your envelope and find x dollars. Let Y be the amount in your opponent’s envelope. You reason that since the envelopes were handed out with equal probability, Y = x/2 or Y = 2x, each with probability 1/2. Thus, your expected winnings from a trade are 1/2(x/2 + 2x) = 5x/4 which is obviously greater than the x dollars you currently possess. With a gleam in your eye, you offer to trade envelopes with your opponent. Since she has made the same calculation, she readily agrees. The paradox of this problem is that the rule indicating that one should always trade is intuitively unreasonable while the method of arriving at the rule seems very reasonable.

We’ll be using WinBUGS; if you’re using MacOS or Linux, you need to install a Windows emulator, e.g., WINE, then download the Windows version of R and install it within WINE, then download and install WinBUGS.

1. Basic notions (Bayesian vs. frequentist data analysis, posterior distribution proportional to prior distribution times the likelihood, Bayesian parameter estimation), Markov Chain Monte Carlo (MCMC), basic examples of Markov chains, simulating the Monty Hall problem, convergence monitoring, why do we care about Bayesian methods: CLG-bayes-winbugs-spring2011-1.r.

   Plan for the following meetings: we will discuss increasingly complex models; for each model, we simulate the data, provide its frequentist analysis using R and its Bayesian analysis with WinBUGS (run from R; no point & click); we start with the mean model, then introduce various linear models (t-tests, ANOVAs etc.) without and with random effects, then move on to generalized linear models.

2. The mean model, simulated data, R analysis, WinBUGS analysis, the structure of WinBUGS models, reexpressing parameters, number of chains, number of iterations, burnin, thinning, the Brooks-Gelman-Rubin (BGR) convergence diagnostic (a.k.a. Rhat), graphical summaries of posterior distributions, essentials of linear models (focus on design matrices), t-tests with equal and unequal variances (simulated data, R analysis, WinBUGS analysis): CLG-bayes-winbugs-spring2011-2.r.

3. Simple linear regression (simulated data, R analysis, WinBUGS analysis), goodness-of-fit assessment in Bayesian analyses (posterior predictive distributions and Bayesian p-values), interpretation of confidence vs. credible intervals, fixed-effects 1-way ANOVA (simulated data, R analysis, WinBUGS analysis), random-effects 1-way ANOVA (simulated data, R analysis, WinBUGS analysis): CLG-bayes-winbugs-spring2011-3.r.

4. 2-way ANOVA w/o and w/ interactions (simulated data, R analysis, WinBUGS analysis), ANCOVA and the importance of covariate standardization (simulated data, R analysis, WinBUGS analysis), linear mixed-effects models—-random intercepts only, independent random intercepts and slopes, correlated random intercepts and slopes (simulated data, R analysis, WinBUGS analysis): CLG-bayes-winbugs-spring2011-4.r.

5. Generalized linear models, Poisson “t-test” (simulated data, R analysis, WinBUGS analysis), Binomial “t-test” (simulated data, R analysis, WinBUGS analysis), Binomial GLMM (simulated data, R analysis, WinBUGS analysis), GLMMs that take into account inter-annotator disagreement (simulated data, R analysis, WinBUGS analysis): CLG-bayes-winbugs-spring2011-5.r.