Layman's Summary

When someone conducts an experiment, he needs a way of measuring how reliable his results are. For example, when I flip a fair coin, it should come up heads about half the time. If I flip the coin ten times and get six heads and four tails, does that mean the coin is biased? What about seven heads and three tails? Eight heads and two tails?

Statisticians have provided us with precise ways of answering this question. The exact calculations we have to do depend on the type of data we're working with; for example, proportions (like the coin example) require certain kinds of statistical tests, while averages require other kinds. I was interested in applying statistical tests to a measurement called d', which is a way of measuring how well someone can perceive something (for example, how well a person can hear the difference between two kinds of speech sounds). This paper describes two ways of testing d' that have previously been tried, and compares them to a third way that, to my knowledge, has not been tried before.

Summary

Experiments that measure d' often test for significance by using something like a t-test on the group means of d's obtained from two different conditions - for example, comparing the d's of the subjects in one condition to the d's of the subjects in another condition. However, since each d' is calculated from many separate observations, it is possible to perform statistical tests on individual values of d' without grouping them. This is useful, for example, in comparing sensitivity in two conditions for a single subject in a within-subjects design. This paper compares three methods for obtaining confidence intervals around a single d' - an analytic approximation, a computational method that assumes no response bias, and a computation method that uses Maximum Likelihood Estimation. (The third method, to my knowledge, has not been previously described in the literature.) The accompanying R code implements each of these methods.

Downloadable Stuff

(See CV for citation information.)

R Code: Confidence Intervals for d'
This is the R code that I wrote to find confidence intervals for individual d' values. The functions it contains, and the statistical methods used, are described in the paper below. The MLE method runs in polynomial time based on the precision to which the confidence interval is calculated; I find that on my machine, the time required to calculate the underlying values for hit and false alarm rates to three decimal places is quite reasonable.

Paper on Confidence Intervals for d'
2009 A comparison of three methods for calculating confidence intervals around d-prime
This paper describes the MLE method I use in other work to calculate confidence intervals around d', and compares it to two other methods.