Aaditya Ramdas

The lady keeps tasting coffee: opening the doors to interactive experimental design and causal inference via betting and cooperation

We revisit Fisher's classical randomized experiment from the early 1920s titled The Lady Tasting Tea. To recall, algologist Dr. Muriel Bristol was asked to taste 8 cups of milky tea, 4 made one way (milk first) and 4 made another (tea first). These were presented to Muriel without labels, Fisher conjecturing that she could not differentiate between the two types, while she claimed the opposite. Despite apparently being one of the first documented instances of randomization-based causal inference, the experimental design was rather unfair to Muriel, requiring her to perfectly identify the cups in order to rule out random guessing. This paper revisits the experimental design to make it more favorable to Muriel. We allow her to keep tasting more cups of tea that are randomized one cup, or one pair, at a time. Muriel is also given feedback: after each guess, she is told whether she was right or wrong, allowing her to learn, thus honing her tastes over time. We use martingale techniques to demonstrate how to perform sequential causal inference in such settings, where the experimenter and participant can (should?) cooperate. Preliminary results from this summer on a willing participant (my partner Prof. Leila Wehbe) with a different beverage (coffee) will be reported.