Cynthia Dwork on Measuring Our Chances: Risk Prediction in This World and its Betters

Dr. Cynthia Dwork is the Gordon McKay Professor of Computer Science at the Harvard University John A. Paulson School of Engineering and Applied Sciences and Affiliated Faculty at Harvard Law School.
Abstract:
Prediction algorithms score individuals, assigning a number between zero and one that is often interpreted as an individual probability: a 0.7 “chance” that this child is in danger in the home; an 80% “probability” that this woman will succeed if hired; a 1/3 “likelihood” that this student will graduate within 4 years of admission. But what do words like “chance,” “probability,” and “likelihood” actually mean for a non-repeatable activity like going to college? This is a deep and unresolved problem in the philosophy of probability. Without a compelling mathematical definition, we cannot specify what an (imagined) perfect risk prediction algorithm should produce, nor even how an existing algorithm should be evaluated.
Outcome Indistinguishability, a notion with roots in complexity theory, provides an avenue of attack. Outcome Indistinguishability (OI) frames learning not as loss minimization-the dominant paradigm in supervised machine learning-but instead satisfaction of a collection of “indistinguishability” constraints. OI considers two alternate worlds on individual-outcome pairs: in the natural world, individual outcomes are generated by Real-Life’s true distribution; in the simulated world, individuals’ outcomes are sampled according to the predictive model. OI requires the learner to produce a predictor in which the two worlds are computationally indistinguishable (Dwork, Kim, Reingold, Rothblum, Yona, STOC 2021). The notion has provided a generous springboard, first and foremost in machine learning, and, very recently, in complexity theory.
Outcome Indistinguishability generalizes multicalibration, a concept arising in the study of algorithmic fairness (Hébert-Johnson, Kim, Reingold, Rothblum, ICML 2018). A question lingers: both (1) our qualifications, health, and skills, which form the inputs to a prediction algorithm, and (2) our chances of future success, which are the desired outputs from the risk prediction algorithm, are products of our interactions with a systematically inequitable real world. How, and when, can we hope to simulate not this world, but a better world, one for which, unfortunately, we have no data at all (Dwork, Reingold, Rothblum, FORC 2023).