Andrew Gelman: Learning from mistakes

Links mentioned in the talk:
Election poll example: web.archive.or...
Nudge example: statmodeling.s...
statmodeling.s...
This talk: statmodeling.s...
American Statistical Association (ASA), Section on Statistical Learning and Data Science (SLDS)
January webinar: Learning from mistakes
Record: January 30, 2024
Presenter: Andrew Gelman is a professor of statistics and political science at Columbia University. He has received the Outstanding Statistical Application award three times from the American Statistical Association, the award for best article published in the American Political Science Review, the Mitchell and DeGroot prizes from the International Society of Bayesian Analysis, and the Council of Presidents of Statistical Societies award. His books include Bayesian Data Analysis (with John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin), Teaching Statistics: A Bag of Tricks (with Deborah Nolan), Data Analysis Using Regression and Multilevel/Hierarchical Models (with Jennifer Hill), Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do (with David Park, Boris Shor, and Jeronimo Cortina), A Quantitative Tour of the Social Sciences (co-edited with Jeronimo Cortina), and Regression and Other Stories (with Jennifer Hill and Aki Vehtari).
Andrew has done research on a wide range of topics, including: why it is rational to vote; why campaign polls are so variable when elections are so predictable; the effects of incumbency and redistricting; reversals of death sentences; police stops in New York City, the statistical challenges of estimating small effects; the probability that your vote will be decisive; seats and votes in Congress; social network structure; arsenic in Bangladesh; radon in your basement; toxicology; medical imaging; and methods in surveys, experimental design, statistical inference, computation, and graphics.
Abstract: We learn so much from mistakes! How can we structure our workflow so that we can learn from mistakes more effectively? I will discuss a bunch of examples where I have learned from mistakes, including data problems, coding mishaps, errors in mathematics, and conceptual errors in theory and applications. I will also discuss situations where researchers have avoided good learning opportunities. We can then try to use all these cases to develop some general understanding of how and when we learn from errors in the context of the fractal nature of scientific revolutions.
For more information about or to join ASA SLDS, visit
community.amst...
www.amstat.org/

[@dangernoodle2868]

On the topic of being an asshole when giving criticism. It's like Gelman says, you're best positioned to try and see through the delivery to the content but on the other hand as someone giving feedback it's important to be clear so that the other party doesn't have to do that work. It means that it's on everybody to try and communicate clearly but also for us to acknowledge that the rough delivery comes not from bad people but from people who are feeling something which motivated them to say something in the first place but that can pollute the message. Being shocking is useful to catch people's attention but especially once the dialogue is going you need to cut it out ASAP. But if we need to rely on shock to cut through noise then ideally you don't rely on that but rather find a way to make the environment less noisy so that consensus is enough to make the right conversation happen.