Difference in Differences (The Effect, Videos on Causality, Ep 51)

Please visit www.theeffectbook.net to read The Effect online for free, or find links to purchase a physical copy or ebook.
The Effect is a book about research design and causal inference. How can we use data to learn about the world? How can we answer questions about whether X causes Y even if we can't run a randomized experiment? The book covers these things and plenty more. These videos are meant to accompany the book, although they can also be viewed on their own.
This video relates to material found in Chapter 18 of the book.
A version of this video without background music can be found here: • Difference in Differen...
It's the most commonly-used research design in observational causal inference! Difference-in-differences is a pretty simple idea that can be applied any time you have a treatment that went into effect at a particular time, but it only applied to some groups and not others. How does it work? This video explains.

[@kasberge7164]

Hi Nick! I really enjoy your book as well as videos! One thing, I haven't found information on however is whether "true" panel data is required for DiD or if pooled cross-sections (for example repeated survey data) will do as well. I'd be great to receive a response. Cheers from Germany, thanks for making all these resources available!

[@NickHuntingtonKlein]

Pooled cross sections work fine, although you'd want to be sure that the sampling process isn't changing over time. Also it makes adding covariates trickier (see Sant'Anna and Zhao for some tips there)

[@kasberge7164]

@@NickHuntingtonKlein THANKS! I checked out the paper. Thanks for doing what you do!

[@kasberge7164]

@@NickHuntingtonKlein it's me again. Sorry... I have encountered another issue. As I am relying on this repeated cross-sectional survey data (random, relatively large samples), I am unsure which criteria to employ in order to find a suitable control group. I am using geography (more precisely macroeconomic and demographic characteristics of regions and their similarity in these respects assesed by a set of indicators from statistical agencies) but this is not exactly guaranteeing that groups are similar. Against the backdrop of the random sampling, do I have to worry about that?

[@NickHuntingtonKlein]

@@kasberge7164 If by random sampling you mean that treatment was randomly assigned, then you're fine. If you really mean random sampling, then that doesn't really help you here. I'd recommend drawing a DAG (as described in chapters 6-8) to see if you can actually match on the factors necessary to make parallel trends plausible. Alternately, try a parallel trends sensitivity approach like Rambachan and Roth 2022.

[@kasberge7164]

@@NickHuntingtonKlein Heyo, sorry for the issues with vocabulary and imprecision with regards to econometric language... by random sampling I mean the procedure with which the data I use was obtained... it can be broken down to regional levels (remains, however, individual-level data as usual with surveys). So I figured that, theoretically, respondents' location could be a suitable approach for constructing a control group. The treatment group arises "naturally" as they are subject to an, in my case, EU policy.