Time, Interrupted: Measuring Intervention Effects with Interrupted Time-Series Analysis - Ben Cohen

PyData LA 2018
How can we estimate the impact of a historical event where there is no way to run a controlled experiment? For example, we may want to assess the impact of a TV campaign or account for lost sales during an outage. This talk presents a brief overview of interrupted time series analysis, a technique commonly used in econometrics and public health that is designed to address this type of problem.
---
www.pydata.org
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.
PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.
00:00 Welcome!
00:46 Introduction
02:41 What is Interrupted Time Series Analysis
03:53 A/B Testing
04:53 How to measure the impact of a national TV campaign
05.44 Geo-targeting
06:25 How can we know if something we did had an effect
08:05 Counterfactuals
09:23 Interrupted Time Series
13:01 Building a time series counterfactual
14:01 Non-stationarity
15:30 Auto-correlation
16:15 Independent and identically distributed assumptions
17:52 What should the model include
19:45 Prediction intervals
22:19 Prophet library
23:26 Training and prediction
24:53 Assess accuracy of the model
26:20 Compare predictions to observations
26:53 Lift analysis
27:00 Samples from the posterior predictive distribution
27:31 Pointwise vs cumulative estimates
29:38 Answering probability-based questions
30:05 Threats to validity
30:53 Change in the underlying process
32:49 Confounding variables
33:47 Model misspecification
36:27 Q&A
36:35 Business applications
38:26 Situations where it worked or didn't
39:45 Comparing different channels of advertisement
40:50 Data preparation for Interrupted Time Series
41:55 Ramp-up period before measuring the effect
43:06 Assessing whether the counterfactual is correct
S/o to github.com/fsammarc for the video timestamps!
Want to help add timestamps to our KZbin videos to help with discoverability? Find out more here: github.com/numfocus/KZbinVi...

[@amym7075]

really clear presentation! he's such a good communicator

[@ramadhanbagindo1030]

It is like R Package causal impact by google

[@akashchandrayan7255]

Isnt this just an application of forecasting?