Why it's important to split your data

Why should you use separate datasets for exploratory data analysis (EDA) and for statistical hypothesis testing? The logic of statistical inference is all about surprise: do these data surprise you? Well, it's hard to be surprised by patterns in data that you've seen already...
This video is a 30 second summary of how not to be a charlatan with data: don't use the same datapoint for generating your hypothesis and testing your hypothesis. Split your data!
Learn more on my blog:
bit.ly/quaesita_charlatan
bit.ly/quaesita_sydd
or
Watch this episode of my ML/AI course:
bit.ly/mfml_049
If you find my musings useful, show some love to those subscribe and share buttons.

[@denpluzhhnikov4969]

"watching you just to have a good mood" - professional head of data science at my company:)

[@greensock4089]

"We don't need to split our data, we have so much of it that p-values are always tiny and significant" - head of data science at my company...

[@Flylikea]

One input is just one input. Isn't that what we are talking about? So, if, for example, one notices a sudden increase or decrease in something, then they'd have to gather a larger dataset to check if what they saw means something or is just normal. (I mean... this is the point, right??? I perform EDA, notice a pattern (eg specific demographic group and risk for churn), that's the inspiration. To test the rigor of my inspiration/observation I'll have to check a greater or different time period or across different cities)

[@mitto20]

How?