Train/Dev/Test Set Distributions (C3W1L05)

[@DistortedV12]

Where is the video where he talks about the training distribution?

[@ninalee7833]

What's methods can be used to identify/check the distribution difference between training and testing datasets?

[@meandkg]

3 months wasted is not bad compared to approx. 1 year for me (I'm still a student, better to make this mistakes now rather than later)

[@韩毅-s6l]

What exactly does the distribution mean? I suppose it's the distribution of the feature. However, most people talking about one feature distribution but what about we have many features? One answer in my head is joint kernel density estimation + KL divergence. It would be quite expensive though. I am not sure. Someone have the answer？

[@rjohnson0186]

韩毅 It refers to the data generating process (DGP). You want to train a model on data that looks just like the data you will observe in production. A simple way is to randomize your data and then split into training, test, etc. But there are also times when you might need to do stratified sampling because you might have a rare class and you want to ensure it is present across all splits

[@davidsevic8317]

why exclude africa as one of the regions lol