Train, Test, & Validation Sets | How to Train Machine Learning Models (Properly!!!)

The Notebook: colab.research.google.com/dri...
Thank you for watching the video! You can learn data science FASTER at mlnow.ai!
Master Python at mlnow.ai/course-material/python/!
Learn SQL & Relational Databases at mlnow.ai/course-material/sql/!
Learn NumPy, Pandas, and Python for Data Science at mlnow.ai/course-material/data...!
Become a Machine Learning Expert at mlnow.ai/course-material/ml/!
Don't forget to subscribe if you enjoyed the video :D

[@GregHogg]

Take my courses at mlnow.ai/!

[@djjiang3718]

Just want to say Thank you, Greg! love your content always good stuff! Keep up with hard work! Thanks for teaching and sharing !

[@GregHogg]

I really appreciate this, thanks so much and you're very welcome 😄😄

[@tartlord7061]

Great and easy to follow video, thanks!

[@natureclipsreal]

If I am using cross validation during the training process, can I still use a validation set after training the model?

[@panagiotisgoulas8539]

Cool. thanks for that.

[@arsheyajain7055]

Oooh awesome!!

[@PeterPan-xp2gl]

Hey Greg, thanks for your video! Could you explain why you convert the dataframe to a numpy matrix? Since the whole procedure also works with data frames?

[@ismailucuran4523]

probably its because he didn't use scikitlearn treain_test_split

[@kshitijalawandi4443]

This is awesome....

[@GregHogg]

Thank you!

[@mayreyes4402]

when I try to find mean absolute error for linear_test_ preds the result when I run it is "ValueError: could not convert string to float: 'KS'" what should I do with that?

[@itowindsor]

One of your feature might be a categorical one. Therefore your models can't work with it. One way to overcome it is to one hot encode this feature. It will increase the dimension of your features spaces (as you will have one more dimension for each category) but every algorithm will understand it

[@emekaanammah9207]

Where did you download the california housing test and train dataset from?

[@GregHogg]

Google Colab. Free cloud env

[@nayliafiqah4072]

Thank you so much;

[@GregHogg]

You're very welcome!!

[@user-ip9zo8zp4c]

Is it possible that you wrote the arguments of 'mean_absolute_error' in the wrong order?

[@jyotikapatil2354]

Only video explaining actual hyperparameter tuning using validation test.

[@diazjubairy1729]

How to set the proportion of train, val and test of our dataset ?

[@GregHogg]

How technically, or what numbers should you use?

[@diazjubairy1729]

@@GregHogg train : val : test = 50 : 20 : 30

[@GregHogg]

@@diazjubairy1729 is this a question?

[@diazjubairy1729]

@@GregHogg that is the proportion that i want, how to split the dataset like that ?

[@GregHogg]

@@diazjubairy1729 sklearn train test split (do it twice), or shuffle a pandas DataFrame and get split indices

[@pogiribhanuteja4650]

When will be the next live session..?

[@GregHogg]

I think I'll be busy this weekend unfortunately. If not, this weekend, otherwise probably the weekend after.

[@mehediazad1780]

you confused me by your variable name