154 - Understanding the training and validation loss curves

[@dr_flunks]

This is outstanding. This is the first video to cover an actually useful process for developing a model from scratch in terms of arch decisions. Anyone else know of similar content?

[@ashutoshshinde5267]

Benefited immensely learning about the training and validation curves. Thank you!!

[@DigitalSreeni]

You're so welcome!

[@satrioyudanto9856]

The explanation is so clear for my deeper understanding about underfitting and overfitting phenomenon. Thanks! It's really helpful!

[@DigitalSreeni]

You're very welcome!

[@HosseinKianAra]

Thank you for your wonderful clips. Please also teach about epoch and batch size. Thank you

[@nerlia2854]

what if the number of trains Found 1896 images belonging to 12 classes. test Found 528 images belonging to 12 classes. and validate Found 276 images belonging to 12 classes. what is the epoch, batch size and patience so that the training & val accuracy and Training and val loss plot graphs match and are close together?

[@nazymsatbekova8863]

Thank you very much for the video, it is concise and covers a lot of cases for the learning curves. Exactly what i was looking for!

[@DigitalSreeni]

Great to hear!

[@farazshaikh4967]

Thank you very much for the amazing video. This is one of the most important topic that no body talks about!!

[@limotto9452]

Thank you very much sir for the video. You have enlighten me on the model evaluation for neural network. Any advice on optimising the hyperparameters?

[@zfolwick]

my validation accuracy is 98%. Submitting a real image to it... Not getting 98% accuracy.

[@Messiah-000]

Excellent series that covers a lot of important concepts that many tutorials typically do not cover in great detail.

[@DigitalSreeni]

Glad you think so!

[@Johan-b4l]

If I have a model that has only one dense layer and it immidiately over fits, does it mean that the data is unrepresentative? The trainning loss goes down and val loss goes up immidiately. Sorry for my english.

[@tlotlisangnkhase9475]

what will be the difference between reshuffling and changing random_state

[@soumyabrata111]

How nicely explained! Thank you so much

[@shankargonti8609]

Can you please provide the ppt of this lecture.

[@asraajalilsaeed7435]

Can you state how draw fid metric curve, i know the equation and i can get number score but i cant draw it😢

[@mahdibelhadi2252]

Can Loss's value drop to zero?

[@DigitalSreeni]

Loss functions normally minimize a value to zero. In such loss functions, you will get zero loss if the model perfectly fits to the training data. This almost never happens. You will see a very small value but not zero.

[@samuelkandolo1549]

Which among those plots represent a curve with noise?

[@ahmadneishabouri4306]

You mentioned a couple of times to keep changing the random_state in train/test splits and to choose the appropriate model based on their performance to each of these splits. But, doesn't this mean you are leaking information from your test set to train set? This way you'd choose your model while seeing the test set and this may not generalize to other unseen data. I haven't heard all your lectures but probably you are advising to include a separate validation set (train/validation/test) in your split. This would solve this problem and as much as trivial it might sound, it is a big problem in DL in my opinion. Very nice and informative video by the way, thank you!

[@alonsomoran1646]

yes! I also wanted to mention this... we shouldn't mix validation and test data! Thank you for the video tho

[@yizhenwu5707]

Thank you for this helpful video. You are the best teacher!

[@pammikumari9726]

Sir I need cascade of two pre-trained deep learning model code plz help me

[@smoothumut]

finally I have understood, thanks a lot

[@ZaTheZee]

Thank you that was really helpful

[@IrfanKhan-oh7kb]

Wisconsin breast cancer data set is not available

[@faez8220]

very useful information, ty sir

[@kumala-win]

wow! That is an amusing and clear explanation with justifications, thanks, Dear and keep it up!!

[@aomo5293]

Hi there. After training random foreset regressor on data I got the following scores: model.score(X_train,y_train) 0.88 model.score(X_test,y_test) 0.11 How can we intetpret this result ? Thank y

[@emadqalaja246]

when i used your code i have received this error " Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 30 but received input with shape (None, 7)" if you can help me please

[@emadqalaja246]

history = model.fit(X_train, y_train ,verbose=1, epochs=50, batch_size=64, validation_data=(X_test, y_test)) in this section

[@hemantsah6254]

Good morning sir. I am a research scholar and developing a ANN model. I am using four variables=Y=fx1,x2,x3... Twenty years of long panel data with 24 regions. I am getting problem to select weight and nodes to get training and validation for a good model. How to much weight and nodes should i assign to get a good model? kindly advice me. Thank You sir.....

[@muhammadgulfam1869]

So if the validation and training accuracy are almost same .... let's say 62% and don't improve as the training epochs increases. What does that represent? I assumed having validation and training accuracy the same is good but the fact that the accuracy is low like 62% means model doesn't have much to learn? How to improve that?

[@ilhamrizaldywidy744]

So what is the connection between loss and accuracy.. actually i didn't know how to explain it properly. and then i wanna ask something. i had descent accuracy at 75% and validation at 71%, but my loss is also high, accuracy at 66% and validation at 75%, is that means that my training model isn't good enough?

[@mmskawish]

I bet you are a professor... Your explanations suffice the surface level understanding and enable to fundamentally understand the concepts... For sure, those concepts could be polished enough to convert them into knowledge after sufficient self-study... In-short, you are a "virtual professor" ♥

[@harres8694]

very good video my friend, love you

[@jerkmeo]

Thanks for sharing how to determine the loss curve. very useful and concise.

[@Shaan11s]

This was so helpful I was struggling to see the big picture here and now I feel much more equipped! Thanks

[@skynetpro549]

hello sir, sir my training shows only "accuracy and loss" after each step, i want it to show validation loss and validation accuracy after each STEP, can u please help me?

[@teeg-wendezougmore6663]

Great video. Thanks for sharing. Is it possible to get zero as loss value? I noted on your graphics that the minimum value of loss 0.1. this value can be 0 ?

[@KamalkaGermany]

Hi, is there a way to train the same model multiple times (for example 10 times) using the same data. Also it would be great if you can make a video for increasing the breadth (filters eg. 8, 16, 32, 64) and the depth (layers eg. 3, 4, 5) and training them 10 times each and then may be take the mean of the say mean_squared_error and plot the bar graph of all the 12 combinations. If there is already a tutorial on this please share the link. Thanks in advance!

[@kannu5039]

Just a question after epoch 100 both the curve is just doesn’t do anything and it’s stable so what does it mean? Should we limit the epoch ? Is 100 epoch already good?

[@vijayasarathir4270]

thank you so much.as said training accuracy is higher than validation accuracy, but reason why you get validation accuracy higher than training accuracy. If it is so, how to correct it. Thanks in advance

[@RajeshSharma-bd5zo]

Great video, as always amazingly explained!!

[@DigitalSreeni]

Thanks!

[@IrfanKhan-oh7kb]

@@DigitalSreeni Wisconsin breast cancer data set is not available

[@extrememike]

Very good an practical explanation. Yo have another subscriber.

[@RafiKhalil]

Thank you, Sir, can you please make a video on Mask RCNN.

[@Ishant875]

This was too helpful.

[@mk2103.]

This video is so good. The ideas were clearly explained and shown through the graphs. The examples cover a lot of cases you might encounter! I will definitely recommend to others and rewatch this video if I am ever feeling confused. Thank you so much! You are a great teacher!

[@qandos-nour]

Very useful , thanks

[@adithaugustine3251]

Thank you! This video was very helpful..

[@kasun611]

Thank you for the video. Don't you need to split the data first and then scale - so that test data is not leaked to the model?

[@dmz2885]

Thanks a lot this really help me a lot the way you explain small small things in between that really helps

[@vaibhavvaibhav6118]

I don't know how exactly to thank you. Simply amazing !!!

[@DigitalSreeni]

You're very welcome!

[@Mojtaba-Sirati-Amsheh]

please make a tutorial about transformers for image regression tasks.

[@ozodbekozodov0609]

Great! Very clear and concise, and potentially all pitfalls are well-explained! I would appreciate if you talk about learning rate, batch size and kernel size and their impact on training and validation loss curves. Many thanks!

[@quantumbyte-studios]

Very informative, didn't get bogged down in the code

[@saidokieh2493]

Hello sir how to split the data into 3 dataset pls I need that code could u refers to me ...thanks so much

[@TheHarpanOnly]

Do you already make video about when to go up the layer or neuron?

[@DigitalSreeni]

kzbin.info/www/bejne/mKKlg3iKbNmVn7M

[@DONTfakeTOshow]

WHen running this Fit with early stopping and model checkpoint to save the best models. from keras.callbacks import EarlyStopping, ModelCheckpoint es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=50) mc = ModelCheckpoint('models/model-{epoch:03d}-{accuracy:03f}-{val_accuracy:03f}.h5', monitor='val_loss', mode='min', ver... I am obtaining an error eyError: 'Failed to format this callback filepath: "models/model-{epoch:03d}-{accuracy:03f}-{val_accuracy:03f}.h5". Reason: \'accuracy\'' any idea why?

[@DigitalSreeni]

Try replacing val_accuracy with val_acc and accuracy with acc. I wonder if you are using different names in your metrics and other locations.

[@ajilisaaliyar3156]

Sir please add tutorials on interpretable machine learning technique s

[@masbro1901]

thank you SIR !

[@rupeshnaik7626]

thank you so much sir, for this video

[@derejesenay8928]

my validation loss is very high and validation accuracy less than training accuracy by 20% but my dataset is small 2164 sentence in Bi-LSTM text classification please help if you are ok

[@DigitalSreeni]

For small datasets you will have smaller validation data. That means the validation curves will look a bit bumpy and the accuracy will be lower than training data. This is quite common. Try increasing the amount of validation data. If that doesn't help you need to change the model and try. May be no model can help and you need to acquire more data. Deep learning is great but cannot do miracles, especially for limited data.

[@ordealdube14]

Clearly explained - much apprecitaed

[@mathewchan503]

very clear and easy to understand video

[@DigitalSreeni]

Glad you think so!

[@arpitpatel7010]

Excellent explanation, just wondering which IDE you are using?

[@sharmisthachakrabarti9880]

That's Spyder

[@Shoz_]

Thank you

[@abdulbareehussein787]

hay :i got this - loss: 0.6026 - accuracy: 0.8960 loss: 1.1142 - accuracy: 0.7748 ist this overfitting?

[@DigitalSreeni]

How can you tell if it is overfitting by looking at loss and accuracy at one data point?

[@abdulbareehussein787]

@@DigitalSreeni i mean , if we have Acuuracy 0.90 and loss is 1.8, is this then an Overfitting?

[@suphotnarapong355]

Thank you

[@mubariknoure5057]

Thanks

[@Mojtaba-Sirati-Amsheh]

Thank you

[@ImranAhmed-tf6uk]

Can you please make a video lecture on K-fold cross validation and evaluation on test datasets with example code. Please explain from scratch, how to split datasets and everything with example. thanks in advance. Your lectures are very helpful to learn. Great work.

[@DigitalSreeni]

Will try

[@benjaminschnepf7362]

your videos are great! thank u

[@guillermoalejandrobastianf7330]

You're the best! Very complex scenarios explained in simple way

[@tilkesh]

Thanks

[@muiistudio7741]

Amazing explanation!!! Loved it. Thank you.

[@jameshopkins3541]

Understanding nothing from here

[@منةالرحمن]

thank you again !! where ever i need y you're there ^_^

[@cwumin2105]

Hi Sir. I have a questions regarding the loss. Which should we check the loss in normal practice? loss per epoch or loss per batch? thanks

[@hirutabera1947]

what it show i got training accuracy of 1.0 and testing accuracy of 0.99 is it good or correct

[@DigitalSreeni]

It shows that you are getting awesome results :)

[@gauthamsk4040]

Perfectly Explained... I wish I get teacher like you.

[@DigitalSreeni]

Well, you have a teacher like me on KZbin :)

[@tymothylim6550]

Thank you very much for the video. Really clear explanations and wonderful slides! :)

[@deandevilliers2799]

No mention of loss rate? xD

[@satinathdebnath5333]

Thank you very much for delivering very detailed and informative videos. I have a doubt from last good fit slide. I understand that the length of training loss should be always greater than validation loss. However, sometimes I see that with some parameter changes such as validation split or batch size the training and validation loss curves toggles. So, training loss curve become shorter than validation loss curve. Could you please hint me what mistake I am doing? If anyone can clear my doubt it would be highly appreciated too. Thanks once again.

[@DigitalSreeni]

I am not sure what you mean by training curves being shorter than validation curves. They both should be of the same length as the x-axis represents epoch number and you'd get some values for training / validation for each epoch.

[@satinathdebnath5333]

@@DigitalSreeni I meant to say training loss curve and validation loss curve. Do they need to have same length on the graph or Training loss curve is usually longer than validation loss curve. From online, I see that validation loss curve tip start after sometime from the start of Training loss curve.

[@389_kavetiupender2]

Nice video. I have doubt .. I'm working on a cnn model... Should I use training and testing dataset or training, validation and testing dataset?? I'm waiting for your ans

[@DigitalSreeni]

I divide my data into training, validation, and testing. I use training and validation during training where I track the loss curves and I use testing data to verify accuracy after the training process.

[@389_kavetiupender2]

Thanks alot sir for quick and clear reply.....

[@limzisin26]

Thank you so much sir. All of your videos really taught me a lot during my journey of completing my FYP!!!

[@DigitalSreeni]

Great to hear!

[@alricliew714]

Well explained the curves.

[@pythonbrothersandfamily]

I learnt a lot, thank you!

[@ZanBor69]

Great explanation! Thanks!

[@paulntalo1425]

Thank you for such a rich and practical example. Which IDE are you using

[@DigitalSreeni]

Spyder IDE, part of Anaconda package.

[@paulntalo1425]

@@DigitalSreeni am two months into learning data science. I am using jupyter Notebook using pip3 and not anaconda. So is Anaconda always a better choice?

[@satadrudas3675]

Beautifully explained

[@anuragkumar1015]

Wow! Best video on this topic.

[@alejandrovillalobos1678]

thank you very much

[@lizbethmontiel-ruiz2077]

Excellent!! Very Well Explained!!

[@abderrahmaneherbadji5478]

Very useful video. Thanks a lot.

[@DigitalSreeni]

You are welcome

[@alexburger7214]

bowed down to you sir ! what a crystal clear explanation

[@DigitalSreeni]

You are welcome.

[@ebrahimahmedal-rahawe3161]

Thanks a lot sir, for all these information..

[@DigitalSreeni]

Always welcome

[@anua7589]

Thank you for the informative video! For the splitting of the data, can we consider a stratified split to get a more representative train and test sample? i.e. X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.25, random_state = 42, stratify=Y)

[@mostafaessam3994]

Hey Could elaborate more how to do stratify sampling ? If we have 9 units (engine data) and we take 6 of them to train the model and 3 of them to test the model Appriciate your reply.

[@cyrilmouttet1856]

Thank you! I was wondering what to do if train and validation converges at Epoch x (like a good fit at 26:40) and Loss doesn't decrease anymore for many more Epochs. Should we try adding epochs till it starts to overfit or stop learning at the first epochs having the lowest Loss?

[@julianaELF1D]

Should have just stopped learning at the first lowest loss. This trick can be performed by keras early stopping... but you need to wait for 10 to 20 epochs to see that your model has absolutely gotten the optimal result (will not overfit and training and validation already nearly constant zero)...

[@thebiggerpicture__]

Thanks, great video!

[@DigitalSreeni]

Glad you liked it!

[@hilalkucuk5]

It is brilliant

[@DigitalSreeni]

Thanks

[@nasgaroth1]

Really awesome, keep going that way ;-)

[@DigitalSreeni]

Thanks, will do!

[@pioner40]

Thanks

[@DigitalSreeni]

Thank you