End-to-End: Automated Hyperparameter Tuning For Deep Neural Networks

[@nithinisfun]

everytime you code, i learn something new. please never stop coding end-to-end in your videos. thank you, you are amazing!

[@priyankasagwekar3408]

This video was really helpful. It was 1 hour bootcamp covering everything about ANN with pytorch- from loading datasets, defining neural network architecture and optimizing the hyperparameters with optuna.

[@sambitmukherjee1713]

Super cool Abhishek. Loved every section, especially the "poor man's early stopping"... ;-)

[@mikhaeldito]

Thank you for sharing your knowledge. This is an amazing tutorial with no inaccessible jargons. 10/10 highly recommend.

[@AnubhavChhabra]

Great explanation! Making lives easier one layer at a time :)

[@ephi124]

I am writing a research paper in this area. I can't wait!

[@Phateau]

Really appreciate the effort you put in the video. This is world class. Thank you

[@lokeshkumargmd]

This is first time I am watching your video. Very informative !!!. Thanks for sharing 😇

[@bhumikachawla9149]

Great video, thank you!

[@shaikrasool1316]

Every time some things new.. thank you so much

[@neomatrix369]

Love the video, Hyperparam optimisation is one of my favs and this video tops it all, so now I gotta do this on my model training! :tada:

[@malachinelson2622]

you prolly dont care at all but does anyone know a tool to log back into an Instagram account? I stupidly forgot the password. I would love any help you can offer me

[@kaysencasen9519]

@Malachi Nelson instablaster :)

[@malachinelson2622]

@Kaysen Casen i really appreciate your reply. I found the site through google and Im waiting for the hacking stuff atm. Takes a while so I will get back to you later with my results.

[@malachinelson2622]

@Kaysen Casen It worked and I finally got access to my account again. I am so happy:D Thanks so much, you really help me out!

[@kaysencasen9519]

@Malachi Nelson happy to help =)

[@TheOraware]

wonderful mate , much appreciated for sharing it

[@priyankasagwekar3408]

For those looking for loading the models and using them on test dataset: model = TheModelClass(*args, **kwargs) model.load_state_dict(torch.load(PATH)) model.eval()

[@yasserahmed2781]

what a gem

[@jeenakk7827]

That was a very informative session. Is Hyperparameter tuning covered in your book? I think I should buy a copy!! Thanks

[@abhishekkrthakur]

Yea. it is but if you just want hyperparameter optimization, watch my other video

[@RajnishKumarSingh]

Love the fun part👌

[@AayushThokchom]

A general question: Is HPO hyped? If ensemble performs much better, should we invest time in HPO given we have limited time? Thoughts!!

[@tiendat3602]

awesome. But one question that, how to deal with overfit and underfit issue while building the end-to-end fine-tuning model ?

[@sindhujaj5907]

Thanks for the amazing video! Here in this example will the hidden size and dropout change for each hidden layer or remain same for the hidden layers?

[@siddharthsinghbaghel441]

Do you have any blogs??, I like reading more than watching

[@kaspereinarson1061]

Thanks for a great video! So just to be clear: you’re using standard 5 fold CV thus optimising for a set of hyper parameters that finds the best loss across (the mean of) all 5 folds. Wouldn’t it be more suitable to split the train data into train / val and then optimize the hyper parameters individually for each fold (nested CV) ?

[@kannansingaravelu]

Hi Abhishek, just landed up on this video. I am not sure whether you addressed this earlier. I am curious to know your preference of torch as against tensorflow or keras.

[@kuberchaurasiya]

Great. Waiting eagerly. Will you use (sklearn)pipelines?

[@abhishekkrthakur]

pytorch

[@MadrissS]

Hi Abhishel, very cool video as always, don't you think we should reset the early_stopping_counter at 0 after a new best_loss is found (line 62 at 41:20 in the video). Thanks !

[@priyankasagwekar3408]

I have 5 models saved for each fold at the end of execution. If I am not wrong they are essentially the same model saved 5 times. I was looking for a way to load the models and use them on test dataset. Pytorch Documentation shows following way, model = TheModelClass(*args, **kwargs) model.load_state_dict(torch.load(PATH)) model.eval() now initialising the model object (step 1) is an issue in the absence of logs and knowledge of exact architecture of best model. Also you need to define optuna sampler seed to reproduce the results.

[@avinashmatani9980]

Do you have any videos, if I want to learn the basics of what you did at the start. Like for eg: at the start you created a class.

[@stilgarfifrawi7155]

Great video . . . but when can we get a mustache tutorial?

[@oligibbons]

Why do you keep the same number of neurons in every layer? How would you change your approach for deep learning models of different shapes?

[@hiteshvaidya3331]

why did you make loss function static?

[@HabiburRahamaniit]

Respected sir , I have a question regarding a problem if we have a variable length input dataset and variable length output dataset how would we train or build a neural network model for that dataset?

[@renatoviolin]

Maybe a Recurrent Neural Network (RNN), that aim to solve this problem of different input size for each sample.

[@RajnishKumarSingh]

Sir, What best trial value tells us after every trial? I have used it with lightgbm seems working but doesn't do well with test dataset After every trial I calculated accuracy it is giving me approx 0.9942 for every trial not same but 1st two digit after decimal is same.

[@valentinogolob9137]

shouldn''t we set the early_stopping_counter to zero each time the valid_loss is smaller than the best_loss ?

[@marwaneelazzouzi2999]

i think we should

[@hasanmoni3928]

How can I buy your book in Bangladesh?

[@ankushjamthikar9780]

What is to be done if I want to tune the activation function as well in the neural network? How and where should include the line of code for it?

[@jamesmiller2521]

Where is your GM hoodie? 😤😁

[@neomatrix369]

Same question from me, why were you not wearing it ;) :P

[@abhishekkrthakur]

next time :)

[@MisalRaj]

👏👏

[@neomatrix369]

Any plans to make videos using other HyperParam Optimisation frameworks? I have a washlist I can share if you like ;)

[@abhishekkrthakur]

check out my other video :) and send the list too please

[@bjaniak102]

What is Julian from Trailer Park Boys doing in your thumbnail though?

[@abhishekkrthakur]

lol

[@jonatan01i]

You could speed up evaluation if you put the prediction in a torch.no_grad() context.

[@mazharmumbaiwala9244]

at 34:42, whats the use of `forward` function?

[@fredoliveira1223]

Its method of nn.Module, when you define a model the forward function is where you define how the data should pass through the layers of your neural network to make a prediction

[@vasudhajoshi4766]

Hello Sir I followed this tutorial to estimate the hyperparameters for my CNN model. When I am freezing the initial layers of my model, I am facing an error in the line: "optimizer = getattr(optim, param['optimizer'])(filter(lambda p: p.requires_grad, model.parameters()), lr=param['learning_rate'])" where param['optimizer'] is 'optimizer':trial.suggest_categorical('optimizer', ['Adam', "RMSprop"]) and param['learning_rate'] and param['learning_rate']: 'learning_rate':trial.suggest_loguniform("learning_rate",1e-6, 1e-3). The error is IndexError: too many indices for tensor of dimension 1. Can you please explain why I am facing this error?

[@Falconoo7383]

I want also for my CNN+LSTM model. If you resolve the error, can you please help me?

[@rubenallaert9654]

Hi, where can I find the code?

[@abhishekkrthakur]

its more of a code along video.

[@nikolabacic9790]

Did not tune random seed smh

[@Prasad-MachineLearningInTelugu]

🧚‍♀️🧚‍♀️🧚‍♀️🧚‍♀️🧚‍♀️

[@Raghhuveer]

You said that this is just a dummy example, how to use such methods in some bigger problems, say training a RCNN?