Tips Tricks 20 - Understanding transfer learning for different size and channel inputs

[@manuelsteele7755]

I have taken a couple of college courses in deep learning, and the course material goes so fast that details like this are missed. This video is excellent for truly understanding those parameters of deep learning present in the many layers of CNN. I am studying for a PhD in AI/ML at Arizona State. Thank you.

[@mariovrpereira]

your content is fantastic and puts together some pieces that sometimes get scattered during learning. thank you so much for all your videos

[@climbscience4813]

The thorough explanation is much appreciated! I think that this is not only for those with gaps in the basics, but also for people really trying to understand the details. Thumbs up!

[@theping1920]

this channel is underrated thanks sir for helping me through rough exams of my university. lots of love from Germany

[@DigitalSreeni]

All the best

[@Sehmiconductor]

Thank you for these videos. Currently I am in PGDBDA program by CDAC, Bangalore and was facing doubts in image processing section of deep learning. These videos helped me alot.

[@nagabilwanth5969]

Thanks for sharing the example to calculate the weights for 9 channels. This shows, you're sharing all your experience by making the video. I really appreciate it. Thank you.

[@DigitalSreeni]

Glad it was helpful!

[@wasiurrahaman7242]

Allah may bless you sir. I having watching your videos last 6 months, your teaching methode is so beautiful and very simple. Please make such video where we can learn from you.

[@falolayusuf9684]

Excellent video. You are doing great! Starting from the basics really helped with understanding.

[@vivek-159-icd]

Such a nice video, sir, even if the longer videos we are ready to see because the content is so beautiful. Thanks

[@DigitalSreeni]

Thanks, Keep watching

[@chitti9974]

very informative, best explanation with code execution. Thanks a lot Mr. Sreeni

[@DigitalSreeni]

You are welcome.

[@maddybharathi]

Thank you very much. Beautifully explained.

[@DigitalSreeni]

You are welcome!

[@oroszgabi]

Dear Dr. Sreeni! Excellent, as always! Could You explain how this methos works with other keras.apllications (e.g. ResNet, Inception, Xception, Mobilenet)? Are there any extra steps? Thanks for You kind answer!

[@kyawnaingwin8300]

Thanks you. It enlightens me how I can control transfer learning according to my needs.

[@DigitalSreeni]

You are so welcome!

[@ksa2011kify]

Thank you for sharing this informative video. Well done for getting to the point.🙏

[@PerryBattles]

Brilliant video! Question: why do we not need to update the parameter count/shape of the subsequent convolution layers if we reconfigure for a 1-channel image? Wouldn't those other convolution layers also have separate kernels for the different channels of input they receive? Do the subsequent convolution layers not have separate kernels for separate channels?

[@hemasowjanyamamidi6123]

Thank you so much for the detailed explanation Sir. I look forward to more such videos in future.

[@DigitalSreeni]

Keep watching

[@hemasowjanyamamidi6123]

@@DigitalSreeni Sure..! Suppose I am using mnist dataset whose image size is 28x28 then the updated vgg16 model for 1 channel is giving error # Negative dimension size caused by subtracting 2 from 1 for '{{node block5_pool/MaxPool}} = MaxPool[T=DT_FLOAT, data_format="NHWC", explicit_paddings=[], ksize=[1, 2, 2, 1], padding="VALID", strides=[1, 2, 2, 1]](Placeholder)' with input shapes: [?,1,1,512]. # Is there any way to resolve this without changing the congiguration of layers in the model?

[@jacobusstrydom7017]

Great explanation. Thanks for the hard work you put in to the videos.

[@DigitalSreeni]

My pleasure!

[@feeham]

Really appreciate your effort to explain this all. Thanks a lot.

[@DigitalSreeni]

You are welcome.

[@somalifairytales9365]

well explained sir, thanks for your time

[@nelbn]

Great video! Very useful!

[@DigitalSreeni]

Glad it was helpful!

[@Mohomedbarakat]

A wonderful video as usual!. Thanks a lot

[@DigitalSreeni]

Thanks again!

[@sophiez7952]

thanks! if I have a css file and I have 15 rows by 24 columns, which means 15*24*1, one channel, can I still apply vgg16 model without making width and height equal ? thanks!

[@mwurzer123]

great work, thank you for taking the time to make a 47 mins video instead of 15, while keeping up the quality! did you publish any papers concerning satellite imagery? greetings from hamburg

[@azamatjonmalikov9553]

Amazing content as usual, well done :)

[@plyap3872]

Thanks Sreeni for a very timely tutorial. I was figuring out how to use VGGNet to train my audio melspectrograms which has only a single layer.

[@DigitalSreeni]

Great to hear!

[@idrisseahamadiabdallah7669]

did it work for you ?

[@plyap3872]

@@idrisseahamadiabdallah7669 The features extracted from VGGNet not very good for audio spectrograms. Not good results.

[@jamescallanan6370]

Thank you. Unreal tutorial :)

[@MuhammadKhan-zu6fx]

I like your great explanation. One request, can you please give number to all your videos. it will help to watch all videos in a sequence. Thanks

[@Kpop11144]

sir i'm using densenet201 for training my model . my input images of size 224x224x1 but the input layer accept only of size 224x224x3 . how to solve this issue and suggest some solution

[@minhajulhoque2113]

Great video!

[@LAKXx]

Very helpful ! Thank you good sir

[@liou8493]

Hey Sreeni! Great videos as always! Your video on the VGG+XGBoost was a lifesaver for me! I was wondering how this would work if I had a sequence of say 9 grayscale images as input for the VGG16 model? Thank you so much!

[@Travellogy110]

Also i want to use two image pairs for camera pose estimation so i want to combine both image channels i mean 3 channel from one and 3 channel from other image ..is it will work as you demonstrate for 9 channel

[@metaforce9514]

Please sir I want your reply it's really important If we change the model architecture just by adding layers which also have learnable weights then how can we use the original model weights in this modified model

[@konstantinossardelis6023]

to input a 9 channel image, can't you just add a convolutional layer with 3 filters on top of the pretrained model ?

[@LAKXx]

This is probably a dumb question, but why did you use flatten instead of gavaragepooling before the dense layers ?

[@NutSorting]

very helpful video, sir what if we have train and test data set with different image dimensions?

[@idrisseahamadiabdallah7669]

nice video, thanks for sharing. By using the technique I can implement Resnet, AlexNet, ect... . right ?

[@oluwaseyibello5647]

Thanks for the video. This is great video with simple explanation. In a case you encounter unable to allocate 7.8Gib with shape (224, 224, 3). How do we solve it?

[@samiulbasirrajib3038]

How the output shape changes to 224*224 from 224*224?

[@jorgefelipegaviriafierro705]

I did this for 4 channels based on CWT (continous wavelet transformed) data obtained of several sensor measurements, but the performance of resnet50 and vgg16 has been quite poor :/ .... any ideas of why? maybe the features that the pretrained model stracted from imagenet are not good for cwt... seems unlikely right? I will keep trying to change the lr and the optimizer. I even removed one channel and leave the resnet as it is, with no great results. Great video btw, thanks.

[@aarifansari9097]

thank u, sir. very helpful video

[@giussepi]

Amazing content! thanks a lot for sharing :)

[@hemanshshridhar]

Great content Sir but i have a doubt whenever i use the vgg_updated as base_model i get the graph execution error sayingCudnn graph failed to build: UNKNOWN: CUDNN_STATUS_BAD_PARAM in tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc(3760): 'conv_op' CUDNN_BACKEND_OPERATION: cudnnFinalize Failed [[{{node sequential/vgg16/block1_conv1/Relu}}]] [Op:__inference_train_function_2696] .......... how to fix this any ideas ?

[@leonatay248]

Thank you sir. How do we preprocess it before feeding it to transfer learning models?

[@shanthisundar7773]

Excellent

[@falahfakhri2729]

Tremendous thanks, My question is, Is it possible to take the average of pretrained model to the UNet model?, Or this works only for the same pretrained model with the updated weights according to our input data, channels in this case?

[@DigitalSreeni]

You can transfer weights from anywhere as long as the shapes match.

[@amritasingh8953]

Thank you so much Sir...can I add new layer in the middle of network which takes previous layers weight . i am using pytorch

[@RanjitKumarMahto-s9f]

Thanks for such a great video. I tried to achieve the same for ResNet101 by updating the weights of 3rd layer(which is the first convolutional layer). But it doesn't seem to work as the updated model is giving some weird accuracy. Can someone please point out what I am missing here

[@zohaibrafique987]

Great Video, Can you please add a video for video classification using transfer learning?

[@chaitanyasharma6270]

what changes would i have to make for resnet50?works fine for xception

[@mohammadmasum4483]

amazing tutorial. Just to the point. You use square shape images (i.e. 1024 * 1024), however, can we use the same technique for non-square images, for instance, 3500 * 1500?

[@DigitalSreeni]

Can be any shape, not just square.

[@bijulijin812]

Is it possible to add more channels VGG model? If possible what could be the weights of new channel? may be we need to manually add random weights

[@DigitalSreeni]

This video exactly addresses your question.

[@fraoney]

great videos. i hope you can make a video on object detection using CNN with HOG and linear SVM classifier

[@learnopticalandmicrowavere7647]

Amazing content

[@DigitalSreeni]

Thanks

[@akashmahale7659]

hello sir, I have a numeric dataset of a crop and i want to perform transfer learning and do prediction can please guide me through it thank you sir, waiting for your reply

[@maanu1094]

by what name the file is uploaded in github

[@karamabdullah7867]

Best wishes

[@DigitalSreeni]

Thanks

[@KashifShaheed]

Great informative video , Thank You. Sir, I have a question If we only change the input layer shape to a single Channel. Could we used the same weight of pre-trained model for single channel or do we must update the weight for our modified model? Please, Sir clear my doubt.

[@DigitalSreeni]

If your question is going from 3 channel to single channel image, you can pick a single channel from the original model or average the 3 channels.

[@KashifShaheed]

@@DigitalSreeni yes. Sir. I mean to take the average of 3 channels and used as a single channel in new modified model. So that we can use for grayscale (single channel images). But, I think going from 3 to 1 channel directly using original model may be we will got error input size not matched.

[@DigitalSreeni]

Please watch this video, this is exactly what I covered.

[@hadjerbch830]

most helpful as laways !!

[@DigitalSreeni]

Glad you think so!

[@kashifqurashi9384]

mri image feature extraction..?

[@maitrysinha8947]

nice learning

[@tapansharma460]

Sir it’s a request that if you can elaborate 3D images like brats

[@martingosnell2937]

brilliant

[@OpeLeke]

this videos attempted to communicate an idea but you were all over the place, this video could be so much better with some structure

[@jacobusstrydom7017]

Thanks!

[@DigitalSreeni]

Thank you Jacobus for your contribution, really appreciate it. Please keep watching.

[@tetengious]

What a nice video but sir, how can we enhance medical images without losing the properties

[@DigitalSreeni]

I am not sure what you mean by enhance medical images without losing properties. May be you are looking for super resolution applications?

[@tetengious]

@@DigitalSreeni I am working on skin lesion disease classification (ISIC 2019 precisely). I am using histogram equalizer to enhance the images, which have to convert to gray, then to color. My question is: won't I loss the image properties doing that?

[@rubyli6109]

really a nice tutorial. Could you help me about how to use it, such as refining the vgg model or use the parameters learned directly? Thanks

[@kamaleshkarthi8586]

Really nice video.....can we do it in yolo v4 model?

[@DigitalSreeni]

Probably, I haven't done it myself.

[@kamaleshkarthi8586]

@@DigitalSreeni one more question if you're free answer me without fail.... Case 1: I trained a yolo v4 model with two classes. Now i has to train same model with adding another two classes. Train the model without losses of previous two classes weight...is this possible. My answer : reserving extra node in output layer. Can i do this? Your answer for case1 : Case 2 : Dataset description: 4k images with two classes and balanced classes. Using this data set i trained two model using tiny-yolov4. My question is: Model 1 : trained all 4k images. 20k max_batches . getting 84% accuracy avg loss 0.12xxx Model 2: Cycle 1 :i trained 3k images with 20k max_batch getting 94% accuracy. Cycle 2 : i trained 1k images with 20k max batch using last weight of cycle 1. After completion i am getting 94% accuracy. And avg loss 0.0xx. My question is both the model i trained same set of images why result is different. Training small set of image is good? Even though i increased model 20k+20k max batch thare is no improvement. Note: cfg file are same for both model. Thanks and regards, Kamalesh [e-mail]

[@Travellogy110]

@DigitalSreeni vgg_model = VGG16(include_top=False, weights='imagenet') vgg_config = vgg_model.get_config() # Change the input shape to new desired shape h, w, c = 1024, 1024, 1 vgg_config["layers"][0]["config"]["batch_input_shape"] = (None, h, w, c) #Create new model with the updated configuration vgg_updated = Model.from_config(vgg_config) ---- Error-ValueError: Input 0 of layer "block1_conv1" is incompatible with the layer: expected axis -1 of input shape to have value 3, but received input with shape (None, 1024, 1024, 1) please it sound working for you but i am facing this error...

[@Manisha-kp1ln]

i m also facing same problem

[@Travellogy110]

@@Manisha-kp1ln please use keras 2.2 it worked for me.

[@khaikit1232]

Hi may I ask what is the rationale behind averaging the 3 channels? I can understand that it is definitely better than just taking the weights of a single channel, but how did you determine that averaging was an appropriate method? After much googling on my own, I have heard of others introducing methods such as creating 2 additional channels for the greyscale image before feeding into the VGG16 model. What do you think of this?

[@DigitalSreeni]

What other options do we have other than averaging the three channels? I do not recommend picking one channel out of three as you will lose information. That leaves averaging the channel as a viable option. VGG16 expects a 3 channel image so the normal way of handling it for grey images is by copying the grey image into additional two channels. This is better than any other alternative.

[@yurcchello]

don't average conv matrix

[@davidsling]

24:50

[@vineetdave7323]

This is amazing content. Your concepts are super clear. Great help.

[@DigitalSreeni]

Happy to hear that!

[@azamatjonmalikov9553]

Amazing content as usual, well done :)

[@vincente_z6139]

Thanks!

[@DigitalSreeni]

Thank you very much for your kind contribution Vincente.