Zero Padding in Convolutional Neural Networks explained

[@deeplizard]

Check out the blog for this vid here: deeplizard.com/learn/video/qSTv_m-KFk0

[@timharris72]

I have a question about the blog. You don't mention "stride" in either the zero padding or the CNN blog. A lot of the stuff you talk about assumes that the stride is 1 or 2. "Convolutions reduce channel dimensions" only applies when there is a stride of 2 or more or there is no padding. Have you thought about mentioning that these situations only occur when the stride is set to 2 (to reduce the parameters) or that zero padding to keep the output the same as the input only works when the stride is set to 1? It might confuse some people IMO if they read the blog and don't know that there are certain assumptions being made.

[@deeplizard]

Thanks for the feedback, Tim! We'll review the relevant blogs and see about adding this info.

[@bouchrad.339]

Perfect ! Thank you

[@justchill99902]

Guys, I think this KZbinr is one of the most hardworking KZbinr I have ever seen. Always timely popping links to the videos in between, always replying to the questions and we all know how awesome the content is! Keep it up! I am very thankful! Would love to help this channel make content some day in some way (when I have learnt enough lol).

[@deeplizard]

Thank you, Nirbhay! Awesome job going through the series!

[@hxong4733]

I agree with you!

[@bouchrad.339]

Exactly!

[@qusayhamad7243]

exactly the effort in these videos are very great

[@shouryapaul8515]

Honestly, this is one of the best channels for deep learning tutorials.Sooooo good at explaining things with deep insight. Most importantly, she demonstrates visualization of every concept.

[@CosmiaNebula]

0:50 example 4:14 convolution has smaller output than input 5:40 why zero-padding 10:30 Keras code

[@linknero1]

The doctors that were teaching us in the postgraduate courses, they didn't care, and didn't take the time to teach us useful information. Subscribed!

[@lone-warrior-13]

I couldn't understand conventional networks without your videos, thanks alot!

[@AmazingWorld-fw9oc]

Your channel is the reason I got interested in neural networks. Thanks for the great work.

[@NedSar85]

May you have a lot of wellness in your life... thanks

[@1sankey2]

Hey first of all I thank you for uploading this series, secondly "Deeplizard sounds cool and unorthodox" and lastly I liked the way you structured this entire series, short and crisp at the same time easy to understand and lot to learn for a newbie like me. Keep up the good work.

[@tymothylim6550]

Thank you very much for this video! The use of an example to explain the reasoning and concept behind zero-padding was great!

[@jisookim777]

This channel is a geml! Thank you!

[@ameylokhande9918]

Thank you for explaining this concept in such a lucid fashion. Your entire playlist is amazing. Got to learn a lot of new things with conceptual clarity.

[@kitty2002fly]

I love your channel! Have lots to read from lectures, but this just makes it clearer and gives very good interactive visualisations of deep learning theory.

[@divyanshoze8962]

Thank you, words will never be enough to thank you for what you provide us here.

[@smartguy3043]

Hands down the best Deep Learning intro.You have done an amazing job here explaining lot of concepts very clearly.Thanks a lot :)

[@akashtyagi7182]

Liked subscribed and bookmarked. I usually don't comment on KZbin videos but Just came across your channel today and the explanations are so good. You are making ML models no more black box. 👍

[@satyamrai2577]

This explanation is awesome. I understood the concept by only watching it once which is very unusual for me. Thanks a ton!!

[@steelday]

this style of explanation was precisely what I was looking at. Thank you!

[@walaakabbani]

thank you for the great and clear explanation !!

[@younessandi2414]

I'd like to like it twice! great job! I was thinking about this issue for the last two days, since I watched the video on CNNs explained, now I know how to combat it. thanks:)

[@junyin2216]

These videos are really straightforward and clear. Very helpful.

[@emanshlkamy37]

perfect and simple ! please keep going

[@Unknown-yl2yv]

Ma'am your method of explaining is amazing. Thank you for this masterpiece!

[@ThePanagiotisvm]

Thanks for this video! I really understood what padding is.

[@RamRachum]

I love your accent. You sound like the squirrel lady from SpongeBob SquarePants.

[@deeplizard]

😆😆😅 Funniest comment I've read in a while!

[@xxawsomenuke1xx]

hahahha I cant unhear it now.

[@AdrianConley]

Great video. Helped me get through a topic difficult to think about without visualization.

[@deeplizard]

Machine Learning / Deep Learning Tutorials for Programmers playlist: kzbin.info/aero/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU Keras Machine Learning / Deep Learning Tutorial playlist: kzbin.info/aero/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL

[@MrProzaki]

Awsome, u have a nice voice. Explained very well thanks.

[@harikrishnanv6233]

Thank you so much...got the intuition well...really helped me to understand these concepts easily.

[@qusayhamad7243]

thank you very much for this clear and helpful explanation.

[@deepaksingh9318]

Just an awesome explanation.. .. I am just loving ur videos.. Pleas upload more and more.... And keep it up..uh r just doing an awesome job.. 👍👍👍👍

[@deeplizard]

Thank you, deepak! Glad to hear this.

[@madhxxx5460]

Thanks a looooot. This padding issue was always confusing me. Thanks again!

[@mathatistics]

Thank you very much for so much clear explanation.

[@Praveenkumar-ol6ho]

Wow great explanation !!! I wish to give you a hug and say thank you

[@leonardodavinci4259]

Thank You!

[@timharris72]

That was one of the best visual explanations I have seen!

[@deeplizard]

Thanks, Tim!

[@vibekdutta6539]

Love the way u explain, wow! Thanks

[@Sikuq]

Excellent.

[@houssambanbino]

Just amazing how you explain !!!

[@RabiaAthar]

Well structured, any one can get the idea what CNN is and how they work (Y)

[@javierCi]

Thanks you very much!

[@deeplizard]

You're welcome, Castillo!

[@fathahcr7919]

{ "question": "To get the output layer the same as the input layer data, we need to set the padding value as?", "choices": [ "same", "valid", "zeropadding", "None of the above" ], "answer": "same", "creator": "Fathah Kodag", "creationDate": "2020-02-02T02:26:45.813Z" }

[@deeplizard]

Thanks, Fathah! Just added your question to deeplizard.com :)

[@ashifkasala6897]

She is Living Legend

[@adarshnbidari]

A deep explanation for a deep learning 😁😄, ver nice

[@DanielWeikert]

Wow, great work. Could you do even more sophisticated NN as well (GANS,GRU/LSTMs)? I guess the stride in keras is 1 by default but it's also changeable. The padding is kind of a hyperparamter as well or are there specific situations where you definitely recommend "same". (Suppose I do not know whether all my pictures have important patterns at the edge in advance)

[@deeplizard]

Thanks, Daniel! I have the other types of NNs you mentioned on my list as potential topics for future videos. Thanks for the suggestions. Yeah, padding is optional, and sometimes might not be necessary or desired. It is pretty conventional to use zero padding though. The top answer on this stackoverflow question sums up the reasons why this is: stats.stackexchange.com/questions/246512/convolutional-layers-to-pad-or-not-to-pad

[@richarda1630]

great job once again :)

[@kushagrachaturvedy2821]

Awesome video as always! Im just learning about CNNs and I wanted to ask about the 1st layer i.e. densely connected layer. What is it for and what kind of processing is it doing? Thank you

[@ahmedhusham7728]

Excellent tutorial

[@Garentei]

Beautifully explained.

[@ujjwalkumar8173]

i love you :) :).. Amazing videos,, Quality Content

[@RandomShowerThoughts]

Ill comment before watching because I know this video is going to explain this concept very nicely lol

[@RandomShowerThoughts]

Well I was right that was an amazing explanation keep up the great work

[@deeplizard]

Glad to hear your first instinct was correct, Farhanking! :D

[@RandomShowerThoughts]

Another thing, I just started using Keras today as opposed to Tensorflow where I learned everything from Recurrent neural networks to Convolutional Neural Networks, and Keras is extremely efficient. Thanks so much for introducing it to myself. And you guys should definitely do a video on recurrent neural networks and reinforced learning

[@deeplizard]

That's great! Thanks for letting us know! Have you checked out the separate Keras series on this channel? And guess what?... We're currently building an entire reinforcement learning series! Coming soon! 🤩

[@RandomShowerThoughts]

deeplizard oh man!! I’m genuinely excited I’m reading a topics from a book I’ll definitely be watching all your Keras videos. Reinforcement learning is for some reason uncovered on KZbin even though it is still in its super early stages. Keep up the great work.

[@aravindvenkateswaran5294]

You have an RGB image as the input. Does the filter size automatically recognize it and initialize with depth 3 implicitly? For example, the first layer filter would have the shape (3,3,3) Edit: I think I get it. Since its conv2d, you are using a dense layer to just pass in the r and g channel simulating as 2d image samples whilst the b channel acts like feature columns. Do you think using a 2d black and white image would be clearer to show? Am I understanding this right. Awesome video btw!

[@supamdeepbains5172]

Great Content, so does that mean we should always use same

[@yuxiaofei3442]

Amazing is all i have.Really helpful.

[@UrontoShahid]

I'm in love with your explanation, your voice and of course you I guess.

[@peschebichsu]

Nice video! What's that last inteher after the size of the image (e.g. The 16 [after the size 20x20] at 11:45)

[@xxawsomenuke1xx]

This is a fantastic video. I was just wondering is there any negative side to using zero padding?

[@thespam8385]

{ "question": "The output size of a convolutional layer (assuming stride = 1 and dilation = 1) can be computed as follows (remember that adding zero padding augments, in this case, image_size):", "choices": [ "(image_size - filter_size + 1) x (image_size - filter_size + 1)", "(filter_size - image_size +1) x (filter_size - image_size + 1)", "((filter_size ÷ image_size) +filter_size) x ((filter_size ÷ image_size) +filter_size)", "((image_size ÷ filter_size) +filter_size) x ((image_size ÷ filter_size) +filter_size)" ], "answer": "(image_size - filter_size + 1) x (image_size - filter_size + 1)", "creator": "Chris", "creationDate": "2020-02-06T04:49:40.495Z" }

[@deeplizard]

Thanks, Chris! Just added your question to deeplizard.com :)

[@trannhiem1639]

Hi, There your series of Videos I definitely love all of them, in this Video I a bit wondering "is that after each hidden layer that will reduce by arithmetic progression? and from that, if I add more hidden layer our output dimension will decrease dimension after that (By arithmetic progression) and what if I add the zero padding in some layer (ex: I add in the second layer is there any problem with our CNN output ) Thank you so much for your time and love all your videos again!!

[@stydras3380]

first of all: This course and other related videos are wonderful, currently binging thid series. I was trying some stuff in parallel on google colab trying out keras and i was wondering: Is it possible to shift the "basepoint" of the filter in keras? For example if I am considering sequential 1D data I'd want to convolute some datapoints to the left of my basepoint: In that case we'd only have to pad out data to the left (where data long in the past isnt as important as in the future), while on the right the last convolution to be calculated is a close to the edge of my data and takes more datapoints from our data than when we were to 0 pad (where wed only get half the data)

[@fritz-c]

I spotted a slight typo in the code in the article for this video kernal_size= ↓ kernel_size= (in six locations)

[@deeplizard]

Fixed, thanks Chris! :D

[@houmanjafari2963]

Great explanation !

[@jonycastagna1271]

Very good video, brava!

[@yoyomemory6825]

nice lecture! thank you so much!!

[@j.a.1776]

Nice visualization!

[@deeplizard]

Thanks, Julien!

[@tostupidforname]

Is it possible to add paddings for a uneven size reduction? Wouldnt that "shift" the image in one direction because padding is only one one side and therefore change the pattern?

[@anandhu5082]

11:00 So each convolutional layer has exactly 1 filter? (Or can we have more than 1 filter in each layer??)

[@shyamsuresh5700]

Thank you so much!

[@owaguugochukwufranklin3294]

awesome

[@hamidraza1584]

How to specify the numbers of hidden layers and kernals in neural network ?

[@alaapsarkar]

Great video! I have a question, what is the significance of zero? Can we initialise the padding with random numbers? I think it might add info which we might not want, idk just guessing.

[@deeplizard]

Yes, if we use non-zero values, then it will affect the output of the convolution operation in a way we wouldn't want. There wouldn't be a way to distinguish if the non-zero numbers were used only as padding, or if these numbers were actual data from the images.

[@theone3746]

does the values in the filter matter? If so how do you determine them?

[@deeplizard]

The filters are randomly initialized, and then optimal values for the filter are learned during the training process.

[@muhammadshoaibsikander2603]

Superb!!!

[@pavankumarkanaparthi213]

In the code, you have used a dense layer first but in CNN we have to use convolution layer first so that we can take the whole image. Can you why you have taken a dense layer first?

[@rumeetsingh377]

thank you very much :)

[@sgt.mcgragon359]

halo, 1.what is the first and last term represent in output shape?13:04.....example: (none, 20, 20, 16) ... 2. Since we have one dense layer and 3 conv layers in this example, the neural network looks like one input layer, then 4 hidden layers followed up by an output layer...right?...so total 6?

[@deeplizard]

1. The batch size and channels. Here is a resource: deeplizard.com/learn/video/k6ZF1TSniYk These axes can shift around depending on which framework is being used. 2. Don't forget about the input layer. It is implied. Also, the flatten operation is questionable. Pooling is another one that is questionable. Some people call those two layers. However, they don't have any weights (learnable parameters), so some people just look at them as operations within a layer or inside the network.

[@tylersnard]

Can you do a video for RNN's please?

[@prakashr75]

i did't get with the first parameter what does it does in the convolutional function

[@vibekdutta6539]

Ma'am the actual question is that how do I shuffle the images along with the labels. If I shuffle the list of images alone the corresponding labels will get changed, so how do I shuffle both the labels and image together. I tried using the zip() but didn't work. Please help.

[@muzammilayaz4352]

what does the comment flatten() does?

[@deeplizard]

See this one: deeplizard.com/learn/video/mFAIBMbACMA

[@alihussainkhan4459]

I have one question can anybody help, does it add padding after passing through each filter to make it ready for the next filter? or it adds all the required padding at the start?

[@Waleed-qv8eg]

Keep it UP!!

[@Ysiek72]

Nice videos, thanks! But how do we calculate the output size if the stride is not = 1?

[@deeplizard]

Hey Braveness - Suppose we have an n by n input. Suppose we have an f by f filter. Suppose we have a stride of s and a padding of p. The output size is given by this formula: ((n + 2p - f) / s) + 1 by ((n + 2p - f) / s) + 1

[@Ysiek72]

thanks, deeplizard! your videos are perfect btw!

[@yeahorightbro]

Mate - great videos!

[@deeplizard]

Thanks, Daniel!

[@khoansopheaktra5512]

video starts at 6:45

[@GauravSingh-ku5xy]

Hey, I was wondering if it's a good idea to ALWAYS use zero padding. Is it?

[@vidumini23]

❤️❤️❤️❤️

[@dikshajadhav8276]

what is third element in shape and what is param , how they are counted

[@deeplizard]

Hey diksha - The parameters shown in the summary of the model represent how many _learnable parameters_ are in each layer. Learnable parameters are the weights and biases within the network that stochastic gradient descent is working to learn and optimize their values to achieve a minimized loss. The number of parameters in the network is determined by several factors, including the numbers of layers, the number of neurons in each layer, the number of convolutional filters, the size of the filters, etc. That being said, the calculation for the number of parameters in a convolutional neural network is a bit more involved than in a network that only contains standard dense layers. I may do a video that shows how the calculations are done, but in the mean time, here is a simple example for how they're done for a network containing only dense layers. Think of a model that has an input layer, the first hidden dense layer with 16 nodes, the second hidden dense layer with 32 nodes, and the output layer with 2 nodes. First hidden layer parameters = 16 weights + 16 biases = 32 parameters Second hidden layer parameters = 32*16 weights + 32 biases = 544 parameters Output layer parameters = 32*2 weights + 2 biases = 66 parameters Total parameters = first + second + third layer parameters = 32 + 544 + 66 = 642 total parameters

[@deeplizard]

Hey diksha - I just released a video on this topic and thought to circle back around to you to share it. kzbin.info/www/bejne/ppiWmX2miNSjfrM

[@avishekhbt]

Awesome!

[@sathish1078]

thee Best explanation

[@MuhammadUsman-ln5ov]

My Question is when i use this "model_same_summary()" then it give me this error. --------------------------------------------------------------------------- NameError Traceback (most recent call last) in ----> 1 model_same_summary() NameError: name 'model_same_summary' is not defined i am beginner someone please help me regarding this. thanks.

[@veeraganesh9407]

How is 1s padding different from 0s padding?

[@matharbarghi]

There is no explain about what are these numbers (20,20,3). Only mentioned that our input shape is 20 by 20 pixel, but what is 3 here? We know that an input for CNN is this format [batch_szie, chanell, hight, weight] and so ...

[@deeplizard]

How you refer to the input shape changes across APIs. 3 is the number of channels (RGB). Keras expects the input data shape to be specified as (height, width, channel) using the input_shape parameter in the first hidden layer of a model.

[@atisafarkhah5923]

perfect!

[@neerajkumar188]

Impressive

[@deeplizard]

Thank you Neeraj!

[@neerajkumar188]

Its been 2 years and I am here again. You are creating really great content.

[@shanerooney7288]

I thought this would going to be about *NOT* padding (having zero padding) But instead it was about padding _with_ zeros.

[@deeplizard]

000000 0 🦎 0 000000

[@shanerooney7288]

What I expected: 🦎 What I got: 000000 0 🦎 0 000000 What I really needed: *from* internet *import* knowledge

[@deeplizard]

Haha love it!

[@aashalkamdar9576]

input_shape = (20,20,3)...................what does this mean?

[@deeplizard]

Hey Aashal - It means that the CNN will take in input which has dimensions of 20x20x3. This is height x width x color channels. We have a video and blog that covers this topic in detail here: deeplizard.com/learn/video/k6ZF1TSniYk

[@aakashnandrajog7035]

what if the original image is 28 by 29?

[@sepidet6970]

Aakash Nandrajog 26*27

[@danielegreco1925]

More aggressive explanation (in tone) ;)

[@Views-qf2eg]

Thank you!