127 - Data augmentation using keras

Code generated in the video can be downloaded from here: github.com/bnsreenu/python_for_microscopists

[@RyanAngeloDagale]

Its a twenty minutes video and yet feels like five minutes, it is one of the KZbin tutorial that you don't get bored and stay focus.

[@sajansudhir1859]

Can't move to next video without appreciate your commitment and hard work towards beginner level coders.

[@ALFER38]

Watching this video clarified how data augmentation is used. Very good job!

[@DigitalSreeni]

Glad it was helpful!

[@baibhavchakraborty1560]

I have to do a college project on GAN image augmentation. Couldn't understand how to do. You came to my rescue. Thank you, Sir

[@marydayanaa9130]

Thank you so much sir. Very good explanation. I have just started watching all your videos.

[@harishgoud3628]

most underrated channel , love ur work

[@DigitalSreeni]

I appreciate that. Please help spread the word to your buddies.

[@souravjha2146]

Man your explanation is reallly good....thanks

[@fadilyassin4661]

you said you hope we learn some thing new , hell yes your so good in explaining , I did not see all of your tutorials i hope I can to me they are a refrence to many thhings in machine learning unless you understand the building blocks of ML you can't do any real work you and some few others are doing this amazingly big thank you

[@ademyoussef409]

clear and smooth explanation! Thank you!

[@mehrnazmrz4703]

That was awesome. Thank you very much for such a good tutorial.

[@cuthbertruseruka4461]

Thank you for the training. You are amazing.

[@user-nj2vi4pz5z]

The best learning tensorflow Thank you

[@vimalshrivastava5951]

Very informative video with clear explanation....!!!!

[@limzisin26]

Thanks Sir. This video helped me a lot in my final year project!

[@DigitalSreeni]

Great to hear!

[@darasingh8937]

Dude, you just became my guru! Thank you!

[@DigitalSreeni]

Glad to hear it!

[@saeedag4468]

You are an amazing teacher.. subscribed!

[@Thoughts_of_a_900_ELO]

Greatest youtube lecturer I have ever witnessed 🌝👌

[@DigitalSreeni]

Glad you think so!

[@saikrishnavadali05]

Amazing video!!! Sreeni is the best in AI!!! Very very helpful video..

[@DigitalSreeni]

Glad it was helpful!

[@saranbdsravi]

thank you for covering the various class label options namely single class, multi class within their respective folders. what if i had all the image classes within the same folder? grateful if you could give me pointers on how i can apply augmentation to the image of the weaker class. does the algorithm differentiate between weaker and dominant classes? my setting is all images in one folder and labels within a csv.

[@nguyenhai5455]

A greate video ! But I have a question. When you augmented the input images, What about the mask of those images, Did you also augment the masks base on those augmented images ? My project is semantic segmentation.

[@toninehme]

Thanks for this videos my dear!!! Can you explain which methods are used for NUMERICALS data augmentation?

[@yogeshrohit3147]

Omg 🔥🔥 this is it , what I wanted 💖

[@muhammadroshan7315]

sir should which fill_mode should we apply to segmentation mask of image (consisting of each pixel value corresponding to class with no background)

[@ashilshah3376]

Thank you so much, helped a lot !!!!

[@liwensong8037]

thank you so much, this is amazing!

[@saimahaider1783]

It was really helpful. Keep going.

[@DigitalSreeni]

Glad to hear that

[@EkaterinaGolubeva-pr9ih]

Thanks ! What if we have a later version of tensorflow and keras, should we downgrade our version to run your code ? How would we do that ? Otherwise, is there another way to have the same result ?

[@niladrichakraborti5443]

Very Helpful !!

[@asif1963]

I am new in DCNN. Moreover, I am working on image data augmentation and I have been writing code of augmentation. Moreover, I have 5 classes in the dataset i.e. Grass, Flower, Fruits, Dust, and Leaves Thus, the train set in is also consist of 8 classes. However, after augmentation, all of the augmented data has been store in the train folder but it does not store in their individual class. Besides that, I have been applied the directory manually for example: directory = ('/content/dataset/train/Grass'), save_to_dir = ('/content/dataset/train/Grass') Unfortunately, it does not work, and augmented images, not generated. Therefore, it is possible to store the augmented data in a similar folder where before augmented data exist (raw data folder)? if it can be possible individually (one by one class) that will be also very helpful for me.

[@MARTIN-101]

yes it can be stored. you are maybe saving the augmented data with the same names.

[@mamoon72]

first of all thankyou sir for another great video..❤ 2nd of all i have a question if we are augmenting a multi class problem how can we save augmented images in different class/folders because i saw you save augmented images from to folders in a single folder which will be very difficult to separate manually if i am working with a big dataset ? Thankyou in advance.

[@ShubhamKumar-xy6kj]

you got solution for mentioned query?

[@swathimenon2345]

Thank you for this video. Can you please help to know how to augment images that have multiple label classes already labelled. e.g. one image with different labels like traffic signal, speakers, traffic signs and lights. how to augment image with multiple labels?

[@mahalekshmianil2691]

Best Explanation,sir

[@DigitalSreeni]

Thanks and welcome

[@truongminh8571]

best tutorial ever! great!

[@DigitalSreeni]

Glad you think so!

[@sabaal-jalal3710]

Thanks Sir... very useful

[@DigitalSreeni]

Always welcome

[@mihretdesta9153]

hey sir how do you balance imbalanced image datasets for multi-class image classification using deep learning

[@wabisabi6680]

i have this Problem how can i do this after Augmentation with image generator save the image to his labels in the main Dataset (Multiclass dog-car-..) besucs wen i save the imags after use the function ImageDataGenerator I saw in the folder where the pictures are saved all the pictures are mixed together, because here I don't have a folder where cat alone and cars alone ... but was befor Augmentation all in his class thanks for your answer

[@_ishraqaldagamseh7664]

Great job but how can use this data in deep learning algorithms, or how can distinguish between the two classes to these data

[@ankitagrawal1046]

Sir, In last you have taken two folders of image and then augmented it but after augmentation if I want to save the augmented images in two different folders, how can we do that?

[@zakirshah7895]

hello teacher. I hope so you are doing well, Sir i just applied data augmentation on my dataset (just some simple augmentation) but when i see the output images it also changes the contrast and brightness of the images too i don't know why can you help me

[@adamt7367]

Hi! This is the best tutorial I have come across, I have 1 small issue I was hoping you could help me with! When I am training a folder of images, instead of augmenting and saving 64 photos, its saving over 500 photos, its saving each image 5 times over. Any ideas? Thanks!

[@helimehuseynova6631]

Hello, I am working with pytorch and I use resnet18 model for my dataset. I increased number of images for my dataset. I still get lower accuracy. What can i do for increasing accuracy of model?

[@mohammadmaaz731]

I am doing image augmentation in google colab, so do i need to save images in my google drive?

[@zakirshah7895]

Sir can you please make video on Test time augmentation for trained model

[@naveenfernando1852]

Hi Sir, How can we perform data augmentation when the dataset contains with imbalance data. Class A : 2000 images, Class B: 5000, Class C: 1000 images .. When we use flow_from_directly it augment all these images without any weight balancing Also in cases like this(multi class imbalance data) how can we customize the number of augmented images per class.. Because at the end, to create a accurate ml model we should input equal number of samples from each class right. Thanks and best regards.

[@DigitalSreeni]

Please watch my video on the topic of imbalanced data. kzbin.info/www/bejne/rIClk36ErM5pfdE kzbin.info/www/bejne/jILYe6l9id91ndk

[@tutankamon31]

Thank you.

[@ShubhamKumar-xy6kj]

when generated from the folder,how to retrieve the labels of each image

[@mehrnazmrz4703]

That was what exactly I wanted

[@gauravjha8986]

that is very good explanation

[@DigitalSreeni]

Thank you.

[@skullgaming5961]

I want to the augmented images to be saved in their respective class folders, how can i do that.

[@FanFanlanguageworld1707]

Hello, it's little confusing with the batch_size and i>20, which one defines the nb of image augmented? what's the role of batch_size ?

[@AlexDerBar]

I know I'm like a year late to this comment but if anyone still sees this, batch size inside the flow method is used when passing the generator directly to the model.fit() method when training a model. In that case the generator makes the decision on how many images it should generate per epoch/batch. That's my understanding at least.

[@ashilshah3376]

@@AlexDerBar thank you it helped :)

[@refreshyourlife3099]

Hi sir, please i have question, how i can saved augmented images with a specify named?

[@ravinirala9990]

what is use of batch_size = 16, it created 21 augmented image

[@govtjobs7063]

Sir please make vedio for augmentation of mri dataset....

[@reegee8321]

Hello, thanks for this video, quite helpful. I would like to perform augmentation on my target(labels). This are not images but different classes of my images. How can that work. Or can my newly generated images automatically create classes on their own?

[@DigitalSreeni]

Doesn't make any difference if your input is images or non-images. For images you have the luxury of generating new data just by rotation and other operations. For non-image data you think of what makes sense and apply that transformation. For example, you can randomly copy some data or drop some data.

[@reegee8321]

@@DigitalSreeni Thanks for the prompt response. But how do I perform augmentation on this type of data? It's a csv file (labels)

[@alamintalukder1691]

Thanks a lot

[@aomo5293]

Have y made a video showing how to work local images ?

[@zhannalopuliak8782]

Thank you!

[@lilianetalba4732]

Thanks God this Channel😭, i'm a health data scientist and i really need to learn how to programming better

[@DigitalSreeni]

Learning to code is fun and it is not difficult, if you put in some initial effort. Good luck.

[@lilianetalba4732]

@@DigitalSreeni thanks sir

[@iftiyarkhan7310]

great explanation

[@DigitalSreeni]

Glad it was helpful!

[@iftiyarkhan7310]

@@DigitalSreeni Hope to get more real-life projects examples with your great explanation. glad to be a part of your channel.

[@lilianetalba4732]

Thanks sir

[@divyaramesh1146]

I am getting value error when i am doing for directory. It is telling, i must have rank 4 and passed an array with shape. What should i do sir

[@sabaal-jalal3710]

I am working in a project with around 9k MRI images(including augmented images), my question is can I still use my MacBook Air normal CPU for deep learning image classification, or should I buy GPU? please advice... Thanks

[@DigitalSreeni]

MacBook Air CPU and resources is not enough for deep learning. I am not aware of Nvidia GPU that just fits into a MacBook which means you need a new system with GPU and this can be expensive. The best way is to try Google Colab and work with their free GPU.

[@sabaal-jalal3710]

@@DigitalSreeni Thank you... I will try that.

[@blblist]

First of all, thanks! Secondly, should not we expect the augmented data to be organised in two folders (one for each class as the original data is)? I mean, the augmentation aims to have more training data and how this will be possible if the generated samples are not labelled?

[@DigitalSreeni]

This video shows how augmentation is done and please do watch my video 150 where I warned about issues with keras for semantic segmentation. In that video I showed how you can save images and labels separately. In general, you define augmentation method and augment images of multiple classes along with labels based on the application.

[@blblist]

@@DigitalSreeni Thank you so much, that's kind of you!

[@ewananmo2404]

@DigitalSreeni Sir, I am trying to augment video data. I have extracted the frames from the video. In order for my data to make sense, I have to apply the same augmentation to all of the frames of a video. Can you please tell me how can I achieve that with ImageDataGenerator?

[@DigitalSreeni]

Fix the seed for augmentation, it performs same operations on each image/frame.

[@revanthkurakula9530]

thanks a lot

[@guidolippolis5436]

Hi Sreenivas, I'm creating a segmentation model using traditional machine learning and my dataset is composed by 53 images. Since I think I have to use half of the dataset as train dataset and the other half as test dataset, how can I improve the performance of the model? I can't use data augmentation because I have few images, according to what you said in the video. Thanks in advance

[@DigitalSreeni]

It depends on what size these 53 images are, if they are 256x256 then the dataset size is too small for deep learning. If they are 1024x1024 and if the size of your objects are relatively small and you have many of them, then you can divide each image into 256x256. I assume you are referring to semantic segmentation. You may get decent results with traditional machine learning (e.g., Random Forest, XGBoost, etc.). Do not forget to consider using pretrained networks as feature extractors. I did videos on these topics, for example, videos 194 and 195.

[@guidolippolis5436]

@@DigitalSreeni Yes, I'm referring to semantic segmentation. I have 53 images of people's eyes, they were captured by a medical device and they are 952x940 (or vice versa, I don't remember). My task is to create a segmentation model of the 3 regions: pupil, iris and infrared rays (which are projected to the pupil by the eye tracking device). Then, I have to calculate the pupil, iris and infrared diameters

[@muhammad529]

i have a folder in google drive which contains subfolders of images. I want to augment all these subfolders to the count of that folder which contain maximum images. i want to store all these augmented images in separate folder

[@poojithpoosa2109]

Can we use data augmentation for transfer learning and extract feature from the data then use it to classify using other ml model?

[@DigitalSreeni]

Data augmentation just transforms your input images. Therefore, you can treat them like any other regular image and use same approaches that you used for un-augmented images.

[@sajidhussain3618]

❤

[@nur__bijoy]

well explained. Instead of saving all augmented images within a directory, I want to save augmented images into respective classes. How can I do that?

[@DigitalSreeni]

Not sure what you mean, to your hard drive as different classes in various directories?

[@wi71630]

hi, when i run this code for multiple images i = 0 for batch in datagen.flow(x, batch_size=16, save_to_dir='augmented', save_prefix='aug', save_format='png'): i += 1 if i > 20: break i received an error ValueError: ('Input data in `NumpyArrayIterator` should have rank 4. You passed an array with shape', (0,))

[@NisarAhmad-ch3kc]

It means your program in not reading images from folder. You can also read images using cv2.

[@archimittal3810]

Hii,I am facing the same issue...Could you please tell me how did you rectify this error?

[@exploringMyself998]

Can you explain the significance of using batch_size in flow?

[@DigitalSreeni]

Please watch this video: kzbin.info/www/bejne/j17dpKqma76rnJI

[@neiltripathi3680]

Thanks sir ❤️

[@DigitalSreeni]

You are welcome!

[@LimLiYih]

Hey sir, really need your help. What if I want to do data augmentation together with PCA? Hope will get your reply!!!!! Thanks a lot

[@DigitalSreeni]

I never tried it as I never felt the need. But if you would like to perform augmentation with PCA then just do your PCA analysis first and redefine your X based on PCA. Then you can just augment the normal way using SMOTE or ADASYN. The keras way of augmentation is applicable to images but for structured data you can use one of the above to synthetically create new data.

[@hcmftm7862]

dear i cant load my image any help please

[@RiaD505]

not working on large dataset ..

[@chouchou2445]

hello! am i the only one who found not image and mask matching it does not pick the right mask befor augmenting please help !!!!!!!!!!!!!

[@chouchou2445]

found it ^_^ take a look on your augmented file if ever you had the same issue i can share with y what i changed

[@jagilamkumarchandra2419]

good

[@DigitalSreeni]

Thanks

[@nadeemnadeemakhtar4852]

Hello Sir.. Sir I am using ur code for my augmentation task.. But my images are grayscale.. And hence it is creating an error on SIZE..

[@nadeemnadeemakhtar4852]

If u can help me.. It would be a great pleasure

[@nadeemnadeemakhtar4852]

When I read images from my dataset.. It always generates an erro on rank 4..and doesn't dusplay channel dimension... How to resolve this.. Kindly help me

[@markcuello5]

PLEASE HELP ME

[@mihretdesta9153]

Please reply to me, sir ...

[@fatemehahmadpour711]

How convert Keras ImageDataGenerator into Numpy Array? image and label

[@DigitalSreeni]

Image data generator object give you a batch of images each time you call it. To walk through these batches of images you can use .__next__