How to deal with Imbalanced Datasets in PyTorch - Weighted Random Sampler Tutorial

[@AladdinPersson]

A tip that I didn't mention in the video is when you're iterating through the dataset to create the sample weights is to iterate through dataset.imgs rather than just dataset. This will run much faster because we are not resizing, performing transformations and so on which we do not need to do when we are only interested in the labels of the examples.

[@amnesie148]

Hi Aladdin , First of all thank you for the super good video, but I didn't figure out which part should be " iterate through dataset.imgs ", can you point it out?

[@imveryhungry112]

the pytorch weighted random sampler is an amazing pytorch feature. Thanks for talking about it here.

[@sahasamanecheppali547]

You sir, deserve way more subscribers for the consistency and the diversity of the topics you choose. Keep up the good work.

[@wolfisraging]

Awesome video bro, this has been really helpful. I'd like to share one trick of mine: Applying more random and strong data augmentation to the examples that are limited, and less random augmentation on examples that are quiet enough, and then making sure that each batch to the model receives an equal number of examples for each unique label. The only side effect here would be that you'd have to write your own custom dataloader that does that 🥱, and to be honest, it's not easy 😂, but once you set it up then its just a matter of copying and pasting for next projects :) Thanks again for the video.

[@AladdinPersson]

Hey Wolf! :) That sounds interesting, I would like to check it out if you could you share an example of such a custom dataloader?

[@rosacanina674]

Thank you so much for your great content! I was wondering if the loader in the video is the train_loader? do you also apply oversampling on dev_train and test dataloader?

[@jacoblee6246]

Great video！A small bug: The orders of traversal between os.walk and datasets.ImageFolder are different. In the github code, We cannot guarantee that the smaller number of samples will get a greater sampling weight.

[@yashrunwal5111]

Hi, how can we use the WeightedRandomSampler for Object Detection task?

[@mochametmachmout4467]

To tackle the imbalance one can also use the Focal Loss function. The kornia library has it.

[@rafaelmahammadli667]

Hello, How can we apply under-sampling on your case? Oversampling and undersampling can be applied together? Thank you

[@Zulle863]

Does we have to call get data loader for once for train and once for test set or just a single time for the entire dataset?

[@kirankharel929]

Thanks Aladdin for all your videos, they are really awesome and informative

[@AladdinPersson]

Really appreciate it bro

[@yanfeng5519]

Great lecture.

[@clariolee]

人像加入视频时候舒服多了！！不知道看哪的时候就看脸哈哈哈。

[@fasolya99]

why did you multiply the sample_weights (which is zero) by len(dataset) ?

[@frankrobert9199]

Great lectures.

[@thantyarzarhein5459]

Awesome video and this channel is so underrated in DL community. I would like to know if there will be paper implementation tutorials in the future ?

[@AladdinPersson]

Yeah for sure. Any paper in particular you'd wanna see?

[@aishwaryaagarwal3540]

@@AladdinPersson arxiv.org/pdf/1905.05908.pdf This one if possible. Can you make some tutorials on compositional zero-shot learning and implement some papers in that field.

[@0730pleomax]

Thanks Aladdin! Would you mind recommend us your learning resources? I mean most of your teaching content are pretty rare in any ML/DL book.

[@AladdinPersson]

I'm just sharing solutions to problems I face. Googling, reading docs and github, no good learning resource there unfortunately

[@hassanrevel]

Thanks man

[@sebastianamaruescalantecco7916]

There is a bug in the program that can cause you a lot of trouble. After line 22 add this line of code -> subdir.sort() os.walk() won't traverse your folders in alphabetical order so you have to sort the generator before appending the calculated weights. The could should look something like this: for root, subdir, files in os.walk(root_dir): subdir.sort() if len(files) > 0: ....

[@bryanyan8346]

The same problem arises for me, thanks for pointing out!

[@hamidmahmoodpour3659]

why we can not use shuffle while using sampler?

[@ZulkaifAhmed1]

Dude you are awesome. I like your pytorch tutorials. But would love if you could use google colab for next ones.

[@mariaanson6537]

What is the editor or Ide you are using

[@nishantyadav6341]

Aladdin, you deserve more subscribers. And you need to charge more :) Just joined as a member.

[@AladdinPersson]

Appreciate you bro

[@helimehuseynova6631]

Hi , If we have 3 classes, how can we do it ?

[@kelixoderamirez]

permisiion to learn sir. thank you

[@sahil-7473]

Superb!! I implemented AugMix DataAugmentation myself to increase the minority label's samples. My question is should we stick around one data augmentation technique which is state of the art OR we should try all of the others technique? Thanks

[@AladdinPersson]

I prefer to use RandAugment

[@mustafabuyuk6425]

Randaugment is one of the best augmentation method, it will improve your model performance and I was using weight normalization like this nSamples = [346,168,106] # class samples normedWeights = [1 - (x / sum(nSamples)) for x in nSamples] normedWeights = torch.FloatTensor(normedWeights).to(device) print(normedWeights) nn.CrossEntropyLoss(weight=normedWeights) is your second method different than this

[@AladdinPersson]

I prefer oversampling (and you can still use RandAugment) rather than class weighting as it seems you're doing in the example

[@НиколайНовичков-е1э]

Thanks!

[@TheAcujlGamer]

Great video!

[@Tripdin]

Thank you

[@kirtipandya4618]

How can you use pytorch and tensorflow so well. You must have invested so many hours of practice. How do you do that? Which API you like more for ML? In Keras you handle images differently and in pytorch you handle differently. It amazing to see that you have mastery over both. Well less people can do that. Usually people like to work with only one API.

[@AladdinPersson]

I like PyTorch more. I think you overestimate me for sure, I'm way worse than what you think and need to google everything. You just see the refined version in the video and not the horrible mistakes I make :)

[@jimmychen4796]

@@AladdinPersson You are too humble bro! A nice lesson learned: ), good job, and keep going!

[@shaharweksler1203]

@@AladdinPersson maybe you can make a video about how you get stuck and your train of thoughts and google queries to solve it

[@mattiagatti1200]

Hello, I still don't understand how replacement = True differs from replacement = False, may you explain me it please? Thank you :)

[@erdi749]

Another amazing tutorial! I wonder if you have a patreon page where we can support you, it is well deserved. I have a question as well. I have a large imbalanced dataset. I need to call getloader function to get train and test loaders for hyperparamter tuning. Scanning the entire trainset in each function call makes the code slower. Do you suggest a work around? Thank you!

[@amirhosseindaraie5622]

Hi Aladdin, Your code does not work and the result is still imbalanced. What should we do?

[@AladdinPersson]

Why does the code "not work"?

[@jijie133]

Great video！

[@sakib.9419]

I've been trying to do this in TensorFlow with ImageDataGens for over 5 hrs now, Could you help out?

[@AladdinPersson]

I'm not sure how you do it in Tensorflow

[@sakib.9419]

@@AladdinPersson That's fine, d'you have a discord btw?

[@apurbasarkar6918]

how to do it in tensorflow? I'm struck

[@tirthadatta2072]

Can u give me solution about computational cost or memory allocation problem related to oversampling imbalance dataset. It is too hard for me to buy gpu because of high cost and it is also problem for many of us. Can anyone here give solution related to the issue?

[@유영재-c9c]

dataset link over!

[@robinranabhat3125]

Recap at 9:00

[@seanbenhur]

Bro..are you in linkedin!?

[@AladdinPersson]

Yeyup, same name :)

[@seanbenhur]

@@AladdinPersson Request sent😌

[@ArunKumar-sg6jf]

Do in TENSORFLOW also bro

[@AladdinPersson]

I don't know a good way to do it in Tensorflow... :\ Maybe someone else knows and can help out?

[@wolfisraging]

I've been using it in almost every other projects... but never understood what's so random about it? Why random in its name? 🙂

[@AladdinPersson]

Donno actually 🤔