MULTI-LABEL TEXT CLASSIFICATION USING 🤗 BERT AND PYTORCH

[@theartificialguy6898]

Hey guys, i messed up the part where I was supposed to get the labels but rather calculated as multi-class problem using argmax..., what you can do is just make a custom function and check for each output probability that if it is > 0.50 (or your own threshold), show only those (assign label 1), else don't show (assign label 0).

[@kachrooabhishek]

from multi label to multi class we only need to change the loss function right ?

[@javed514]

The Artificial Guy, the video is much appreciated. You made a lot of things simplified and explained things in a very useful manner. Liked and subscribed.

[@theartificialguy6898]

Thank you for your kind words, keep supporting!!

[@anupamadeo2455]

I have a doubt. I am new to this but in the last part where you show an example and do argmax to get physics how is it multilabel there?

[@masterevil3347]

Guy, I have the same doubt with you

[@theartificialguy6898]

Yes you are right, you don't have to use argmax there, or you can just define a custom function let's say, if prob > 0.50, show those labels, else don't show.

[@sshot1000]

Thanks for the video. I see your input data contains multi-label (not just multi-class) and you used argmax function on the output, which gives only single label, missing multi-label predictions.

[@nicholast.7297]

Hey i want to ask, why you keep extending val_targett and val_outputs list? Wouldnt it be better if we just set outputs to empty every iteration and set target once in the beginning? Thanks

[@devanshchandrawat8851]

Dude the video is great. just a suggestion, for people who are not familiar with OOPS this video will cause complications. But explained simply and in a great way

[@zhivebelarus560]

Thank you for great work on this video!

[@JXiong9999]

Is the train and test data sets avaible?

[@MY-qc4wi]

hi thank you for the great video! I have one question, how do I create ckpt_path and best_model_path on S3? I tried to apply your code in Sagemaker but ran into some errors because Sagemaker doesn't recognize these two path.

[@Shivanroy]

please provide test dataset as well.

[@hudaalfigi2742]

thanks for your great explaination, but i have question how can i use the best weights that i have saved to train another model with same dataset?

[@javed514]

Can we show some more metrics in this model?

[@runjhunsingh2348]

tried just everything but getting 38% hamming score accuracy on my multilabel classificastion of 24000 dataset into 26 labels, please suggest something

[@ShaidaMuhammad]

Please make a video on Multi-class text classification as well.

[@theartificialguy6898]

Sure !! Do like and subscribe to the channel !!

[@ShaidaMuhammad]

@@theartificialguy6898 please make the video. I'm stuck on this problem from the last 2 days. I'm following your multilabel code for multiclass.

[@theartificialguy6898]

@@ShaidaMuhammad hi, the video is out, do check it out and hit the like button: kzbin.info/www/bejne/raGcc6iajqyNr6M

[@ShaidaMuhammad]

@@theartificialguy6898 okay

[@nawelyala7431]

Great video bro. Could you share with me the test data, or show me how can I scrap such a data. Thanks

[@nawelyala7431]

I found the test data, but if you know a method to scraping such a data, I'll appreciate your help.

[@theartificialguy6898]

@@nawelyala7431 hey, glad you figured it out! Also, you can obtain these type of data from kaggle, or if you really want to scrap data yourself, you can use various APIs with the help of which you can have access to certain data you require, also you can use BeautifulSoup library to scrap the data directly from a website.

[@amitpatel3071]

@@nawelyala7431 where is the test data ?

[@wyldcard00]

the trained_model line doesnt work, raises a NotImplementedError

[@machinelearningwithrohitna6089]

bro where's the dataset

[@carloscampos7709]

Awesome video! Thanks!

[@luiscao7241]

Hi The AI Guy, Thank you very much for the video. Can I ask you what is the format of multi-text classification ?

[@ryadbelkacemi6893]

Hello, thanks for your tutorial, i actually have a question. Why did you do "val_df = train_df.drop(train_df.index).reset_index(drop=True)" ? Because when i do it on my own version of the dataset (a custom one), it give me a dataframe with no rows, and then when i execute the training algorithme it raises a div per 0 exception. Could you please just help me figure out the format expected at that point ? Thank.

[@theartificialguy6898]

The idea was to take 80% of train data (randomly) and left 20% for val data, so as you can see in that cell, first I set the train_size = 0.8, then I randomly sampled 80% of data from train_df (You know what you should do, just rename your main df to something different like "df" instead of train_df when loading the data. Because you are (me) setting the 80% sampled train_df itself to train_df in that cell so that is why it is showing 0 in your case. And in next line, you are creating your val_df with what's left (excluding the indexes of train_df data).

[@ryadbelkacemi6893]

@@theartificialguy6898 Thanks for your response, i've finally figured it out :)

[@theartificialguy6898]

@@ryadbelkacemi6893 great 😀 don't forget to like and subscribe if it helped you 😀

[@shawnding1703]

Thanks so much for this wonderful video!!!!!!! I would like to know after the training process as: trained_model = train_model(EPOCHS, train_data_loader, val_data_loader, model, optimizer, ckpt_path, best_model_path) How can I apply the model to the test the test_dateset to see the accuracy and what kind of label will be labeled to the test_dataset? I will be very grateful if you could tell me how can I apply the model to test_dataset.

[@theartificialguy6898]

Hey, glad I could help! Just want few things in return, like, subscribe and share among your friends :). Okay, so to test on an example, you can head over to this video's description and go to the GitHub code. There in the last cell I've showed how to test on a single test sample. Now to make predictions on a whole test data/df, create a dataloader like "train_data_loader" and "valid_data_loader" , then use the code snippet below "Validate the model" comment. Hope this helps you!

[@hilalpataci]

@@theartificialguy6898 Hi there, same question here :) do you have any plan to get f1 for each label in test data? ps I already subscribed and you are a star in that domain :)

[@theartificialguy6898]

@@hilalpataci thanks a lot for your kind words and sub ♥️, coming back to your question, yes I will definitely make some changes/add the logic to calculate every metric including F1 score, accuracy etc. Keep sharing :D

[@theartificialguy6898]

New video is out in which I showed how to handle imbalanced text dataset and calculate various metrics like accuracy, precision, recall, f1-score. kzbin.info/www/bejne/q5Pbl4qGnJeXgJI

[@nikhiljadhav2580]

how do we apply GCN to this

[@Sara-he1fz]

This is supposed to be a muti-label classification but at the end, you take the argmax and find one label for each example. How can we find the final multi label for each example?

[@theartificialguy6898]

yes i messed up that part, what you can do is just make a custom function and check for each output probability that if it is > 0.50 (or your own threshold), show only those (assign label 1), else don't show (assign label 0).

[@Sara-he1fz]

@@theartificialguy6898 There is also no calculation of accuracy. You referred the output probabilities as the accuracy which is incorect.

[@theartificialguy6898]

@@Sara-he1fz yes, as it was a quick application i wanted to show, so i didn't include calculating the accuracy of model, there is already a video that I've uploaded regarding calculating accuracy of model or you can also find other resources on KZbin... I referred probabilities of each class as accuracy and not accuracy of the "model", i didn't mean to misguide...

[@junderfitting8717]

Anyone know why we shuffle train_dataset but don't shuffle validate_dataset?

[@luukkwee8295]

Hi The AI Guy, thank you very much for the video. Do you perhaps know what went wrong? I got the following message: "TypeError: dropout(): argument 'input' (position 1) must be Tensor, not BaseModelOutputWithPoolingAndCrossAttentions"

[@machinelearning3518]

Set return_state_dict false in hf bert model

[@etsuyu2680]

How do I load the model after I have saved it? with the checkpoint and save model, can you give me a sample code on how you would do it?

[@theartificialguy6898]

Hey, check the description for the code and there you can find a function called "load_ckpt" that's the utility you want! And do like the video and subcribe to the channel :D.

[@zahrasiraj106]

hi can we use hierarchical multilabel text classification ? what would be the process for further steps from this multilabel text classifier can you guide .thankyou

[@theartificialguy6898]

Yes we can use it, will provide some resource for you. Hope it helps : github.com/RandolphVI/Hierarchical-Multi-Label-Text-Classification

[@zahrasiraj106]

@@theartificialguy6898 thankyou for this link but i already have this link available i am a new bee and i actually want to improve heirarchical cluster labeling using text classification technique is this possible technically or logically ?

[@theartificialguy6898]

@@zahrasiraj106 yes why not, but at the moment, I'm not so sure.

[@zahrasiraj106]

@@theartificialguy6898 can you help me in thig regard if we can work on it then may be i would be able to write a good journl paper including your name

[@alizhadigerov9599]

Your val_df is empty DataFrame. That's why this line throws division by zero error: trained_model = train_model(EPOCHS, train_data_loader, val_data_loader, model, optimizer, ckpt_path, best_model_path)

[@shahed8762]

how to fix that?

[@chaitraashok1956]

where you able to fix that?

[@alizhadigerov9599]

@@chaitraashok1956 I don't remember

[@varshiniuj4379]

Hello the AI guy, I tried to implement this with my dataset of 8 classes and one column of context. But when i am training the model with train_model, I am getting an error from the loss function. It states that "ValueError: Target size (torch.size([32,8])) must be the same as input size (torch.size([32,6]))". Kindly let me know where I need to do the correction as none of the methods in the internet works for me. Thanks in advance.

[@sarthakjain5973]

in your BERTclass constructor change the output from 6 to 8.

[@varshiniuj4379]

@@sarthakjain5973 Thank you

[@meceffeukada3767]

How can i evaluate the model using F1 Score ?

[@theartificialguy6898]

Check out this video: kzbin.info/www/bejne/q5Pbl4qGnJeXgJI

[@cCcs6]

Can someone explain me this: train_size = 0.8 train_df = train_df.sample(frac=train_size, random_state=200).reset_index(drop=True) val_df = train_df.drop(train_df.index).reset_index(drop=True) I dont understand the 3rd line. Like that val_df has only column names without any data -> len(val_df) = 0 and my validation is leading to a ZeroDivisionError. Shouldnt it be something like? val_df = train_df.sample(frac=0.2, random_state=200).reset_indext(drop=True)

[@vanyabhatia1554]

Also for the problem of empty val_df use this train_size = 0.8 train_val = train_df.sample(frac=train_size, random_state=200).reset_index(drop=True) val_df = train_df.drop(train_val.index).reset_index(drop=True) train_df=train_val

[@leabryan5717]

his code has problems

[@shahed8762]

got zero division error trained_model = train_model(EPOCHS, train_data_loader, val_data_loader, model, optimizer, ckpt_path, best_model_path)

[@RominaDeLuise]

I also get this error! ZeroDivisionError: division by zero valid_loss = valid_loss/len(validation_loader) Were you able to resolve this error?

[@RominaDeLuise]

The problem is that *val_df is empty* You can solve this by replacing: train_size = 0.8 random_state=200).reset_index(drop=True) val_df = train_df.drop(train_df.index).reset_index(drop=True) by these lines: train_df2 = train_df.sample(frac=train_size, random_state=200) val_df = train_df.drop(train_df2.index).reset_index(drop=True) train_df=train_df2.reset_index(drop=True)

[@shahed8762]

Thank you, yeah solved that later one.

[@karthikreddy-vy6ym]

Hi, I am trying to implement bert, but my loss is not reducing at all. my batch size is 10 and epochs is also 10.

[@theartificialguy6898]

There could be many reasons, first you should always do EDA on your dataset, try to remove stop words, punctuation, etc and try multiple approaches to solve the problem.

[@karthikreddy-vy6ym]

@@theartificialguy6898 yes, I did reduce my learning rate to 1e-6 and added L2 reg along with better stop word and special character removal, got better result once, but its very inconsistent. Now I am not able to recreate the training result.

[@LuxinWang-j5i]

Hi Artificial guy, thank you so much for the videos, and it's really helpful for my study, but I met one problem when I try to run the code by myself. After I ran the train_model, and save in the best_model_path, I try to apply the model to my test data by using: model = torch.load(best_model_path), model.eval(), it returns that 'dict' object has no attribute 'eval', and the type(model) returns . Can you help me with this? thank you soo much:)

[@omerahmaad]

I am getting IsDirectoryError

[@maliha_abroad]

same. did u fix it?

[@omerahmaad]

@@maliha_abroad I did I can send you the code

[@ricardodiaz8964]

@@omerahmaad how did you fix it?

[@omerahmaad]

@@ricardodiaz8964 I have to check the code

[@omerahmaad]

@@ricardodiaz8964 is it possible if you can email me

[@ysjang05050]

Hi, The AI Guy. Thank you for your informative video, thanks to that I could practice with my simple data set. And one thing I wondered is that how you mount the curr_ckpt and best_model.pt file into your model. Youd dowloaded the files locally on your PC from somewhere and mount beforehand? I didn't see that part in this video so...

[@jarorkwong8042]

Once you create a new model instance, you can load the checkpoint .pt file directly into it by using the syntax "model.from_pretrained('xxxx.pt')".

[@ysjang05050]

@@jarorkwong8042 Thank you!

[@vishruthdilip]

Hi, great video. Could you please explain how data imbalance is handled?

[@theartificialguy6898]

Thank you, and yes I am working on the same topic to handle imbalanced text dataset! Please do like and subscribe to the channel if you find the videos helpful!

[@vishruthdilip]

@@theartificialguy6898 sure! I have a question. Should the data for each label be balanced? Or should it be balanced for each unique combination of labels?

[@theartificialguy6898]

@@vishruthdilip video is out: kzbin.info/www/bejne/q5Pbl4qGnJeXgJI

[@masterevil3347]

Hello, AI guy, I want to ask multi-label means [0, 1, 1, 0 ,0]? just one example can have more than one label at the same time?

[@theartificialguy6898]

Yes!

[@theartificialguy6898]

Watch this video to understand MULTI-CLASS classification: kzbin.info/www/bejne/raGcc6iajqyNr6M

[@theartificialguy6898]

@@egorsoldatkin1436 yes i messed up that part, but it's the easy part, what you can do is just make a custom function and check for each output probability that if it is > 0.50 (or your own threshold), show only those (assign label 1), else don't show (assign label 0).

[@GeoffroyDaumer-n2g]

heyyyy slow down

[@xv0047]

This is not very helpful. I feel like I am learning nothing by watching this. Just copy and pasting code.

[@simawpalmer7721]

Whatever you have done with the audio, it is stupid AF, why ? WTF ?? The voice should be centered in the middle (mono) not dance all over the place, even in Music you'll hear the vocal dead in the center to focus the listener attention not to distract it

[@GeoffroyDaumer-n2g]

slow up dude

[@vanyabhatia1554]

Hello Guys I used this code for the first 5 "ABSTRACTS" to label in the top three topics. Check if this can help instead of argmax() for i in range(0,5): example = test_df['ABSTRACT'][i] encodings = tokenizer.encode_plus( example, None, add_special_tokens=True, max_length=MAX_LEN, padding='max_length', return_token_type_ids=True, truncation=True, return_attention_mask=True, return_tensors='pt') model.eval() with torch.no_grad(): input_ids = encodings['input_ids'].to(device, dtype=torch.long) attention_mask = encodings['attention_mask'].to(device, dtype=torch.long) token_type_ids = encodings['token_type_ids'].to(device, dtype=torch.long) output = model(input_ids, attention_mask, token_type_ids) final_output = torch.sigmoid(output).cpu().detach().numpy().tolist() items=[] #print(train_df.columns[1:].to_list()[k for k in (np.argsort(final_output)[::-1][:3][0])]) for k in np.argsort(final_output)[::-1][0][:3]: items.append(train_df.columns[1:].to_list()[k]) print ("Top three research topics for {} Abstract type is: {}".format ((i+1),items))