HuggingFace Crash Course - Sentiment Analysis, Model Hub, Fine Tuning

In this video I show you everything to get started with Huggingface and the Transformers library. We build a sentiment analysis pipeline, I show you the Model Hub, and how you can fine tune your own models.
Get my Free NumPy Handbook: www.python-engineer.com/numpy...
✅ Write cleaner code with Sourcery, instant refactoring suggestions in VS Code & PyCharm: sourcery.ai/?... *
⭐ Join Our Discord : / discord
📓 ML Notebooks available on Patreon:
/ patrickloeber
If you enjoyed this video, please subscribe to the channel:
▶️ : / @patloeber
The Huggingface transformers library is probably the most popular NLP library in Python right now, and can be combined directly with PyTorch or TensorFlow. It provides state-of-the-art Natural Language Processing models and has a very clean API that makes it extremely simple to implement powerful NLP pipelines.
Resources:
huggingface.co/
huggingface.co/models
huggingface.co/transformers/c...
PyTorch Course: • PyTorch Tutorials - Co...
TensorFlow Course: • TensorFlow 2 Beginner ...
~~~~~~~~~~~~~~~ CONNECT ~~~~~~~~~~~~~~~
🖥️ Website: www.python-engineer.com
🐦 Twitter - / patloeber
✉️ Newsletter - www.python-engineer.com/newsl...
📸 Instagram - / patloeber
🦾 Discord: / discord
▶️ Subscribe: / @patloeber
~~~~~~~~~~~~~~ SUPPORT ME ~~~~~~~~~~~~~~
🅿 Patreon - / patrickloeber
#Python
Timeline:
00:00 - Introduction
00:43 - Pipeline
09:19 - Model And Tokenizer
15:20 - PyTorch classification
21:55 - Saving And Loading
23:33 - Model Hub
31:30 - Fine Tuning
----------------------------------------------------------------------------------------------------------
* This is an affiliate link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏

[@patloeber]

Do you like HuggingFace?

[@nicklansbury3166]

Yes. Do I win? 😎

[@trevormuchenje1553]

very much!

[@jieweiyang3567]

yes

[@scar2080]

Ja because of you 😁🤣. Du bist (Best )🥂

[@saminchowdhury7995]

I can't believe it's free

[@JoshPeak]

The last 6-8 minutes of this video is exactly what I have been trying to hunt down as a tutorial. Thank you!

[@oliverguhr8746]

Thanks for using my model :)

[@just_ign]

There are so many videos out there that show how to use huggingface's models with a pipeline and making it seem so "easy" to do things, which it is. But unlike those videos, this one really shows how we can use models natively and train them with our own cycles. Instead of portraying things as "easy", you decided to show how to actually get things done and I absolutely loved that!! Thanks for the tutorial :D

[@netrahirani3147]

I feel like I've hit a jackpot! It took me forever to find such an easy-to-learn video. Das war sehr gut! Danke!

[@prettiestthing]

Loving this ❤! Please do a series on this 🥳

[@HuevoFriteR]

Thanks for the tutorial buddy, it was amazing!

[@parttimelarry]

Excited about this one, thanks!

[@cexploreful]

i've seen lots of tutorials... this is the best of all!

[@caiyu538]

Clear explanation for beginner. Great

[@vijaypalmanit]

Very nice explanation, many things got cleared I had confusion about eg tokenizers. Really liked the video and your way of teaching. Expecting more like fine tuning bert on custom dataset, please make video on it.

[@aidarfaizrakhmanov1901]

Maaan! I liked how you started the tutorial: well-explained and sweet for the beginners. Starting from Pytorch classification, you probably assumed "enough with beginners, let's level up 100x times lol". Many lines of code with arguments you wrote, require some googling, hence a quicky high-level explanation of those could do magic. Nevertheless, thanks for making this video mate.

[@juliank7408]

Thanks! Well explained!

[@philipp5636]

Holy shit this just saved my and my thesis from a week of pain. Thank you very much!

[@Lakshraut]

Your presentation is excellent.

[@SanataniAryavrat]

Thank you Patrick... this was much awaited course... can you please create a full length tutorial including deploying an "dashboard app" on docker

[@mathsharking]

Good tutorial

[@annarocha9769]

Thank you soooooooo much for this, suscribed :)

[@kinwong6383]

This is really powerful and efficient for real world usage. I wonder if Kaggle have a rule to ban people doing this on competitions. We almost hear Patrick speaks German. That was so close! Danke for the video!

[@robosergTV]

Please make a whole series on this :) There is also a very nice framework on top of this called "simple transformers"

[@patloeber]

thanks for the suggestion

[@canernm]

Hello, thank you for the extremely valuable video. I do have one question however. During the fine-tuning process, in the first case where we use Trainer(): as far as I can tell, the model and the data are not in the GPU by default, and we also do not move them there (as we do in the custom PyTorch training loop). I tried it in a notebook and when I run the command "next(model.parameters()).is_cuda", where model is the from_pretrained() model, it returns False. Still, moving the model to the GPU would be the same even in this case (with the trainer), by doing from_pretrained('...').to('cuda'). However, when we only have a dataset and we dont create a dataloader, I am not sure how to move it to the GPU. Do you now perhaps? I would appreciate it a lot!

[@haralc]

OMG! Thanks for this video! Don't have to deal with French accent anymore!

[@sanjaybhatikar]

How would you do neural transfer learning (retraining) by unfreezing only the fully connected layers? I was given to understand that this is the proper way to fine-tune a deep learning model, not retraining all model parameters.

[@mairadebayser5383]

Nice video. It seems that my work in 2015 while at IBM Research which was exactly the same thing presented in this video has been widely accepted in the Machine Learning community. Cool.🤗

[@patloeber]

Thank you! Yeah the ML community has grown a lot :)

[@mays7n]

very helpful, thanks aaaaaa loooot

[@WalkAloneLive]

I was ready to subscribe for you for second time :D

[@patloeber]

yeah :)

[@haralc]

Hi, would you please make video for the text-generation and question-answering, from dissecting how the pipeline does it and then fine-tuning?

[@imdadood5705]

I am simple man! I see Patrick I like the video!

[@patloeber]

❤️

[@UsmanMalik57]

Hello, For fine-tuning multiclass text classification model, the approach remains same? Te

[@aseemsrivastava3995]

A series on this would be reallly great! Like the one you have with PyTorch. In that series you can go for some very complex architectures in NLP publications. viz. using standard lstm/gru with bert tokens and at linear layers + softmax is easy. I am asking if you could show implementing other attention tweaking strategies or other similar complex architectures that people do these days in publications would really help us!

[@soulwreckedyouth877]

How do I finetune the German Sentiment by Oliver Guhr? Can I just follow your steps or do I have to take care with a special Tokenizer or anything? Cheers and thanks for your work

[@LuisMorales-bc7ro]

I love you patrick

[@Mike-jr7re]

Patrick, do you know how to remove the models from the HD? I'm seeing that for each model it downloads it directly on the mac. Due to space problem, how can I remove them if I don't use them anymore. Thanks a lot!

[@abhishekriyer]

@Patrick: Could you pls share the code link for the above ??. Or it's already there I am unable to find it

[@xieen7976]

hi, where is the. "train_test_split" function come from?

[@xingyubian5654]

goated video

[@haralc]

Would you please make another video with the latest version of the libraries?

[@sumapriiya]

I tried to fine tune the RobertaModel to a custom dataset using the Trainer object, and then saved the model and tokenizer to my Google drive. But on retrieving the model and predicting it on a validation dataset gives me the same class prediction(all with negative values), do you have any idea why? Thanks for your help.

[@shanukadulshan7154]

hey bro, how are you executing importing lines only once?(I noticed they turned in to grey)

[@KamalSingh-zo1ol]

Great video, can you make video on how to change cache from default directory to other drive?

[@darraghcaffrey4082]

Can someone explain what's going on with these two lines of code as its only explained with Tensorflow on hugging face. I understand it's a dictionary but its a little confusing item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()} item['labels'] = torch.tensor(self.labels[idx])

[@ironF5]

the fine tuning is done with supervised dataset, how to do in case of self-supervised case? inwhich the data is not labeled but the model retrains on your data and make judgments?

[@philcui9268]

Hi Patrick, this is a nice tutorial. Can we have the access of the code?

[@nirash8018]

36:02 How would you go on and make specific predictions?

[@komalkukreja4441]

While loading xlm robera from my machine which I saved as .bin or .pth, I am getting incompatible key error while loading saved model from my local machine for evaluation

[@straightup7up]

I'm confused, if I'm using a model from hugging face on my desktop, does the model communicate with remote cloud services when running the model?

[@pandapanda5889]

Hi, what should I do when I have a lot of comments and posts without labels? I'm a beginner and what I see on the Internet so far is always text data with labels such as movie reviews etc.

[@md.rafiuzzamanbhuiyanafrid2087]

Good one. Please, Can you share the github link for fine-tuning script ?

[@user-xc4mt2fo7i]

so for the german part, you can get the same result without the attention mask?

[@v1hana350]

What is the meaning of fine-tuning and Pre-trained in Transformers?

[@andrea-mj9ce]

Is the code that you typed available ?

[@jaypie9092]

I'm using visual Studio and have done all the installs and it is not working. I have the venv started and installed PyTorch and the transformers. I have it in the project directory. Am I missing something?

[@jesusmtz29]

I like your tutorials. However, just one small critique. Sometimes I feel you're just reading code for me. I can do that but i think the value of yt tutorials is to explain why we do certain things. Otherwise im just punching lines in. Sorry if this sounds harsh I don't mean it that way

[@testingemailstestingemails4245]

how to do that trained huggingface model on my own dataset? how i can start ? i don't know the structure of the dataset? help.. very help how I store voice and how to lik with its text how to orgnize that I an looking for any one help me in this planet Should I look for the answer in Mars?!

[@mandilkarki5134]

Yayyyy

[@benxneo]

does it support R also?

[@user-sc2er8th1w]

where to get the source code?

[@yuandi9410]

Hey I can't find the model of license to activate it it's doesn't show up????

[@AshishSingh-753]

Patrick What to choose doing projects for computer nlp or choose only one

[@patloeber]

I recommend focusing on one in the beginning until you are comfortable. but do what interests you the most :)

[@AshishSingh-753]

Thanks IAM a biology and math student if you have any idea about how to use AI on this topics inform me Patrick

[@trevormuchenje1553]

is there a specific reason for using pytorch instead of tensorflow for this task

[@patloeber]

nope, both are fine. I just had to choose one here ;)

[@trevormuchenje1553]

@@patloeber okay great. Thanks for the wonderful tutorial

[@pathikghugare9918]

which PyCharm theme are you using?

[@patloeber]

Dracula

[@haralc]

Trying to make this work with text-generation pipeline but to no avail... feel so dumb .... from transformers import pipeline generator = pipeline("text-generation", model=model, tokenizer=tokenizer) generator(["the quick brown fox"])

[@ubaidulkhan]

Are you using a local GPU?

[@artemkeller2571]

You show how to use your own tokenizer, but you are not explaining what it is and why would i possibly want to use a different one :( Aswell as what batch is,logits, what are all those strange numbers and how can it be any usefull. Also what is actually a pytorch. And many other stuff. Its like following your steps without understanding of what im actually doing right now. Still the best explain i found so far tho... Thanks!

[@andrea-mj9ce]

Some links are broken

[@airepublic9864]

Have the same voice of kevin from ,data school

[@736939]

I didn't get your fine-tuning, because (As I understood) Fine tuning means that you should freeze some part of your neural network (by setting requires_grad=False) and train only some part (usually output layers) of your model, and after it unfreeze layers.

[@enriquecarbo9096]

500 likes :)

[@soumilyade1057]

most blogs and videos contain the same information. An already prepared dataset comes with certain benefits. Merely going through a snippet of code doesn't help much. 😑

[@salemsheikh254]

DUll

[@smnt]

Is anyone else triggered by the fact that he kept calling the "huggingface" a "smiley face"?