Hugging Face Transformers: the basics. Practical coding guides SE1E1. NLP Models (BERT/RoBERTa)

Practical Python Coding Guide - BERT in PyTorch
In this first episode of the practical coding guide series, I discuss the basics of the Hugging Face Transformers Library. What is it? how does it work? what can you do with it? This episode focuses on high-level concepts, navigating their website and implementing some out-of-the-box functionality.
Intro: 00:00
What is Hugging Face's Transformer Library: 1:12
Hugging Face models: 2:00
Navigating the Transformers documentation: 8:56
Coding with Transformers - installation: 11:55
Using pre-defined pipelines: 12:45
Implementing a model through PyTorch: 14:08
Tokenisers, Token IDs and Attention Masks: 16:28
Output from the model: 25:26
Outro: 27:26
This series attempts to offer a casual guide to Hugging Face and Transformer models focused on implementation rather than theory. Let me know if you enjoy them!
In future episodes, I will be retraining a model from the Transformers Library (RoBERTa) on a downstream task: a multi-label classification problem. In an attempt to spot subtle sentiment attributes in online comments. Make sure to subscribe if you are interested.
Check out my website: www.rupert.digital
----- Good learning material for theory (Transformers / BERT)
Attention is all you need paper: arxiv.org/abs/1706.03762
BERT paper: arxiv.org/abs/1810.04805
RoBERTa paper: arxiv.org/abs/1907.11692
Jay Alanmar illustrated articles: jalammar.github.io/illustrate... (check out his BERT one too)
Chris McCormick: mccormickml.com/ (check out his youtube series on BERT / Transformers)

[@paruldhariwal]

It was really the most simplified and to the point video I watched on this topic. Great work!!

[@hugovassard7496]

This is what I search for days, thank you so much for these very clear explanations on Hugging Face basics !

[@user-qj4zr1pj9y]

This guy really knows what he is talking about. Excellent walkthrough , Thank You !!

[@aurkom]

Eagerly waiting for more videos from this channel!

[@stevefurness9833]

Fantastic Tutorial mate you are a natural teacher. I was clicking around Huggingface like a noob for hours until I found your video. Thanks so much for this.

[@rupert_ai]

Thanks Steve! Appreciated

[@rohitkochikkatfrancis]

You're the reason i can now understand how to code such things easily. PLEASE KEEP DOING WHAT YOU'RE DOING. COZ NO OME EXPLAINS THESE THINGS LIKE U DO

[@rezamazaheri6469]

Thank you really really much for explaining fundamental terms.

[@HeadshotComing]

This guide is pure gold man! Thank you so much! Wish you'd upload more videos, this is truly amazing. 😁

[@rupert_ai]

Thanks so much! : )

[@littlebylittle2237]

Thanks for your awesome walkthrough! Looking forward to your next video as well 😃

[@gk4539]

this video is worth watching the time!!

[@minty1110]

Well done to this young man. Well structured and nicely explained to cater to non experts

[@rupert_ai]

Thanks Minty!

[@alishehper1]

Great work! Would love to see more videos from you.

[@pablog3703]

Awesome video mate! Really appreciate how clear you explain everything

[@rupert_ai]

Glad it was helpful! :)

[@sunilsurendrasingh7736]

Nicely explained 👌. Looking forward to more such videos, thanks for sharing.

[@rupert_ai]

Thanks a lot!

[@stevenclinton3585]

Outstanding explanations, thanks a ton!

[@juliank7408]

Thanks! Really well explained

[@jamesnorton4953]

Very helpful 👍

[@anubhavanand6573]

Nice intro to hf library. Really helpful 👌

[@rupert_ai]

Thanks :)

[@adriendemaegdt5902]

Pretty helpful ! Keep going bro ! Thanks from France

[@rupert_ai]

Cheers dude! You're welcome from the UK !

[@mdnayemuddin5595]

Useful video. Thank you.

[@vijayendrasdm]

Thanks for detailed explanation.

[@rupert_ai]

You're welcome, thanks for watching :)

[@qandos-nour]

Great explanation Thanks

[@luis96xd]

Excellent video, It was well explained and helped me lot into hugging face models, thanks

[@rupert_ai]

Thank you Luis! Glad you found it helpful

[@odechristopher1241]

Amazing tutorial

[@smartasiankid]

Hey man, just found your channel and your videos are very good!

[@mma-io4uf]

great video!

[@rishabhindoria1696]

really helpful, thank you!

[@rupert_ai]

Thanks! :)

[@RuloGames1]

Thanks!!!

[@kostik]

This is great. Hope you'll continue with LLMs.

[@TheMarComplex]

Thank you so much!

[@animeshkumar1684]

Very useful, thanks 👍

[@rupert_ai]

Thanks!

[@user-qj4zr1pj9y]

Anime -sh Kumar

[@paulkoehler5017]

very good explanation thank you very much

[@rupert_ai]

Thanks for watching Paul!

[@ArunKumar-bp5lo]

helpful thanks

[@joacosolbes9283]

Great content mister

[@rupert_ai]

Thanks ! :)

[@Madaaguu]

Great video! Is it possible to apply either the manual or the pre-made pipeline approach to a dataframe/dataset, instead of just those small lists? I wanted to solve a binary classification task but have 2 different text columns. Thanks for the video, good stuff man!

[@april147]

I've been looking for something like this for weeks! Easy and Simple explanation! Thank you so much for this informative video! 🤍

[@rupert_ai]

Thank you very much April! Glad you benefited from it :)

[@megacapojuan]

🤩🤩

[@JuantheJohn1]

hello @rupert ai, thank you for the amazing tutorial. It says in the tittle "SE1E1". Does that mean there are more episodes coming out? thanks!

[@rupert_ai]

Check out my channel for more hugging face and computer vision videos and tutorials! :)

[@yousefsalah9539]

Very useful, is there any chance you can fine-tune roberta for sentence simplification? You will do me soo good

[@coneuer15]

This is fantastic! You should be a professor in University!

[@rupert_ai]

You're too kind! Thank you!

[@gowthamkrishna6283]

Hey, Nice content!! Can you please make a video explaining your microsoft challenge task using Roberta model?

[@rupert_ai]

Hey, glad you enjoyed it. Sure, are you after a video on using RoBERTa on 'The Microsoft Research Sentence Completion Challenge' ? Happy to try get that done soon as a next video :)

[@gowthamkrishna6283]

@@rupert_ai yeah , that would be helpful. Thnx brother 😊

[@karansaxena96]

Hi Rupert, I liked your video. I have one question, sorry if this is something you already answered, I want to create my own bert model from scratch using a specific dataset that's based on my use case. Any tutorials could you please suggest that I can follow for that? Thank you so much in advance.

[@dsaiohd2d2qad]

you just need to stack some transformer encoders , and train them using your dataset with two specified tasks in the original bert paper, namely NSP and MLM.

[@rupert_ai]

Hi Karan, sure - it sort of depends what dataset you are using and what task you are trying to achieve? If you look at my other video on multi-label classification then you can get a good template for doing any task with hugging face with any dataset! Let me know if I can help any further :)

[@Probly]

you dont need to build bert from scratch if you're only looking to re-train with your own dataset. You can add a machine learning layer on top of BERT where bert is used for word embedding and your layer(s) is used for optimizing BERT based on a data set you provide.

[@athulvinod6169]

bro ,am trying to create a speech to text system where only like 30 words need to be recognized...from what i know CNN can do the trick..but do i need to go for end to end models for speech recognition.hope you could reply me asap

[@rupert_ai]

hey bro, if I were you I'd look at: huggingface.co/docs/transformers/model_doc/speech_to_text

[@abhishekprakash9803]

create a model for automated long question and answers generation models

[@not_amanullah]

Zoom in can't see clearly 😢