HuggingFace Fundamentals with LLM's such as TInyLlama and Mistral 7B

chris looks under the hood of huggingface models such as TinyLlama and Mistral 7-B. In the video Chris presents a high level reference model of large language models and uses this to show how tokenization and the AutoTokenizer module works from the HuggingFace transfomer library linking it back to the HuggingFace repository. In addition we look at the tokenizer config and Chris shows how Mistral and Llama-2 both use the same tokenizer and embeddings architecture (albeit different vocabularies). Finally Chris shows you how to look at the model configuration and model architecture of hugging face models.
As we start to build towards our own large language model, understanding these fundamentals are critical no matter whether you are a builder or consumer of AI.
Google Colab:
colab.research.google.com/dri...

[@ukaszrozewicz7488]

The best video I've watched on KZbin about LLM so far. You explain complex topics in an accessible language, clearly and understandably. You are doing a very good job. I'm eagerly waiting for the next videos :)

[@ThomazMartinez]

same here

[@chrishayuk]

Wow, thanks!, this one actually took a long time to get right, glad you liked it

[@mindurownbussines]

Thank you so much Chris I truly believe if one has a great understanding of a subject he can teach it clearly and you simply did that ! God bless you

[@chrishayuk]

You are too kind, and thank you. Glad it was useful

[@kenchang3456]

Excellent explanation. Although I don't have a use case to fine-tune a model currently, I presume I will eventually it'll be great to have what you've shared in my back pocket. Thanks a bunch.

[@chrishayuk]

Awesome, glad it was useful

[@janstrunk]

Great video! Looking forward to your next videos…

[@chrishayuk]

Yeah, next ones in series will be fun, glad you’re enjoying it

[@Jaypatel512]

Amazing way to get people comfortable with the model architecture. Thank you so much for sharing your knowledge.

[@chrishayuk]

Glad it was useful

[@BipinRimal314]

Looking really forward to the next video.

[@chrishayuk]

next one dropped

[@keithrule4240]

Great video. Just the right amount of detail. Thanks.

[@chrishayuk]

Glad it was helpful!

[@narenkrishnanGenius]

very well explained and useful

[@chrishayuk]

So glad to hear that, thank you

[@kennethmichaelreda]

Insanely valuable video. Thank you!

[@chrishayuk]

Glad it’s useful

[@atifsaeedkhan9207]

Thanks being so so in details. That was really a refresher for me. Glad someone like you is doing such a good work.

[@chrishayuk]

thank you, very much appreciate that

[@prashantkowshik5637]

Thanks a lot Chris.

[@chrishayuk]

glad it was useful

[@wadejohnson4542]

For the very first time, I finally get it, thanks to you. Thank you for your service to the community.

[@john6268]

How does the tokenizer decode sub-word embeddings? Specifically, how do you determine which sequence is concatenated into a word vs. standing on its own? As shown, the answer would be decoded with spaces between the embeddings, which wouldn't make "Lovelace" into a word.

[@chrishayuk]

Certain tokens will have spaces others won’t so _lace would be a different token from lace. I have a deep dive of the tiktoken tokenizer where I spend a lot of time on this. I am planning to do a building a tokenizer vid soon as part of this series

[@chrishayuk]

how the tokenizer for gpt-4 (tiktoken) works and why it can't reverse strings kzbin.info/www/bejne/hH7SeXuJjMtkg9E

[@john6268]

@@chrishayuk Thanks, I'll check out the other video and looking forward to the next one.

[@ilyanemihin6029]

Thank you! This video brings light into the black box of LLM magic)

[@chrishayuk]

more to come, the next set of videos reveal a bunch more

[@nguyenhuuanhtuan5360]

Aways awesome content ❤

[@chrishayuk]

Super glad it’s useful, thank you

[@BiranchiNarayanNayak]

Excellent tutorial to get started with LLMs.

[@chrishayuk]

Glad you liked it!

[@javaneze]

great video - many thanks!

[@chrishayuk]

Glad you liked it!

[@AncientSlugThrower]

Great video.

[@chrishayuk]

thank you

[@MannyBernabe]

great video. Thx!

[@chrishayuk]

thank you

[@tec-earning8672]

Great job sir, one video for me sir how to build llama APIs i want use my train own model now i want using in my website ..

[@chrishayuk]

That’s where we are are working up to, but you can check out my existing fine tuning llama-2 video

[@huiwencheng4585]

Bro, just turn-on the Big Thank so I can donate you

[@chrishayuk]

lol, not gonna happen but appreciate the gesture and glad you like the videos