Fine-tuning Large Language Models (LLMs) | w/ Example Code

[@ShawhinTalebi]

UPDATE: Someone pointed out that the fine-tuned model here is overfitting, so I created an improved example that uses transfer learning: kzbin.info/www/bejne/aoKrmGt_rZ2KjbM 👉More on LLMs: kzbin.info/aero/PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0 -- References [1] Deeplearning.ai Finetuning Large Langauge Models Short Course: www.deeplearning.ai/short-courses/finetuning-large-language-models/ [2] arXiv:2005.14165 [cs.CL] (GPT-3 Paper) [3] arXiv:2303.18223 [cs.CL] (Survey of LLMs) [4] arXiv:2203.02155 [cs.CL] (InstructGPT paper) [5] PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware: huggingface.co/blog/peft [6] arXiv:2106.09685 [cs.CL] (LoRA paper) [7] Original dataset source - Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics.

[@salamsoftyt]

❤

[@salamsoftyt]

I

[@beaux2572]

Honestly the most straightforward explanation I've ever watched. Super excellent work Shaw. Thank you. It's so rare to find good communicators like you!

[@ShawhinTalebi]

Thanks, glad it was clear 😁

[@yannicbrose]

Fully agree!

[@otorishingen8600]

Finally a linear tutorial actually talking about what, how, why without trying to funny with meta humor and fluff Thank you - great explanation

[@JaishreeramCoder]

You have explained this so clearly, that even a novice in NLP can understand it.

[@lukaboljevicboljevic]

Such a great video. This is the first one I watched from you. You explain everything so nicely, and in my opinion you provided just the right amount of information - not too little, so it doesn't feel superficial and you feel like you've learned something, but not too much, so that you can take what you've learned and read more about it yourself if you're interested. Looking forward to seeing more of your content!

[@ShawhinTalebi]

Great to hear! Glad it was clear :)

[@xugefu]

Thanks!

[@ShawhinTalebi]

Thanks for the support! Glad it was helpful :)

[@srinivasguptha9538]

One thing that really standout for me is not using Google Colab for explanation. Explaining all code without scrolling helps the audience better grasp the content as it goes with the flow without waiting for the code to execute and helps the audience to remember where the variables were defined and all. Great approach and thanks for the amazing content!

[@ShawhinTalebi]

Thanks, that's good feedback! I'll keep this in mind for future videos.

[@checkdgt]

Just came to this video from HF and I have to say, I love they way you describe this! Thanks for the great video!

[@ShawhinTalebi]

Great to hear! Thanks for watching :)

[@balubalaji9956]

Hey KZbin algorithm , I loved this video . suggest me more of them

[@ShawhinTalebi]

Here's the series playlist: kzbin.info/aero/PLz-ep5RbHosU2hnz5ejezwaYpdMutMVB0

[@junjieya]

A very clear and straightforward video explaining finetuning.

[@ShawhinTalebi]

Glad it was clear :)

[@fakharmursaleen9889]

Hands down one of the most best explanations on youtube keep it up homie

[@jonathanleroy269]

This video series is underrated. Loved it, thank you!

[@Josia-p5m]

You're a fantastic communicator - this was very helpful. I would love to see more walkthroughs like this that have a good balance of theory, math and python.

[@ShawhinTalebi]

More to come!

[@BRK326]

Nice job, efficient , straight forward, enough detail for learning big picture without confusion.

[@egemenklc2515]

Hey KZbin, I really liked this kind of machine learning and fine tuning topics. Please recommend me more of these.

[@ShawhinTalebi]

Glad you liked it :)

[@ayyanarjayabalan]

Excellent way of teaching. Keep doing this kind of good work.

[@scifithoughts3611]

Great video Shaw! It was a good balance between details and concepts. Very unusual to see this so well done. Thank you.

[@ShawhinTalebi]

Glad you enjoyed it!

[@saraesshaimi]

excellent simple explanation to the point. Love it !

[@yoffel2196]

Wow dude, just you wait, this channel is gonna go viral! You explain everything so clearly, wish you led the courses at my university.

[@ShawhinTalebi]

Thanks for the kind words! Maybe one day 😉

[@AbdulademAljamel-n5d]

I was struggling to understand some details, before this video, thanks a lot

[@ShawhinTalebi]

Great to hear. I’m glad it helped!

[@abdullaa9478]

Thanks!

[@ShawhinTalebi]

Thank you 🙏🙏🙏

[@dendi1076]

this channel is going to hit 6 figure subscribers at this rate

[@ShawhinTalebi]

I hope so 😅

[@sreeramch]

Thank you for the detailed explaination line by line. Finally a place, I can rely on with working example

[@ShawhinTalebi]

Glad it was helpful!

[@alikarooni9713]

Even though this was high level instruction, it was perfect. I can continue from here. Thanks Shahin jan!

[@ShawhinTalebi]

Glad it helped!

[@rubencabrera8519]

This was one of the best videos on this topic, really nice man, keep going.

[@ShawhinTalebi]

Thanks! Glad it was clear :)

[@thehousehusbandcn5074]

You are the man! No BS, just good useful info

[@ShawhinTalebi]

Thanks, glad it was helpful 😁

[@arunshrestha791]

Clear Explanation, Amazing

[@salmaelbarbori579]

Clear and straightforward to the point, thanks a lot for making this valuable content accessible on ytb💡

[@ShawhinTalebi]

Happy to help!

[@Random-bq8qc]

Thanks for the well defined video because it helped me prepare my proposal related to this topic.

[@azizhassouna9919]

Im really gratful for youre work , you really help me when I had no one to ask .

[@jasoncole3253]

Well done, even if I already knew all this shit it was really nice to listen to your clear explanation

[@ShawhinTalebi]

lol! Glad you enjoyed it :)

[@lotusluthor]

Finally something that I can grok!!!! Fantastic tutorial!

[@Akshatgiri]

This is gonna come handy. Thanks for breaking it down

[@ShawhinTalebi]

Happy to help!

[@EigenA]

Great video, I wanted to hear further discussion on mitigation techniques for overfitting. Thanks for making the video!

[@saadati]

Amazing video Shawhin. It was quite easy to follow and stuff were clearly explained. Thank you so much,

[@ShawhinTalebi]

Thanks! I'm glad it was clear and helpful

[@janjiavicii1328]

very explicit explanation. Thanks!

[@soudaminipanda]

Thanks for these great videos. I really love the fact that you focus on building an intuitive understanding as opposed to throwing jargons. Could you please start a langchain series?

[@ShawhinTalebi]

Great suggestion! I'll add that to my list. In the meantime, I have a code tutorial using LangChain here: kzbin.info/www/bejne/ZpTJaKmwgsSXkJIsi=Iyc1GzQ7g-A_n5Rr&t=1359

[@SaharAbo-y5d]

Your style of conveying information is wonderful. Good luck to you

[@alokkale2718]

Bro you are the best instructor

[@ShawhinTalebi]

Thank you, glad it was clear 🙏

[@bruceyin603]

Very good explain about fine tune, thanks Shaw

[@richardpinter9218]

Fantastic video. Thanks for the upload. Keep up the good work, you're awesome 😎

[@ShawhinTalebi]

Thanks, I’m glad you liked it 😁

[@sakarsalunke3245]

Indepth overview of fine tuning!

[@kevon217]

Excellent walkthrough

[@ShawhinTalebi]

🙏

[@melliott117]

Really great content. I love your balance of details and overview. It’s made it easy for me as a newcomer who is interested in details. My only criticism/advice is that you edit to remove silence. This is great for minimizing pauses mid sentence. But it would be helpful to have slightly more time at the end of each thought/point. Pausing for that extra 0.25 seconds at the end of a coherent teaching point helps greatly.

[@ShawhinTalebi]

Thanks, that's good feedback! I do get a bit heavy-handed with the edits 😅

[@alex70301]

Best video on llm fine tuning. Very concise and informative.

[@ShawhinTalebi]

Thanks! Glad you liked it :)

[@rma1563]

I needed to know how parameter efficient finetuning works to finetune a voice encoder for emotion detection task. This video helped me a lot. I used LoRA for it. Thanks ❤

[@ShawhinTalebi]

Glad it was helpful!

[@mookiejapan7351]

Wow! Amazing make-up! If it wasn't for the voice, I wouldn't believe this is actually David Cross!

[@ShawhinTalebi]

Haha, I was wearing jean shorts while filming this 😂

[@RealNikolaus]

Very good video and explanation!

[@ShawhinTalebi]

Glad it helped!

[@sanderkempen6744]

Thanks

[@ShawhinTalebi]

Wow, thanks so much! I’m glad it was valuable

[@sanderkempen6744]

@@ShawhinTalebi just out of sheer curiosity, how much do you get to keep from those euros?

[@ShawhinTalebi]

Not sure, this is the first time this has happened 😂. Google website says 70%

[@sanderkempen6744]

@@ShawhinTalebi i guess that is more than I expected. Anyway thanks for the videos and enjoy your ?coffee? (No idea what the prive levels are in your natural habitat)

[@ajeethsuryash5123]

Very informative video. Thanks for sharing

[@upadisetty]

Best video i saw. thanks a ton for sharing. glad i found right place

[@PrinceSinghal-j8m]

i was amazing ....thanks for uploading Shaw

[@ShawhinTalebi]

Thanks, happy to help!

[@Wei-KuoLi]

Very good & simple showcase, thanks

[@payam-bagheri]

Great video, Shawhin!

[@ShawhinTalebi]

Thanks, glad you enjoyed it!

[@hoseinmirhoseini6013]

Thank you for the great content.

[@simplyshorts748]

Great video! I love good explainations

[@Mastin70]

Fantastic explanation.

[@tintumarygeorge9309]

Thank you, Keep up the good work

[@ShawhinTalebi]

Thanks, happy to help!

[@younespiro]

amazing video, very well explained

[@ShawhinTalebi]

Glad it helped! Thanks :)

[@yb3134]

Very well explained

[@tgyawali]

I found you in youtube just today. Your presentation style, quality of content is very good. Keep up the great work. I am very passionate about AI technology in general, have been trying to conduct basic trainings to undergraduate college students and would love to connect to collaborate if you are interested. Thank you for doing this!

[@ShawhinTalebi]

Thanks for watching! Glad it was clear :) Feel free to set up a call if you like: calendly.com/shawhintalebi

[@tgyawali]

@@ShawhinTalebi Thank you. I will set up some time to connect.

[@Kevin.Kawchak]

Thank you for the discussion

[@swaykao1703]

thank you! I like your vedios, they help me a lot!

[@ITforGood]

Thanks Shaw, very helpful.

[@ShawhinTalebi]

Glad it was helpful!

[@heatherbrm]

here, you earned this: 👑

[@ShawhinTalebi]

Thanks 🤴

[@jijie133]

Great video. Thank you.

[@Bboreal88]

My next question after this video would be on how to pack this fine-tuned model into a UI and deploy.

[@ShawhinTalebi]

Great question. I discussed how to create a chat interface with Hugging Face + Gradio in a previous video: kzbin.info/www/bejne/oJLRYWqdodSUiMk

[@NateKrueger805]

Nicely done!

[@ShawhinTalebi]

Thanks!

[@bitschips]

So educative, thanks a lot!

[@ramp2011]

Excellent..... Thank you for sharing

[@ShawhinTalebi]

My pleasure, glad you liked it!

[@zeusgamer5860]

HI Shaw, amazing video - very nicely explained! Would be great if you could also do a video (with code examples) for Retrieval Augmented Generation as an alternative to fine-tuning :)

[@ShawhinTalebi]

Great suggestion. I have a few follow-up use cases planned out and RAG will definitely be part of it.

[@BamiCake]

​@@ShawhinTalebimaybe also how to fine tune openai model too?

[@ShawhinTalebi]

Just dropped! kzbin.info/www/bejne/aoOkp32qaMuKpas

[@chilupuriharshitha7261]

Hey, It was good but did your Model also take a lot of time to get fine-tuned while you were Pluging everything to the Trainer Class as for me it's taking nearly i lost the time it is taking but 10 epochs are getting trained for every 0.05 it/s.

[@ShawhinTalebi]

Are you running this locally?

[@aldotanca9430]

Very clear, thanks!

[@ShawhinTalebi]

Thanks Aldo!

[@raffelravionaldo463]

Thank you, is a nice video and you have a clear explanation, actually I try to do this with GPT Neo model (EleutherAI/gpt-neo-1.3B) but when I do training, the Training Loss always have no log values and Validation Loss always NaN (when do it with BERT or distilbert, is run perfectly), Do you have any suggestions or reading resources to fix this?

[@ShawhinTalebi]

Hard to say without seeing the code. Perhaps you can share more info here: www.shawhintalebi.com/contact

[@machireddyshyamsunder987]

Thankyou very much it is really very useful .

[@ShawhinTalebi]

Happy to help!

[@arthurs6405]

This was beautifully described. I wish you had provided a Linux alternative for the "model.to('mps/cpu'). I have a linux workstation and a p100 gpu. Also, you did not include the means to save your newly trained model. I think most of us students would appreciate knowing how to save the model locally and to huggingface. Thanks for your efforts.

[@ShawhinTalebi]

I do fine-tuning on a Linux machine here: kzbin.info/www/bejne/jqHSfHVpqtiggcU

[@goinsgroove]

Loving the info! Do you have a video on self-supervised training? I want to train a llm to write in my style.

[@ShawhinTalebi]

Not yet! But I think I'll do this for my next video because of your comment :)

[@goinsgroove]

@@ShawhinTalebi AWSOME!

[@nandkumarghatge5152]

great video.

[@MannyBernabe]

Excellent walk-thru. Thank you, Shaw!I was getting errors on the new model. Switching the device worked for me. # Check if CUDA is available and set the device accordingly device = 'cuda' if torch.cuda.is_available() else 'cpu' model.to(device) # Move the model to the appropriate device (GPU or CPU)

[@ShawhinTalebi]

Thanks Manny! That's a good note, I wasn't able to test the code on a non-Mac machine.

[@lucycaffrey-maffei6666]

thank you sooo much for this content, it's so helpful and clear!!! if i don't use LoRa when fine-tuning and instead just specify `Trainer(model=AutoModelForSequenceClassification.from_pretrained( model_checkpoint, num_labels=2)` (which is what huggingface does in most of its documentation), what type of parameter training is that performing? retraining all of the parameters?

[@ShawhinTalebi]

Yes, that will do full retraining!

[@KaptainLuis]

So nice video thank you soooo much!!❤

[@ShawhinTalebi]

Happy to help 😁

[@vicenteenriquemachacaarced2103]

Thank you so much for that video. ¿From where did you get TRAINING loss metrics? In the console and in the trainer_state.json, I only see evaluation metrics.

[@ShawhinTalebi]

Great question. The training loss is predefined as a property of the base model, so no need to redefine that explicitly.

[@totalcooljeff]

Random question i how do you edit you audio clips together to make them so seamless because idk where to mate them. And great video by the way 👍

[@ShawhinTalebi]

I use iMovie :)

[@rbrowne4255]

Fantastic job on this overview, as for other videos, I don't see many videos on Inference scaling, i.e requirements for concurrency, latency etc...what are the hardware requirements i.e number of GPUs per systems or number of systems, etc

[@ShawhinTalebi]

I'm glad it was helpful :) That's is a great suggestion. I will add it to my list. Thank you!

[@pawan3133]

Thanks for the beautifully explanation!! When you said, for PEFT "we augment the model with additional parameters that are trainable", how do we add these parameters exactly? Do we add a new layer? Also, when we say "%trainable parameters out of total parameters", doesn't that mean that we are updating a certain % of original parameters?

[@Throwingness]

Very good. Very fast and also easy to follow. As far as future content, keep us posted about how to do LoRA on quantized models. How can the future be anything but LoRA on quantized models?!?!?!?

[@ShawhinTalebi]

Thanks, glad you liked it. Video coming this quarter on exactly that!

[@zsmj820]

Nice video !

[@liaoyixu6882]

Very amazing video!!! I have one question: when I use your code to fine-tune the model with my own dataset, but since my dataset is too large it leads to memory error (not gpu memory) when I read the dataset, what should I do to avoid this issue? Can I read and fine-tune in a small batch?

[@ShawhinTalebi]

You can try reducing the batch size. Also happy to help troubleshoot via office hours: calendly.com/shawhintalebi/office-hours

[@FalahgsGate]

thanks for sharing this video ....but Is the technique of fine-tuning model for the custom dataset better than the technique of using the RAG system for LLM Apps. or reverse

[@ShawhinTalebi]

While the best choice of technique will depend on the specific use case, here's the intuition I've gathered. RAG is great when you want the AI to have access to specific knowledge. Fine-tuning is great when you want the AI to provide responses in a particular format and style. Additionally, the techniques are not mutually exclusive, so they can be combined to potentially provide even better results.

[@naehalmulazim]

Greetings! Really nice tutorial! THANK YOU for including Lora! I need to train an Llm on a higher level language we wrote in C++, to produce our code. It's all private infrastructure. Time isnt an issue but I'd like to do it locally on a mac m2 if I can and was considering Lora on a tiny llm. Is this going to be possible?

[@ShawhinTalebi]

While I haven't done that myself, that is surely possible. The challenge I've run into is that many open-source models don't work so easily on Mac, but I plan to figure it out and many video about it.

[@ShawhinTalebi]

Update: I got a fine-tuning example working locally on my M1 Mac Mini :) Here's a similar example running on my Mac: kzbin.info/www/bejne/aYGsopuah9-brqc

[@evan7306]

Thank you for your great tutorial! What I don't understand is how to use the fine tuned model as an API so we can use it on website. Do you have any tutorial about that?

[@ShawhinTalebi]

Great question. I haven't covered that yet, but a common approach is to containerize the model using Docker and equipping it with an API using a library like FastAPI. Then you can host that on a local server or via a cloud provider.

[@devtest202]

Hi thanks!! A question for a model in which I have more than 2,000 pdfs. Do you recommend improving the handling of vector databases? When do you recommend fine tunning and when do you recommend vector database

[@ShawhinTalebi]

Great question! Generally, fine-tuning and RAG have different strengths. Fine-tuning is great when you want to endow the model with a particular style or to tailor completions for a particular use case, while RAG is good to provide the model with specialized and specific knowledge.

[@lauraharyo1128]

Thanks a lot for such a straightforward walkthrough! I tried a similar code for a text generation model, but I keep getting the error 'ValueError: prefetch_factor option could only be specified in multiprocessing. Let num_workers > 0 to enable multiprocessing.' Do you know why this keeps happening? I've even tried changing the torch version, but it's not working.

[@ShawhinTalebi]

Not sure that could be. Does the machine have a GPU?

[@lauraharyo1128]

@@ShawhinTalebi Thanks for your help! I figured out the issue was an outdated Linux kernel.

[@amanpreetsingh8100]

This was a great video. I have one question though. In the LoRA demonstration in your video(at ~14 minutes) you mention this operation (W0 + BA)x = h(x), in this how the sum (W0 + BA) is possible, as W0 has dimentions d*k, and output of operation BA would have the dimentions r*r. This matrix sum is not mathematiaclly possible. So can you elaborate more on this...

[@ShawhinTalebi]

Good question! The math works out here because B is d x r and A is r x k, therefore BA will be d x k.

[@amanpreetsingh8100]

@@ShawhinTalebi 👍

[@SolomonManuelraj]

Nice Video. I need your help to clarify my doubt. When we do the PEFT based finetuning, the final finetuned model size (in KBs/GBs) will increase by the additional parameters ( base model size + additional parameters size) . In this case base model size will be lesser and final finetuned model size will be more. Deploying the final finetuned model in the edge devices will be more difficult because of the limited edge device resources. Are there any way adapters / LoRA can help in reducing the final finetuned model memory size so that easily we can deploy the final model in the edge devices? Your insights will be helpful. Currently i am working in the vision foundation model deployment in the edge device where i am finding it difficult to deploy because of vision foundation model memory size and inference speed.

[@ShawhinTalebi]

Great question. PEFT methods like LoRA only reduce the number of trainable parameters not the total number of parameters. And to your point, the storage requirements actually increase in the case of LoRA! To reduce the final model size, you will need to fine-tune a smaller base model. Hope that helps!

[@samadhanpawar6554]

Can you recommend any course where i can learn to build llm from scratch and fine-tune in depth

[@ShawhinTalebi]

Paul Iusztin has some good content on that. Hands-on-llms: github.com/iusztinpaul/hands-on-llms More resources: www.pauliusztin.me/

[@RocktCityTim]

Hi Shaw - this answered so many questions about specializing an LLM in concise terms, thanks! One question that I'm running up against is physical machine abilities (CPU Speed/Cores, System Memory, GPU cores and memory, and storage speeds. In my case, I have a 32/64 core/thread Epyc CPU on PCIE4.0 MB with 128GB of DDR4 RAM and a PNY/NVIDIA RTX A5000 with 24GB DDR5 VRAM and 8192 CUDA cores dedicated to ML/AI (video is via a separate RTX A2000 GPU). With that info, what should I be looking at as a starting point that will take full advantage of those specs in local mode?

[@ShawhinTalebi]

Wow that's a lot of firepower. While I'm less knowledgeable about the ML engineering side of things, I'd suggest checking out DeepSpeed: github.com/microsoft/DeepSpeed. They have several resources on training/running large models efficiently.

[@RocktCityTim]

@@ShawhinTalebi Thatnks for the pointer. And thinks for all of your output. I've picked up some great information.

[@elrecreoadan878]

Would a botpress with a vector kb connected to chatgpt would be enough for Q&A ? When fine tuning starts to be neededvand is there an inexpensive way to do it with no or low code? Thank you!

[@ShawhinTalebi]

This depends on the use case. However, taking a quick-and-easy no code approach to start is never a bad idea. It typically gives you a sense of how sophisticated approaches will pan out. Fine-tuning will come into play when the "quick-and-easy" starts to becomes too inconvenient (or expensive) due to the scale of the solution. Hope that helps!

[@aketo8082]

Thank you. Is there a chance to create own LLM on own computer? A small version? Thank you for information.

[@ShawhinTalebi]

It depends what you consider a "Large" Language Model. ~100M parameters is probably the practical limit for (heavy-duty) consumer hardware, at least for now.

[@aketo8082]

@@ShawhinTalebi Maybe there is a small standard LLM available, which is possible to extend/train/finetune with own data. So the first step for the language rule are available. I have now idea if this is possible, that's why I ask, but could be possible.

[@harshanaru1501]

Such a great video ! Wondering how self supervised fine tuning works. Is there any video available on that ?

[@ShawhinTalebi]

Thanks! I found this on self-supervised fine-tuning: kzbin.info/www/bejne/i6SYi3mmZpuDl7c