LoRA & QLoRA Fine-tuning Explained In-Depth

[@DanielTompkinsGuitar]

Thanks! This is among the clearest and most concise explanations of LoRA and QLoRA. Really great job.

[@titusfx]

🎯 Key Takeaways for quick navigation: 00:00 🤖 *Introduction to Low Rank Adaptation (LoRA) and QLoRA* - LoRA is a parameter-efficient fine-tuning method for large language models. - Explains the need for efficient fine-tuning in the training process of large language models. 02:29 🛡️ *Challenges of Full Parameter Fine-Tuning* - Full parameter fine-tuning updates all model weights, requiring massive memory. - Limits fine-tuning to very large GPUs or GPU clusters due to memory constraints. 04:19 💼 *How LoRA Solves the Memory Problem* - LoRA tracks changes to model weights instead of directly updating all parameters. - It uses rank-one matrices to efficiently calculate weight changes. 06:11 🎯 *Choosing the Right Rank for LoRA* - Rank determines the precision of the final output table in LoRA fine-tuning. - For most tasks, rank can be set lower without sacrificing performance. 08:12 🔍 *Introduction to Quantized LoRA (QLoRA)* - QLoRA is a quantized version of LoRA that reduces model size without losing precision. - It exploits the normal distribution of parameters to achieve compression and recovery. 10:46 📈 *Hyperparameters in LoRA and QLoRA* - Discusses hyperparameters like rank, alpha, and dropout in LoRA and QLoRA. - The importance of training all layers and the relationship between alpha and rank. 13:30 🧩 *Fine-Tuning with LoRA and QLoRA in Practice* - Emphasizes the need to experiment with hyperparameters based on your specific data. - Highlights the ease of using LoRA with integrations like Replicate and Gradient.

[@Vinayakan-s4y]

I have been using thiese techniques for a while now without having a good understanding of each of the prameters. Thanks for giving a good overview of both the techniques and the papers

[@mandrakexTV]

This is the best detailed video and nicest explanation on youtube right now. I do think your channel will grow because you are doing an EXCELENT job. Thank you man.

[@andrepemmelaar8728]

Very useful! Marvelous clear explanation with the right amount of detail about a subject that’s worth understanding

[@gayathrisaranath666]

Thanks for this clear explanation about the topic! Your way of relating back to research papers is very interesting and helpful!

[@drstrangeluv1680]

I loved the explanation! Please make more such videos!

[@naevan1]

I love this video man. watched it at least 3 times and came back to it before a job interview also. Please do more tutorials /explanations !

[@thelitbit]

great video! referring to the paper and explaining each thing in detail really helps understand the concept to the fullest. Kudos!

[@steve_wk]

I've watched a couple other of your videos - you're a very good teacher - thanks for doing this.

[@SanjaySingh-gj2kq]

Good explanation of LoRA and QLoRA

[@VerdonTrigance]

It was incredible and very helpful video. Thank you man!

[@SantoshGupta-jn1wn]

great video, i think the best explanation i've seen on this, i'm also really confused about why they picked the rank and alpha that they did.

[@user-wr4yl7tx3w]

This is really well presented

[@YLprime]

Dude u look like the lich king with those blue eyes

[@practicemail3227]

True. 😅 He should be in acting career ig.

[@EntryPointAI]

You mean Lich King looks like me I think 🤪

[@varun_skywalker]

This is really helpful, Thank you!!

[@brianbarnes746]

Great explanation, best that I've seen

[@anujlahoty8022]

Loved the contnt! Simply explained no BS.

[@omarsherif88]

Very useful, thank you!

[@CatarinaReis-g3y]

Thisa saved me. Thank you. Keep doing this :)

[@markironmonger223]

This was wonderfully educational and very easy to follow. That either it makes you a great educator or me an idiot :P Regardless, thank you.

[@EntryPointAI]

let's both say it's the former and call it good! 🤣

[@stutters3772]

This video deserves more likes

[@Sonic2kDBS]

Some nice details here. Keep on.

[@AbdoGhazala-y5p]

can you share the presentation document

[@louisrose7823]

Great video!

[@archchana7756]

very well explained, thanks :)

[@nachiketkathoke8281]

really grate explanation

[@user-wp8yx]

I'm pulling for another vid on alpha. Oobabooga suggests twice your rank. The Chinese alpaca lora people use a rank 8 with alpha 32 and I guess it worked. I've tried high alphas that make the model kinda crazy. Need guidence.

[@EntryPointAI]

When in doubt, set alpha = rank for the effective scale factor to be 1. There are better ways to have a larger impact on training than bluntly multiplying the change in weights, like improving your dataset or dialing in the learning rate.

[@user-wp8yx]

@@EntryPointAI this does make sense they way you put it. Thanks so much for your reply!

[@aashwinsharma8194]

Great explanation...

[@TheBojda]

Nice video, congrats! LoRA is about fine-tuning, but is it possible to use it to compress the original matrices to speed up inference? I mean decompose the original model's original weight matrices to products of low-rank matrices to reduce the number of weights.

[@rishiktiwari]

I think you mean distillation with quantisation?

[@EntryPointAI]

Seems worth looking into, but I couldn't give you a definitive answer on what the pros/cons would be. Intuitively I would expect it could reduce the memory footprint but that it wouldn't be any faster.

[@TheBojda]

@@rishiktiwari Ty. I learned something new. :) If I understand well, this is a form of distillation.

[@rishiktiwari]

​@@TheBojdaCheers mate! Yes, in distillation there is student-teacher configuration and the student tries to be like teacher with less parameters (aka. weights). This can also be combined with quantisation to reduce memory footprint.

[@RafaelPierre-vo2rq]

Awesome explanation! Which camera you use?

[@EntryPointAI]

Thanks, it’s a Canon 6d Mk II

[@Gayatritravelandfitnessvlogs]

Thanks a ton!

[@nafassaadat8326]

can we use QLoRA in a simple ML model like CNN for image classification ?

[@SergieArizandieta]

wow I'm noobie in this field n I been testing fine-tunen my own chatbot with differents techniques, n I found a lot of stuff, but It's not commonly find a some explanation to understand the main reason of the use of it, ty a lot < 3

[@kunalnikam9112]

In LoRA, Wupdated = Wo + BA, where B and A are decomposed matrices with low ranks, so i wanted to ask you that what does the parameters of B and A represent like are they both the parameters of pre trained model, or both are the parameters of target dataset, or else one (B) represents pre-trained model parameters and the other (A) represents target dataset parameters, please answer as soon as possible

[@EntryPointAI]

Wo would be the original model parameters. A and B multiplied together represent the changes to the original parameters learned from your fine-tuning. So together they represent the difference between your final fine-tuned model parameters and the original model parameters. Individually A and B don't represent anything, they are just intermediate stores of data that save memory.

[@kunalnikam9112]

@@EntryPointAI got it!! Thank you

[@ArunkumarMTamil]

how is Lora fine-tuning track changes from creating two decomposition matrix?

[@EntryPointAI]

The matrices are multiplied together and the result is the changes to the LLM's weights. It should be explained clearly in the video, it may help to rewatch.

[@ArunkumarMTamil]

@EntryPointAI My understanding: Orignal weight = 10 * 10 to form a two decomposed matrices A and B let's take the rank as 1 so, The A is 10 * 1 and B is 1 * 10 total trainable parameters is A + B = 20 In Lora even without any dataset training if we simply add the A and B matrices with original matric we can improve the accuracy slighty And if we use custom dataset in Lora the custom dataset matrices will captured by A and B matrices Am I right @EntryPointAI?

[@EntryPointAI]

@@ArunkumarMTamil Trainable parameters math looks right. But these decomposed matrices will be initialized as all zeroes so adding them without any custom training dataset will have no effect.

[@NathanielMaymon]

What's the name of the paper you referenced in the video?

[@EntryPointAI]

Here's LoRA: arxiv.org/abs/2106.09685 and QLoRA: arxiv.org/abs/2305.14314

[@tgzhu3258]

so good!!

[@UfcFan-d6s]

Amazing for struggling students. Love from Korea😂

[@egonkirchof]

Why do we call training a model pre-training it ?

[@EntryPointAI]

Not sure if that's a rhetorical question, but I'll give it a go. You can call it just "training," but that might imply that it's ready to do something useful when you're done. If you call it "pre-training" it implies that you'll train it more afterward, which is generally true. So it may be useful in being a little more specific.

[@chrisanderson1513]

Saving me somr embarrassment in future work meetings. :) thanks for sharing.

[@princekhunt1]

Nice

[@mugomuiruri2313]

good

[@Code-and-Chords-s2g]

8:06 Qlora

[@Ian-fo9vh]

Bright eyes

[@DrJaneLuciferian]

I wish people would actually share links to papers they reference...

[@EntryPointAI]

LoRA: arxiv.org/abs/2106.09685 QLoRA: arxiv.org/abs/2305.14314 Click "Download PDF" in top right to view the actual papers.

[@DrJaneLuciferian]

@@EntryPointAI Thank you, that's kind. I did already go look it up. Sorry I was frustrated. It's very common for people to forget to putlikes to papers in show note :^)

[@vediodiary1754]

Oh my god your eyes 😍😍😍😍everybody deserves hot teacher😂❤

[@Larimuss]

QLORA let's me train on a 4070ti with only 12gb vram. Though I can't go over 7b model

[@nabereon]

Are you trying to hypnotize us with those eyes 😜

[@rohitvishwakarma2871]

Gojo ?

[@619vijay]

Eyes!

[@TR-707]

Ahh very interesting thank you! *goes to fine tune pictures of anime girls*

[@kritarthlohomi3305]

bradley cooper in limitless tf

[@ecotts]

LoRa (Long Range) is a physical proprietary radio communication technique that uses a spread spectrum modulation technique derived from chirp spread spectrum. It's a low powered wireless platform that has become the de facto wireless platform of Internet of Things (IoT). Get your own acronym! 😂

[@EntryPointAI]

Fair - didn’t create it, just explaining it 😂

[@Ben_dover5736]

your have beautiful eyes

[@EntryPointAI]

Thank you!

[@coco-ge4xg]

omg I always distracted by his blue eyes😆and ignoring what his talking

[@partymarty1856]

blud why your eyes like that