LoRA explained (and a bit about precision and quantization)

[@khangvutien2538]

This is one of the easiest to follow explanations of LoRA that I’ve seen. Thanks a lot.

[@DeepFindr]

Glad you found it useful!

[@InturnetHaetMachine]

Another great video. I appreciate that you don't skip on giving context and lay a good foundation. Makes understanding a lot easier. Thanks!

[@omgwenxx]

Amazing video, feel like I finally understood every aspect of LoRA, thank you!

[@DeepFindr]

Glad it was helpful :)

[@pokhrel3794]

The best explanation i found in internet

[@vimukthisadithya6239]

This is a perfect explanation for LoRA I found so far !!!

[@teleprint-me]

I've been scouring for a video like this. You're the best explanation so far!

[@CarolineWu0719]

thank you for your great explanation

[@aurkom]

Awesome! Waiting for a video on implementing LoRA from scratch in pytorch.

[@lakshman587]

Thanks for the video!!

[@chrisschrumm6467]

Nice job with summarizing transfer learning and LoRA!

[@shubhamtalks9718]

Bro u killed it. Best explanation. Trust me I have watched all tutorials but all other explanations were shitty. Please create one video on quantization.

[@amulya1284]

you make the best explanation videos everrrr! is there one on how to train custom models using LORA?

[@mohamedezzat5048]

Thanks a lot Amazing explanation, very clear and straightforward

[@abhirampemmaraju6339]

Very good explanation

[@beyond_infinity16]

Explained quite well !

[@nomad_3d]

Good summary! Next time it would be great if you add headings to the tables that you show on the video. Sometimes it is hard to follow. For example, what is computational efficiency? is it inference time or inference time increase over the increase in performance (e.g. accuracy, recall, etc.)? Thanks.

[@marjanshahi979]

Amazing explanation! Thanks a lot!

[@이길현-p7f]

perfect video

[@k_1_1_2_3_5]

What an excellent video!! Congrats!!

[@WangZhen-r2d]

great video to explain lora！ thanks

[@unclecode]

Yes, indeed was hrlpful! Do you have a video on quantization?

[@sougatabhattacharya6703]

Good explanation

[@Canbay12]

Thank you very much for this amazing vide. However, although this was probably only for demo purposes of a forward pass after LoRA finetuning; the modified forward pass method you`ve shown might be mislieading; since the forward pass of the function is assumed to be entirely linear. So, does the addition of the LoRA finetuned weights to the base model weights happen directly within model weights file (like .safetensors) or can it be done on a higher level on pytorch or tensorflow?

[@ScoobyDoo-xu6oi]

Do you know what is \Delta W? How is it defined?

[@msfasha]

Brilliant

[@SambitTripathy]

After watching many LoRA videos, this one finally makes me satisfied. I have a question: I see in the fine tuning code, they talk about merging lora adapters. What is that? Is this h + = x @ (W_A @ W_B) * alpha ? Can you mix and match adapters to improve the evaluation score?

[@moonly3781]

I'm interested in fine-tuning a Large Language Model to specialize in specific knowledge, for example about fish species, such as which fish can be found in certain seas or which are prohibited from fishing. Could you guide me on how to prepare a dataset for this purpose? Should I structure it as simple input-output pairs (e.g., 'What fish are in the Mediterranean Sea?' -> 'XX fish can be found in the Mediterranean Sea'), or is it better to create a more complex dataset with multiple columns containing various details about each fish species? Any advice on dataset preparation for fine-tuning an LLM in this context would be greatly appreciated. Thanks in advance!"

[@binfos7434]

Really Helpful!

[@ahmadalis1517]

XAI techniques on LLMs is really interesting topic! When you would consider it?

[@susdoge3767]

gold

[@flecart]

good job!

[@ibongamtrang7247]

Thanks

[@prashantlawhatre7007]

please make video on QLoRA

[@aron2922]

Another great video, keep it up!

[@xugefu]

Thanks!

[@DeepFindr]

Thank you!

[@ArunkumarMTamil]

how is Lora fine-tuning track changes from creating two decomposition matrix? How the ΔW is determined?

[@ScoobyDoo-xu6oi]

same question

[@henrywang4010]

Great video! Liked and subscribed

[@poketopa1234]

I think the lora is scaled by the square root of the rank, not the rank.

[@Menor55672]

How do you make the illustrations ?

[@darshandv10]

What softwares do you use to make videos?

[@dennislinnert5476]

Amazing!

[@ad_academy]

Good video

[@alkodjdjd]

As clear as mud

[@anudeepk7390]

Is it a compliment or no? Cause mud is not clear.

[@iloos7457]

​@@anudeepk7390😂😂😂😂😂😂

[@prashantiyer6659]

It was pretty clear why u hating

[@kutilkol]

Ideot read paper. Lol

[@keremkosif1348]

thanks