QLoRA paper explained (Efficient Finetuning of Quantized LLMs)

[@IgorAherne]

Thank you, that's a beautiful explanation! One thing I struggle understanding, is the term quantization blocks in 4:30 - why we need several of them. In my understanding from the video, we ponder about using 3 blocks of 16 bits to describe a number. Which is 48 bits and is more expensive than 32-bit float. But couldn't we just use 16*3 = 48 bits per number instead? Using 48 bits (without splitting it) would give us a very high precision within [0,1] range, due to powers-of-two I did ask GPT, and it responded that there exists a 'Scale Factor' and a 'Zero-Point', which are constants that shift and stretch the distribution in 6:02 Although I do understand these might be those quantization constants, - I am not entirely sure what the 64 blocks described in the video are 6:52 Is this because of the Rank of Matrix-Decompositions is 1, with 64 entries in both vectors?

[@pierluigiurru962]

Your videos on LoRA finally made the concepts click for me. It was clearly explained! Thank you for the content you make

[@AIBites]

Glad it helped. Welcome 😊

[@haz5248]

That was really well explained with intuitative diagrams and explination. Thanks for the video , just subscribed .

[@AIBites]

thank you! :)

[@cacamaricano]

Thanks for connecting the dots!

[@AIBites]

glad you liked! :)

[@SudarakaYasindu]

Awesome explanation! ❤

[@AIBites]

glad you think so and thank you indeed :)

[@vuluu4942]

Thank you for the explanation! I find that it's very helpful!

[@huitangtt]

Very well explained

[@AIBites]

Thanks so much 😊

[@JaishreeramCoder]

amazing explanation

[@AIBites]

Glad you think so!

[@wilfredomartel7781]

Waiting to see it.

[@AIBites]

Sure! :)

[@rahul.vpoojari6553]

Thank you sire

[@AIBites]

my pleasure Rahul! :-)

[@yayasy1362]

I don’t understand why you say that LoRA is fast for inference… in any case you need to forward through the full rank pretrained weights + low-rank finetuned weights.

[@AIBites]

ah yes. If only we could quantize the weights, we can do better than the pre-trained weights. You are making a fair point here. Awesome and thank you! :)

[@yayasy1362]

@@AIBites Yeah, if only we could replace the pretrained Full-Rank weights by the Low-Rank Weights... really nice video and illustrations! Thanks a lot!

[@yeduniya657]

Hey. I need your help. I have a curated set of notes and books and I wish to use it to finetune a model. How can it be done?

[@AIBites]

would you like to see a fine-tuning video on text data? Would that be useful? Do you have any suggestions on the dataset I can show fine-tuning on?

[@yeduniya657]

@@AIBites Yes, I have suggestion. Finetuning a model on my journal in which I have written about the truth of nonduality and illusionary nature of reality. I am also actively curating books on truth, and would love your help.

[@haz5248]

@@AIBites That would be very help. There arent many good videos on fine tune out there.

[@AIBites]

hope the fine-tuning video was of some help

[@AIBites]

hope the fine-tuning video was of some help