Deep Dive: Quantizing Large Language Models, part 1

[@matthewrice7590]

Thanks for this, Julien

[@juliensimonfr]

you're welcome :)

[@joaogalego7229]

Watching this at 1.25x speed. High-quality content as usual. Keep it up, Julien 💪

[@itayatelis2898]

Love your content! thank you!

[@juliensimonfr]

Glad you enjoy it!

[@road2nohand]

Glorious Content :D

[@juliensimonfr]

Glad you like it!

[@Joe-nh9fy]

Great explanation! I have one question... Is it common practice to regularize the LLM cost function like with L2 to reduce the weight "outliers" while training?

[@juliensimonfr]

I don't think there is a strong consensus. It looks like regularization during fine-tuning can help with generalization. There are new ideas too, like noisy embeddings wandb.ai/byyoung3/ml-news/reports/A-New-Method-For-LLM-Regularization--Vmlldzo1ODIyMzIw

[@bibiworm]

I have been wanting to understand quantization for a very long time. Thank you! Would you mind sharing the slides please? Thank you.

[@juliensimonfr]

Hi, you can find the slides on Slideshare at fr.slideshare.net/slideshow/julien-simon-deep-dive-quantizing-llms/270921785

[@jacehua7334]

🔥 🔥 🔥

[@juliensimonfr]

:)

[@AI-Projects24]

Is there any chance to get the slides? Its very well organized and presented. Thank you so much for your work✨🔥🔥

[@juliensimonfr]

Hi, you can find the slides on Slideshare at Slides: fr.slideshare.net/slideshow/julien-simon-deep-dive-quantizing-llms/270921785

[@monishostwal8255]

what is meant by calibration dataset? is it eqivalent to evaluation set?

[@juliensimonfr]

Pretty much, yes. It's used to figure out the "best" hyperparameter values.

[@monishostwal8255]

okay got it thanks

[@caiyu538]

👍

[@juliensimonfr]

😃