Efficient Fine-Tuning for Llama-v2-7b on a Single GPU

[@thelinuxkid]

Very helpful! Already trained llama-2 with custom classifications using the cookbook. Thanks!

[@craigrichards5472]

Amazing, can’t wait to play and train my first model 🎉

[@thedelicatecook2]

Well this was simply excellent, thank you 🙏🏻

[@dinupavithran]

Very informative. Direct and to the point content in a easily understandable presentation.

[@ggm4857]

I like to kindly request @DeepLearningAI to prepare such hands-on workshop on fine-tunning Source Code Models.

[@Deeplearningai]

Don't miss our short course on the subject! www.deeplearning.ai/short-courses/finetuning-large-language-models/

[@ggm4857]

@@Deeplearningai , Wow thanks.

[@ab8891]

Excellent xtal clear surgery on GPU VRAM utilization...

[@KarimMarbouh]

🖖alignement by sectoring hyperparameters in behaviour, nice one

[@andres.yodars]

One of the most complete videos. Must watch

[@manojselvakumar4262]

Great content, well presented!

[@tomhavy]

Thank you!

[@zubairdotnet]

Nvidia H100 GPU on Lambda labs is just $2/hr, I am using it for past few months unlike $12.29/hr on AWS as shown in the slide. I get it, it's still not cheap but just worth mentioning here

[@pieromolino_pb]

You are right, we reported the AWS price there as it's hte most popular option and it was not practical to show all the pricing of all the vendors. But yes you can get them for cheaper elsewhere like from Lambda, thanks for pointing it out

[@rankun203]

Last time I tried it, H100s are out of stock on Lambda

[@zubairdotnet]

@@rankun203 They are available only in specific region mine is in Utah, I don't think they have expanded it plus there is no storage available in this region meaning if you shut down your instance, all data is lost

[@Abraham_writes_random_code]

together AI is $1.4/hr on your own fine tuned model :)

[@PieroMolino]

@@Abraham_writes_random_code Predibase is cheaper than that

[@Ev3ntHorizon]

Excellent coverage, thankyou.

[@dudepowpow]

28 zoom notifications! Travis working too hard

[@karanjakhar]

Really helpful. Thank you 👍

[@Ay-fj6xf]

Great video, thank you!

[@TheGargalon]

And I was under the delusion that I would be able to fine-tune the 70B param model on my 4090. Oh well...

[@iukeay]

I got a 40b model working on a 4090

[@TheGargalon]

@@iukeay Did you fine tune it, or just inference?

[@ahsanulhaque4811]

70B param? hahaha.

[@msfasha]

Clear and informative, thanx.

[@goelnikhils]

Amazing Content of fine tuning LLM

[@nguyenanhnguyen7658]

Very helpful. Thanks.

[@rajgothi2633]

amazing video

[@ayushyadav-bm2to]

What's the music in the beginning, can't shake it off

[@rgeromegnace]

Eh, c'était super. Merci beaucoup!

[@jirikosek3714]

Great job, thumbs up!

[@ggm4857]

Hello everyone, I would be so happy if the recorded video have caption/subtitles.

[@kaifeekhan_25]

Right

[@dmf500]

it does, you just have to enable it! 😂

[@kaifeekhan_25]

​@@dmf500now it is enabled😂

[@bachbouch]

Amazing ❤

[@nekro9t2]

Please can you provide a link to the slides?

[@nminhptnk]

I ran Colab T4 and still got into “RuntimeError: CUDA Out of memory”. Any thing else I can do please?

[@stalinamirtharaj1353]

@pieromolino_pb -Is Ludwig allows to locally download and deploy the fine-tuned model?

[@hemanth8195]

Thankyou

[@pickaxe-support]

Cool video. If I want to fine-tune it on a single specific tassk (keyword extraction), should I first train an instruction-tuned model, and then train that on my specific task? Or mix the datasets together?

[@shubhramishra8698]

also working on keyword extraction! I was wondering if you'd had any success fine tuning?

[@PickaxeAI]

at 51:30 he says don't repeat the same prompt in the training data. What if I am fine-tuning the model on a single task but with thousands of different inputs for the same prompt?

[@brandtbealx]

It will cause overfitting. It would be similar to training an image classifier with a 1000 pictures of roses and only one lilly, then asking it to predict both classes with good accuracy. You want the data to have a normal distribution around your problem space.

[@satyamgupta2182]

@PickaxeAI Did you come across a solution for this?

[@manojselvakumar4262]

Can you give an example for the task? I'm trying to understand in what situation you'd need different completions for the same prompt

[@rachadlakis1]

can we have the slides plz ?

[@SDAravind]

can you share the slide, please?

[@leepro]

Cool! ❤

[@arjunaaround4013]

❤❤❤

[@feysalmustak9604]

How long did the entire training process take?

[@edwardduda4222]

Depends on your hardware, dataset, and hyper parameters you’re manipulating. The training process is the longest phase in developing a model.

[@kevinehsani3358]

epochs=3, since we are fine tunning, would epochs=1 would suffice?

[@pieromolino_pb]

It really depends on the dataset. Ludwig has also an early stopping mechanism where you can specify the number of epochs (or steps) without improvement before stopping, so you could set epochs to a relatively large number and have the early stopping take care of not wasting compute time

[@Neberheim]

This seems to make a case for Apple Silicon for training. The M3 Max performs close to an RTX 3080, but with access to up to 192GB of memory.

[@ahsanulhaque4811]

Did you try on Apple silicon M1 Max?

[@mohammadrezagh4881]

when I run the code in Perform Inference, I frequently receive ValueError: If `eos_token_id` is defined, make sure that `pad_token_id` is defined. what should I do?

[@arnavgrg]

This is now fixed on Ludwig master!