How to Fine-Tune LLama-3.2 Vision language Model on Custom Dataset.

[@sarithamiryala2819]

Nice video

[@nextGenAIGuy490]

Thanks

[@ChandanKumar-nr2vm]

Thanks you sir this video help me to understand the this model in very first video

[@nextGenAIGuy490]

Glad it helped

[@georffreyarevalo3067]

Good video, how can I test the model that push to Hugging Face? Could you please share an example.

[@nextGenAIGuy490]

Thanks. You can use AutoModelForVision2Seq to load your model. You need to pass your model path and use huggingface access token.

[@soulaimanebahi741]

thank you for the démonstration. do you think can we fine tune this model on a videos data?

[@nextGenAIGuy490]

@@soulaimanebahi741 No, We can't.

[@babusd]

Absolutely wrong ! If you dont know say "dont know" . Dont mislead him , fine tuning over video is possible

[@nextGenAIGuy490]

@@babusd relax bro. Do one thing rather than saying show the proof. During training on llama 3.2 vision model they have used image and text pair. Read the model architecture. And if you know show me where they have written we can Fine-Tune vision model on videos.

[@tamilselvan3525]

how long will it takes for the whole process?

[@nextGenAIGuy490]

@@tamilselvan3525 I haven't trained completely because of GPU limitation. So i won't be able to answer. I just wanted to share that its possible to train and how to train. But training time is dependent on dataset, hardware(GPU configuration) and no. of epochs you are training for.

[@tamilselvan3525]

@@nextGenAIGuy490 Okay, thanks.

[@mohammadaqib4275]

w e are fine tuning llama .2 vision model but collate functionwas utilising Qwen2. IS it fine to use Qwen model in collate function while fine tuning llama-3.2?

[@nextGenAIGuy490]

By customizing the collate_fn, we are able to control how the data is prepared. we are using it for batch processing, padding bringing data into format to train the model. Its fine to use it.