EASIEST Way to Fine-Tune LLAMA-3.2 and Run it in Ollama

[@kunalr_ai]

Here are the key points from the video: * Meta released a new family of four different models, including multimodal models, called LLaMA 3.2. * The models are impressive for both language and vision tasks for their respective sizes. * You can fine-tune LLaMA 3.2 for your own custom tasks. * You can use Unslot for fine-tuning and Ollama for running the fine-tuned model locally. * The 1 and 3 billion models are particularly interesting because you can run them on device. * Meta has also released LLaMA Stack, which is their opinionated version of how developer experience should look. * You can fine-tune one of the smaller models on your own data set and then run it locally using Ollama. * You will need to provide your own data set and follow the specific prompt template used by the model. * You can use the official notebook from the Unslot team to fine-tune LLaMA 3.2. * You can use the supervised fine tuning trainer from the TRL library to train the model. * You can save the trained model as a GGf file and then load it in Ollama. * You can create a model file in Ollama and then run the model using the AMA run command. Timeline with tags: 00:00 - 00:15: Introduction 00:15 - 02:00: Meta releases LLaMA 3.2 02:00 - 04:00: LLaMA 3.2 models 04:00 - 06:00: Fine-tuning LLaMA 3.2 06:00 - 08:00: Unslot and Ollama 08:00 - 10:00: 1 and 3 billion models 10:00 - 12:00: LLaMA Stack 12:00 - 14:00: Fine-tuning LLaMA 3.2 on your own data set 14:00 - 16:00: Prompt template 16:00 - 18:00: Unslot notebook 18:00 - 20:00: Supervised fine tuning trainer 20:00 - 22:00: Saving the trained model 22:00 - 24:00: Running the model in Ollama

[@HigherLevelServices]

Response has Hallucinations

[@epokaixyz]

Consider this your cheat sheet for applying the video's advice: 1. Research the different sizes of Llama 3.2 models. 2. Download the Unsloth Fine-Tuning Notebook. 3. Acquire the FineTome-100k dataset. 4. Fine-tune a Llama 3.2 model with Unsloth, using LoRA adapters and prompt engineering. 5. Create an Ollama model file for your fine-tuned model. 6. Run your fine-tuned Llama 3.2 model locally with Ollama. 7. Start building custom AI applications!

[@Criminal_H4_ff]

Broo till step 6 i have completed but i cant run my finetunned model in ollama what should i do now 🤧

[@gramnegrod]

Great video. You make it look so easy! I’m really looking forward to the vision based rag. I’m hoping good vision models with vision rag will open up a lot of creative use cases.

[@engineerprompt]

Here are a couple of examples of vision based RAG: kzbin.info/www/bejne/rWa6eJWLfKZ4adU kzbin.info/www/bejne/enqcgmlmicRja5Y

[@lulzkiller666]

Nice video. Could you please make a video on how to train it on "own" content. Lets say, i have the complete API documentation for an APP, i want to train it on this API documentation so that it can help me code faster with the correct API's. That would be awesome

[@tanguyvansnick3954]

Great tutorial 🔥

[@maxwelikow9119]

Cool video! Do you have an idea how to fine-tune the llama3.2-vision models?

[@deschwedda]

I want to exceed limitations and remove censorships. Is it possible, and how? thank you so much.

[@Incredible_428]

You need to fine-tune it with a dataset which contains uncensored chat data, it should be well mannered/structured so that the model will learn batter patterns

[@deschwedda]

@@Incredible_428 thank you, any dataset recommendations? (llama 3.2)

[@engineerprompt]

look for dolphin models, they are usually uncensored.

[@deschwedda]

@@engineerprompt thank you!

[@tnix80]

If you can jailbreak AI and the woke nonsense, a lot of people are going to want to use your jailbreaking technique/tool. I could see making a lot of money.

[@HarshSingh-cp8mq]

hey i want to build my personal assistant on the LLAMA3.2 and i want to assign a name to it. Also while asking the owner it tells me about meta this also i want to change?? Can anybody guide me

[@AshwaniKumar-r4p]

In the fine-tuning process demonstrated in the video, does the model primarily learn response patterns, or does it genuinely absorb and retain the specific knowledge contained in the training dataset?

[@lewists9475]

can a llama 3.2 vision be finetuned and run in ollama?

[@rezcan]

Great video, thanks can you make a video to show how to fine tune Llama 3.2 90B vision model?

[@dipeshrathore8842]

Great video! Can you please create a video or guide demonstrating Fine-tuning of Llama 3.1 8B First on raw text (books, discourses etc.) Then on instruction dataset (less data 8-10k)? And what's best? 8B-base or 8B-instruct for this?! (I don't wanna lose general chat capabilities)

[@Abhijit_SU]

I have used the same notebook to fine tune my model. I am getting an error saying "Keyerror: name" when i am trying to either push it to HF or saving it locally. After executing the GGUF / llama.cpp Conversion part it is running and then after 3 mins exact it is showing the error every time. Please tell me how did you manage to download the GGUF file locally using the same Notebook which You have provided. Please Help, Thanks In Advance !

[@rickymorty8198]

Getting the same error too

[@rickymorty8198]

Any solutions?

[@raunaksharma8638]

Can we use normal Alpaca type dataset with input , output and instruction here ?

[@lysanderAI]

Could you do a video of finetuning using axolotl + unsloth

[@surajsingh-iw8yt]

wht would be the format of custon tuning dataset, like file format jsonl or someting else

[@tapikoBlends]

is it possible to make fine-tuning using text?(not structured in json format) text will be tomething like instruction

[@aqibjamal1077]

make video for 11b vision model please

[@MuhammadAsif-mm4py]

Can i use this model in my android application? Please help

[@annwang5530]

Can that gguf run locally on DAN or LMStudio?

[@xmagcx1]

x2

[@engineerprompt]

Yup, on almost anything you want, if its based on llamacpp.

[@annwang5530]

@@engineerprompt do you take fine tune tasks? I got a Json dataset I fail to fine tune...

[@yan_yan_1995]

may i know the screen recording software he's using ? it's cute !

[@engineerprompt]

screen.studio :)

[@yan_yan_1995]

@@engineerprompt thank you !!!

[@Qwme5]

Can I finetune this llm with a new langauage like Arabic if so should I use the original tokenizer of llama 3.2. Another question , how much it will cost me on google colab to finetune such small model like 3B.

[@avataraang3334]

Nothing.. T4 gpu gives you around 1-3.5 hours of free resource.. thats plenty so smaller models

[@justinln6019]

Why do I get this error: RuntimeError: Unsloth: The file 'llama.cpp/llama-quantize' or 'llama.cpp/quantize' does not exist.

[@sergiosilveramurcia4216]

Has anyone tried to run it locally on MacOs, does it change the code substantially?

[@jackluka773]

why the heck it has to be so complicated? can't it be wrapped in some easy to use GUI with drop down list creator with description of the consequences for each choice?

[@yufeixu4479]

You go program it then 😂 this is easy already... if you can't do it then don't do it and don't hate buddy

[@equious8413]

This is cutting edge computing science. Paint by numbers aint here yet.

[@大支爺]

Uncensored patch first!

[@robertjalanda]

great video waiting for vision support

[@Criminal_H4_ff]

Brother i got error while doing the command ollama run mymodelname it throws the error as ollama runner function terminated and vocabulary and tokenizer merges files are not found issue what should i do now will you please any contact of yours i need immediate help bruh😮‍💨🥲