EASIEST Way to Fine-Tune a LLM and Use It With Ollama

[@warpdotdev]

What data would you fine tune your LLM on?

[@siaboonleong]

what is your hardware ? are you using intel or amd threadripper ?

[@themax2go]

that depends on the use case... are you asking for what use cases people are aiming to support?

[@themax2go]

but i guess here's one: finetuning qwen2.5-32b on various ai related github prjs that are written in python, with a focus on agentic workflows, maybe individual finetunes for each agent python lib / framework, such as autogen, langchain / langgraph, ...

[@shiftto]

youtube need more chanels alike this. great job

[@Toti330]

I'm using it to fine-tune a Minecraft bot (Mindcraft from Emergent garden)

[@siddhubhai2508]

No over rating, no over talking, streight forward, love it.

[@OilersFlash]

Sadly this may be the first time I disagree with that sentiment. Haha. I loved it but was looking for a course link at the end because it was too much too fast ;)

[@siddhubhai2508]

@@OilersFlash Yea bro (actually uncle, cuz I'm 15 and your dp shows...), I realized that the viewer will need some pre knowledge of LLMs and its workings and also some pre basic knowledge of fine-tuning! But yea the video was good actually!

[@OilersFlash]

@@siddhubhai2508 it is good ;)

[@122333Jordan]

100% this.

[@cjofre]

This is exactly how we are tuning our open source LLMs, the use of unsloth+LORA is key. Validation of the learning is an adventure. We did this over the same tech stack mentioned here. Very good video, of course, lots of details behind each step that cannot be explained in a short video. Great use of lighting and your pace is excellent. Looking forward to more vids.

[@Emocar2]

Thank you straight to the point I usually have some ptsd when looking at tutorials

[@thecaptain2000]

it depends, this is useful, just if you already know how to do it and you just need a refresh on the steps. In other words, you want to watch it just if you do not need it

[@eygs493]

hi i love you

[@dasarivashish]

I appreciate that this content is geared towards those with prior knowledge, but I think it would be really beneficial to create a beginner-friendly video that breaks down the process into clear, step-by-step instructions. That way, newbies like me can easily follow along and learn from your expertise.

[@lilyrooney]

seconding this. its hard to find good resources on making/using your own datasets when its not in such a friendly prepackaged format

[@raybod1775]

Keep searching KZbin.

[@injeranamitmita]

Knowing such a deep technical subject is one thing, but teaching it well is another! well done young lady.

[@Laowater]

amazing quality of editing, sound, video - beyond the programming side! A pleasure for the eyes and ears to watch!

[@first-thoughtgiver-of-will2456]

Great video for people who know coding and local llm but havent finetuned!

[@arianetrek7049]

I was already taken by the clear content but the KEY highlight of simply mentioning Conda shows the high quality of your informations. Subscribed and thanks for the tuto.

[@tekwon]

What a great tutorial, so much info in 5 mins. It took me a while to fill in the gaps because I didn't even know what Jupyter notebook was but I now have this running on my local GPU. You can (just) run this example with 16GB vram.

[@me20000006]

Brilliant video. Straight to the point. Really liked it.

[@gloriamaciam]

Amazing job! Great communicator - I just loved it!

[@ToastyWalrus7]

Easiest subscribe of my life, just wanted something easy and straight to the point

[@sommerliermovie3547]

This is what I am thinking to do. Finding this video maybe saved half a day

[@frobledo5584]

Insanely good video!! Straight to the point and great presentation

[@foju9365]

Five minutes made useful. Thank you for the crisp and neat video. ❤

[@skymeisterkai7589]

Thank you for the simple, straight forward insight. Hope you come out with more bite size knowledge.

[@robertjalanda]

great and short video guys incredible!!

[@AghaKhan9]

Hi there. It's wonderful.. Will you Please share the notebook and also Google Collab notebook?

[@eygs493]

no

[@mr.gk5]

Great video straight to the point, but could you please elaborate more on feeding custom dataset to the script? What are the steps? You wrote it right on the script or did you load it from a different file? Or did you upload to hugginface and use the token from there? Im confused

[@HellTriX]

Every how-to I've found so far glazes over the training datasets. Like here she goes from here is 100k sql dataset, then here are is how the model expects the prompt, but then doesn't show the format of how the dataset is loaded into the model. What does the sql_context look like, what does the sql_prompt look like, what is the sql, and sql_explaination? Fine tuning is one thing but people also need to know how to build and integrate datasets for training.

[@twistyx808]

Agreed. Every video is skipping over detail to actually build your own functionality.

[@jofus521]

That’s kind of the point. It’s open source open secret

[@David-Codes]

Did you find a better video that explains this?

[@Mr.Nobody7495]

Any one found better video than this

[@twistyx808]

@Mr.Nobody7495 unfortunately no. These people making videos about it don't actually know what they're talking about. They are just reading things off a script with no context. If you want to learn more, I can provide some insights on frameworks and approach to a this. Still learning though

[@pabloambrosio2185]

So nice and clear! Tank you a lot!

[@markring40]

Another great video!! Thanks.

[@mongstyt9946]

Immediately subscribed

[@nagalman]

Great video. To the point. Effective

[@chokosabe]

What a clean presentation.

[@FredRohn]

this is great! gonna try this...

[@TGIMonday]

Liked and subbed, so helpful, thank you!!

[@gladiar145]

Great video

[@anandraj2895]

nice and straight forward approach

[@0xngmi]

wouldve been nice if you had shared the full collab code...

[@_didi_the_best_all]

Guys, it is a collab from Unsloth. You can search on Google "Llama-3.1 8b unsloth" and you will find out.

[@BCRBCRBCRBCRBCRBCR]

Appreciate the brevity. Thank you.

[@flutterflowexpert]

Great video! Subscribed! 🎉

[@Hey.MangoJango]

Well explained. Thanks

[@rtATvw]

@warpdotdev In the interest of establishing the repeatability of your result can you indicate whether the output @ 5:01 is unedited? typical? Mine is amending the prompt and typically produces output in the format of the Alpaca training text generating entries of 'SQL Prompt', 'SQL', 'Explanation', 'Question', 'Instruction', 'Company database' in the output.

[@anirudhm]

Yes, same. Would be useful if they share the link to the code.

[@alpuhagame]

Would be nice to see how can we validate that training was successful.

[@anshulsingh8326]

Hi, which is better? Using rag, or training on that rag?

[@Larimuss]

Great tutorial! Thanks. Really like the definitions and explanations rather than just glossing over the code.. would love to see a full tutorial series 😮 I'd like to try to code it myself, but it would be good if you could share the code too.

[@nnpy]

Yoo, what theme are you using in your system?? That's really cool

[@damianoficara6694]

Regarding validation loss , is it normal that requires more time?

[@AIinAgriculture]

Thank you for video. I followed your approach for finetuning the model text to SQL. When I work on my own database, its performance is not good. Unable to generate SQL query. I even used agent and examples queries our database. I am requesting suggestion from you. Thank you

[@adrainhuang6068]

good guide to play by myself

[@lakshman587]

Intresting video!!

[@SMCGPRA]

Could you please let us know what is hardware configuration needed for pc or laptop to run ollama and train llm models

[@BrentLeVasseur]

Excellent tutorial! Doesn’t lowering the bit depth of the model greatly reduce accuracy? What are the pros and cons of doing so? Thanks!

[@jason77nhri]

Thank you for your informative tutorial. I’ve installed Ollama on Windows 10 and currently only use it for interactive conversations through CMD. However, I asked a friend, and they mentioned that Ollama cannot be used for fine-tuning. Why is that? Additionally, Unsloth offers convenient online fine-tuning, but if the training data requires privacy and fine-tuning needs to be done locally or even if the base model is in Safetensors format from others-where should I start? Are there any learning resources or references you would recommend? Thank you!

[@usaid3569]

learn from chatgpt

[@DaveAlexKD]

at 2:00 is 8 Billion not 8 bit.

[@TheArthurfonzarelli]

The 8-bit model is incredibly fast, just not very accurate. 😊

[@rival5016]

During training, why don't we provide an evaluation split?

[@RicardoEmmanuelSanchezMartinez]

It seems that it does not use one, I'm curious though, because in any model, you will need one to test how well one model does at the moment of inference, but I guess that Unsloth does it with the original train dataset, maybe splitting it?

[@telepathytoday]

How long did it take for the training / fine-tuning on your 4090? Thanks for the video!

[@elasdebastos235]

I actually liked the music. Great editing also. I found the proyect idea a little bit boring. You could train it for something actually awesome

[@shreyasbs2861]

Awesome explanation in short video….some KZbinrs consuming 2 to 3 hours for this content

[@JanMasata-b4m]

Is it possible to do this while not having a NVIDIA graphics card?

[@magnusjensen2767]

I would appreciate if this tutorial went more into detail on how to make a dataset comply with the model's promt template. From this video, there is no data processing being done and i think in real cases you would have to work with the data, or write a script, that chnges the data so that it fits with the prompt template (e.g. the Alpaca one she shows in the video)

[@brianchap2680]

Do you ever help companies with their own projects? I'm looking for some help on my build

[@4.0.4]

Can you train 12B on 24GB or is 12B too big? Another question is if you have multi-turn data (conversations) can you finetune on that? The examples I see are for Q:A pairs.

[@TheMariolino2005]

Excellent direct video on fine-tuning, congrats. Could you also share the python source code that you used in the video? Thanks.

[@batmajk]

Hey, I have a question. I'm currently using gpt api tokens to generate models response thru python and run it over and over. Models are using information from knowledge base folder -- it would be easier to manage it through llama? My ideal scenario is to generate content model, by model and then use that output to generate complete story. Which solutions will you recommend for that purpose? Thanks

[@elleryfamilia8291]

if I train a model, will the responses be restricted to just my training data/examples? Or does the model's original training also play a role? For example, say I trained it on model to respond with vehicle specs. I feed it a vehicle name and the model returns the vehicle specs. Say I finetune the model with 50 examples (honda civic, toyota camry, volvo xc90, etc).... however, in reality there are 300,000 possible vehicles. Would finetuning on 50 examples help? Sure, I can put the 50 models in the prompt but then I'd just be wasting tokens.

[@KavinMoudgil]

Can you provide me the link for your ubuntu desktop wallpaper 🙃

[@lesptitsoiseaux]

I have 55K classes with an ungodly amount of transcripts. What'll be better and faster? RAG or finetuning?

[@bnug242]

is the notebook posted anywhere?

[@lelouchlamperouge5910]

So we are just creating loras here, I can have many and mount them as I need like with Stable Diffusion right?

[@sikandarali5117]

I'm working on a RAG application. I'm using a pdf file as a text data and I have clean the data as well using NLTK. I already have ollama install in my local system and also llama2 model. I embed the data in to vector form using ollama embedded model mxbai-embed-large and store it on chromabd. but if I give the prompt and get response, it give me the response but not according to my data. and how do I finetune the model? I just need guidance and sequence so I work on the project. It's important to me. I need your guidance. please reply. I'm using windows10. Thank you!

[@ibrahimgamal7603]

It's easy you should write in your prompt template = """Answer the question based ONLY on the following context: {context} Question: {question} """

[@muhammadumarnawaz9200]

@@ibrahimgamal7603 Could be anything such as you are not using a good doc extractor or your embedding model sucks. Changing chunk and overlap size might help. Don't go for finetuning straight away, do some experimentation and figure out where the problem lies exactly otherwise even finetuning won't work.

[@sikandarali5117]

@@muhammadumarnawaz9200 ok mate thank you for your help

[@alucard00mx]

It was a great video, but I have a question, is it compatible with the new versions called 3.2, especially versions 1B and 3B?

[@iamakshobhyashukla]

hi, can you tell me how I can run a finetuned vision model based on llama-3.2-vision on Ollama. I ask this as unsloth does not support conversion of vision models to gguf .

[@furetosan]

Isn't this RAG? Doesn't chunking fit anywhere?

[@golodiassaid4879]

How do you train on 3Gb pdf and word files locally

[@IvanLesnov]

fine tuneing offline ?

[@programan6391]

Does fine-tuning shouldn't generate small models that theoretically run fully local and avoid spending money on OpenAI?

[@eduardmart1237]

Is unsloath free?

[@MFEeee]

Wow! Can this work on MAC?

[@valenciawalker6498]

Great explanation and presentation of LLM.

[@CiaoKizomba]

can you provide the code you used?

[@GARYGUS]

what is the estimate cost for training such a model?

[@DabnisUK]

Apologies if this is a dumb question, but where is, is there a link to the notebook?

[@MyNomDePlume]

8 bit? you mean 8 billion parameter model.

[@Lord_LindaThePhilosopher]

Where is the data? I did not see any data given to the model to use. I feel like its perfectly possible to have a model "train in realt time" where you tell a model go in to learning mode. Then it will "learn" or "memorize" data you give it. Specifically text based data. And then it has a chance of using it when you ask a question related to that data. Why has this not been done yet? Or has it? I guess what im talking about is somthing like custom GPTs. Which i feel would be eaiser and more worth it. Its just im looking for that but on device.

[@imai_official]

How to collect dataset pls show it

[@Rimbo28]

i think im in love

[@maloyaircraft1174]

Video on formatting data sets to follow? 😅

[@PMe-my1td]

can this be done all from the wsl terminal? I am using ollama, webui and docker, but would like to train some company pdf's better, can this be done?

[@sreeramn5240]

Thanks 🙏

[@infotainment6830]

Is it possible to make your own AI model with this that speaks to you like Jarvis and understands tone, etc?

[@arthurlee8961]

Does this support AdaLoRA or Vera?

[@sheraliabdukakhkharov7899]

Could you share the link for ipynb file?

[@oloix]

Thanks! Although I do have a question why you show so much of yourself... I admit it's nice and pleasant to look at, however I would prefer to look at the code or things you talk about. Especially when I watch your videos at work I would definitely like to avoid someone thinking that I watch some silly videos during work while I actually watch work related stuff. Anyway, really interesting videos so thanks again! 😊✌

[@noachere]

Do you have github repo of the code?

[@WiteBot]

Give us the collab link! Or better yet the llamasql gguf link!

[@frankdearr2772]

Great 👍

[@isbestlizard]

I think I'll just get an llm to make the dataset and make it llms both sides of the data->training loop

[@catarctic]

Congrats for finding such a smartie-cutie as a DevRel for Warp

[@CarlosRodriguez-xk9ot]

Make a video about json config and lobe chat config please or similar

[@benjaminbirdsey3281]

Unsloth has a dependency of triton, which doesn't seem to be compatible with windows.

[@superfreiheit1]

Can you make the code are smaller and remove the background music

[@SosanoMusic]

nice !

[@ItalianTiger955i]

Just for understand all the prerequisites, about the amount of stuff to install, i get absolutely lost and i didn't understand nothing

[@diasdeinvierno8041]

It's a great how to but you missed a "before trainng" and "after training" examples of how the model responded. Good video anyway. Couldn't get much out of it though

[@hasanaqeelabd-alabbas3180]

Inam finding difficulty understanding the converting of data set

[@hasanaqeelabd-alabbas3180]

Is ubuntu on windows ? Or dependent system ? I need more tiny steps to convert the dataset

[@hansgruber3495]

Nice video, but the music is way too disctracting.

[@StephanYazvinski]

nah, only a little distracting.

[@Hey.MangoJango]

Can we train tinyllama to do something similar? Since was trying to run AI on Raspberry Pi 5 (with Hailo AI Accelerator)

[@warpdotdev]

Absolutely!

[@Hey.MangoJango]

@@warpdotdev Thanks! ❤️ from India