Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step - together.ai

Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step - together.ai
👊 Become a member and get access to GitHub:
/ allaboutai
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
In this video i fine tune a Mistral 7B model with a self generated dataset. We will go step-by-step over on how to create the dataset, convert to JSONL, upload it, start the fine-tuning, download the model, convert the model to .gguf and test it.
GH llama.cpp:
github.com/ggerganov/llama.cpp
Reddit:
www.reddit.com/r/LocalLLaMA/c...
00:00 Local Fine Tune Intro
00:18 Flowchart
01:14 Create Mistral 7B Dataset
05:59 Check Dataset
06:56 Upload Dataset
07:26 Start fine-tune job
10:58 Convert model to gguf
12:39 Testing our fine tuned model
16:09 Conclusion

[@luisguimaraes7348]

You explained in one video what several courses struggle to explain in dozens... Take a lot!

[@GeorgeVuckovich3]

I’ve learned more from your channel than any other on YT. Thanks, Kris!

[@OutstandingLife]

2:32 I remember when 25 examples was more than enough to get started with fine tuning. And now less than a year later, 300+ examples is the minimum. It’s amazing how much I’ve learned from this community and your videos. Thank you 🙏

[@dmy_tro]

25 examples qualifies and few shot prompting, not finetuning.

[@seankernitsman6055]

Thanks, Kris, for teaching and keeping us up-to-date in this crazy AI wave! Practical code tutorials are always helpful and appreciated.

[@silentphil77]

Love your work man, learnt so much being a member!

[@darrylrogue7729]

Absolutely loving the content on this channel! Your insights on using AI for fun projects are both informative and inspiring. Each video feels like a new adventure into the potential of AI, and your clear, engaging explanations make even complex topics easy to understand. Can’t wait to see what you’ll explore next!

[@saahilsachdev]

It's amazing at the way you explain the concepts in a super simple manner. Kudos to you. Keep the banging content coming our way :)

[@STRIKE01]

Thank you! I'm definitely going to try this. It's the first time I see the process somewhat clear!

[@AzureRaven6969]

Doing amazing work, Kris!

[@watcher1326]

Talk about an AI with attitude. Thanks for the walk through. I haven't used fine tuning and could use the additional information.

[@harristengku7153]

You know from the value that you produce in your channel. I might just purchase whatever paid content you have. Truly amazing, think about starting a Skool community man. You're great.

[@openminddream]

Always informative and enthusiastic!

[@ghosting-cm7zf]

your videos never disappoint! Keep it up with AMAZING content.

[@JasonBevel]

Love being a member, Kris! Always overdelivering value. This video was a great overview of fine tuning your own model. Simple enough for me to understand. 😂

[@Co0k1em0n5t3r]

I really new to this but maaaaan I enjoy the hell out of these videos. I look forward to making my own as soon as I can

[@nexuslux]

Thanks for the detailed video.. fine tuning seems worthwhile!

[@carstenli]

Another great video, thanks Kris!

[@vaibhavmishra1100]

You are doing really great and we are learning a lot by you

[@pushpaaji]

Amazing. Will try it out today for finetuning

[@Infkkkkk]

This is the most straight forward fine tuning tutorial on youtube ever!

[@djjeffgold]

Thanks for the great content and repos. Super helpful content.

[@getgonged]

This is awesome! Great job!!

[@blacktyg3r]

Interesting, but the time it took to train on that small dataset is.. a lot. As always, thanks for presenting new stuff :).

[@Kasyx]

Thanks! It was very insightful.

[@weekendmakeit7760]

Great Info!

[@AG-yy3fq]

Thanks for sharing Kris :)

[@ygoralves4664]

Amazing video, Kris!!

[@ForTheEraOfLove]

Hey there! I am thinking of signing up and was wondering, how's the membership going for you? Have you had a chance to implement his code? If so, is it working as expected for you?

[@avgplayer]

Value for the money. Thanks Kris :)

Thanks for everything ❤.

[@johnsummers101]

Leaving a comment for the youtube algorithm. Thanks for showing an A to Z with this!

[@josefsteiner8616]

Awesome Video, thank you for you content.

[@Ryan.Youtube]

Looks good!

[@ShikariNagasa]

This looks awesome actually

[@ejh237]

Woot Woot!! Thanks for the fantastic Video! Oh, and for the pick 5 😉

[@eatingalienbrains]

Things are starting to get more and more local baby! 😁

[@FallenChapter]

locals forever! :D

[@joepearson7928]

I would love to see you explore front end options or adapting modalities with local models.

[@digitalasylum369]

This is so cool! I'm gonna make one of these for each of my AI Vtubers. And together is such a great platform, especially with the free $25 in credits!

[@abs9994]

Great info

[@Daweiro]

Thanks great job :)

[@saeedasgaritehrani]

OMG, I was just surprised multidimensionally XD I mean everything is great man. content is beyond rich and email part and answer was the most hellarios moment of my past 2 weeks.just tnx XDDD

[@DavidChristal]

Thanks!

[@trashchenkov]

Nice technical video! Can you dive into the topic of LLM evaluation? For example we fine-tuned model using approach like you showed. How to check the quality of results?

[@luisguimaraes7348]

super cool

[@jonatan01i]

Hello Sir! You say at 12:40 that there is a video showcasing how to set things up. I failed to find it, can you provide a link please?

[@positivevibe142]

Hmmm, so you call a model "local" while you already shared your info in that .jasonl with the AI Platform using API Key!?!? Am I missing something here!

[@positivevibe142]

@@VioFax Right. It's just kind of negates the the notion of 'locality' and the reason why people/users go with offline/local venues.

[@p1nkfreud]

@@positivevibe142yeah I want local so that when the grid goes down and takes the internet with it, I can still learn to build and repair things and grow seeds and hunt food. With the knowledge from local LLMs. Not sure what you guys are doing that you have to hide it so much? Your AI gf get a little frisky? 😂

[@positivevibe142]

@@p1nkfreud Funny ... lol ... but really this is a very crucial thing! Like for a small business, scientific researcher, medical facility, financial agency, legal firm, laboratory facility, ....etc ..... believe or not the list longer than you can imagine ..... Secret Boyfriends lab research 😁

[@Joooooooooooosh]

​@@p1nkfreudset aside about $10K if you want to train locally

[@len5499]

How else would you fine-tune the model? Some of us don’t have massive compute power that requires a substation to power it. 😅

[@Canna_Science_and_Technology]

Nice, I’m going to do this tomorrow…

[@carthagely122]

Most intersting Channel and my english is Bad but hé make things look very soft thanks alot

[@nenesh66]

Nice video :)

[@MugenMangekiou]

That's so hilarious, please share this model with us, I would like to make some tests

[@DemiGoodUA]

How can I fine tune model on own codebase? Is it possible to make dataset from a code?

[@elidelia2653]

very interesting work! thanks for the lessons. I wonder how much of this programming, if not all of it, can be done autonomously. Probubly an expensive agent task, but curious to push the envolope of AI's capabilities. Fully autonomous assistant populations who live in the internet as we live in societies. Fascinating and inevitable.

[@hqcart1]

can you make a video about sentiment? I have a dataset and i want a model to give a score between 0 to 100 (bad to good) about each statement.. I have no idea where to start.. thank you.

[@junmagic8847]

what are the specs of your pc?

[@greatsarmad5147]

I subscribed to the channel but i don't get why it's not letting me in in the github pls someone explain) thx I also tried signing up on the website but i go the 40 promps and tha't it

[@spencerfunk6697]

Cannot find code for the data set maker. I wish u would have gone over how to build that part a bit more cause I can’t find anything about using lm studio to make datasets

[@cognivorous1681]

The future is local!

[@vivekkarumudi]

I mean the speech was actually pretty cool I wonder how you were able to do it

[@TrevorDBEYDAG]

How much it costed approx. 9h of training in this example? Just for a reference please.

[@khoahocgia7671]

You can build a website for people to hire for data training, where you take a portion of the money. You assist them in training and return trained files in quantized format. We have a high demand for personal assistant training, especially for private document sets for individual purposes. Therefore, I think you should consider this. We're not skilled in coding, don't want to complicate each step, but we have large PDF, DOCX documents that need training, and we need the output to be quantized to the lowest "Q2" level to run on popular devices like iPhone or Android, not large-scale quantum versions. We focus on the popularity of handheld devices, as not everyone owns a server or wants to pay a large amount for data running through APIs. I can offer prices from $20 to $50 for a 100MB document set. I hope you'll consider offering a rental training model soon.😊

[@duckyblender]

This is a great idea - creating this soon. I'll keep you updated :)

[@andreasrosenfelder9094]

Sometimes talk to yourself - grandmother wisdom

[@tr1pod623]

Can you please link the python script to convert dataset?

[@subhansajid1435]

Hy bro where I started in Ai

[@simonstrandgaard5503]

Similar to your text datasets. I had luck training using axolotl, and it took around 1 hour using two A40 instance with around 40gb vram. On vast. For around 1 usd per hour. I have not yet tried Together, but Together seem simpler to configure than axolotl.

[@CypherDND]

I can't wait until everything is completely local. They can't legislate what is on your local hard drive! Hope I'm one of the lucky winners! :D

[@nickcochrane6568]

Anybody else getting 401 errors when uploading the jsonl files? Tried it in powershell, CLI, python, know my API key is set correctly, inference models work perfect.

[@sleyeborgrobot6843]

bro if you get 401 errors i dont even know how to help you. thats so many errors.

[@jojojkk1335]

Which u luck for what u doing 👍👋

[@yashinshekh802]

First you said you will use the reddit as dataset but later you created the dataset using the Mistral API itself. Don't know what I am missing here

[@DannyGerst]

hilarious😂

[@David_Mederake]

The best Channel for AI for me. On Point, not too much and every shinything and i learn better english... Thnaks for all @AllAboutAI 🙏

[@laif9857]

hi , whats the difference between lms studio and oogabooga???

[@DoppsPkin]

Nothing lmstudio is just easier to use

[@webnomad1453]

I followed this tutorial but LM Studio spits out all the fine tuned data even when I just type something simple like "hello".

[@gerykis]

No way that I will upload online to train my data . How can I train locally ?

[@TechTechnology-lp5zp]

Please send the sourcecode

[@maheshBasavaraju]

Hope I win the credits😅

[@subhansajid1435]

To generate a passive income

[@Tax1-kz9tu]

Please give me thos source code 😮

[@danielejiofor3493]

You just ruined a good video by mentioning a Nigerian prince.. like can’t a video be just simple and clear without mentioning a country..