Axolotl is a AI FineTuning Magician

[@daniels9420]

Awesome buddy! Definitely would like a video on training data

[@rundeks]

I highly recommend learning UV. It greatly helped me clean up my python environments and makes them much easier to reproduce and work with.

[@Alex-os5co]

Thanks Matt! Another awesome video. Yesss please to dedicate data prep video!!

[@Csepowertrip123]

somewhere is read that distilling a model is best vs finetuning, can you make a video examining the validity of this claim

[@adriangoris2116]

8 H100s in a homelab?!? Do you have a homelab tour video?

[@nickolasyounker]

Yea, its the room youre looking at in the video. I've been there... thats the lab. It is more likely he is using cloud based h100s which you can spin up on any Public Cloud Provider.

[@technovangelist]

that’s exactly it. If I could afford to buy that many of those GPUs....well, i can't even imagine get that past my wife...

[@atlantideresearch]

Thanks !! Please more on fine tuning

[@andrei-xe7nu]

Which is better, finetuning first and then distillation?

[@nickolasyounker]

Conda for the win - isolate your environments and tests folks!

[@wardehaj]

Great instruction video. Thanks! Can you male a video about how you used the 8 h100 for using the finetuning and the costs of using that for finetuning?

[@NicolasEmbleton]

More fine tuning content please 😍. These are awesome.

[@sammcj2000]

I love what Python has done for software development, I hate the ecosystem (mainly its terrible version and package management) and errors.

[@DriftlessCryptoToo]

Bravo! Keep the tech coming! Fine tuning, RAG, ETC

[@thegtlab]

Please make a video on the dataset creation part! That would be amazing! I feel like that’s the hardest part of all of this. Hopefully the dataset is something domain specific instead of teaching it how to respond.

[@technovangelist]

Tell me what you had in mind

[@RyanSmith-rb1ch]

Could you do a video on use cases for fine-tuning?

[@ahmadzobairsurosh5832]

please teach how to create the dataset from a bunch of docs automagically. maybe use agents ?

[@nadoiz]

About the next video dataset. Can you focus, a part at least, on different types of dataset, the use case of it and the difference between Llama3, Mistral, Phi types of datasets? I understand that Llama3 has a system message but Mistral no. Maybe I'm wrong but that's why I need that video. Thanks!

[@gr8tbigtreehugger]

Axolotl is a Dune deep cut (and also a cute amphibian)

[@agenticmark]

only in Mexico :D unfortunately.

[@mshonle]

What do I need to know if I want to use WSL? (I have a Windows machine with Intel + Nvidia 4090. Dual booting is not an option.)

[@technovangelist]

I showed what’s needed for Linux which is what you just said you have

[@mshonle]

I was thinking about how the drivers would work though?

[@technovangelist]

I dont have access to a windows machine so i dont know. But i know a lot of folks used to use ollama on wsl before the team released a window version

[@rajcasio]

Can someone tell me what the practical use cases of such fine tuning are? I believe we can get similar results using a system prompt.

[@technovangelist]

if your use case can be solved with a system prompt, its not a good case for fine tuning.

[@FunnyTitleTakes]

Thank you for the video, I was in the stream and had a question but didn't want to ask to not deviate you from the topic but you talked about PyTorch, since I am struggeling with something the last couple of days and used PyTorch a lot without any success, I wanted to ask you this: I have a mini PC win 11 where I installed Ollama + docker + OpenwebUI and it is working fine but the issue is that it is only using my CPU, while i want it to use my GPU which is a "Intel Arc" Graphics, is it possible to run Ollama on Intel Arc iGPU?

[@technovangelist]

I know they were working on Arc GPU's but not sure what the status is on that, especially if its an integrated unit rather than an actual card

[@FunnyTitleTakes]

@@technovangelist thank you for the answer but I saw some github repository where people have done it but no matter my effort, I can't have it done... and wanted to see if you know a way or agree with one of this github repositories to be the right way to do it...

[@technovangelist]

Sorry I don’t know. I just have Mac computers at my house and haven’t really done much with anything else in a while and none of the hosted solutions seem to have Intel

[@FunnyTitleTakes]

@@technovangelist thank you

[@Mister-Wabbit]

Can you tech us how to use an ai model in mac that uses NPU on my mac and GPU please

[@technovangelist]

Ollama will use the gpu automatically

[@Mister-Wabbit]

@ what can i do with the NPU?

[@hcrrby2996]

Mic sound is still off. Are you testing new settings?

[@technovangelist]

there is still a minor issue that most won't notice, but getting there.

[@MuratBayhan]

Great brother