How To Fine-Tune the Alpaca Model For Any Language | ChatGPT Alternative

In this video I will show you how to fine-tune the Alpaca model for any language. And it only costs $3! How did I figure this out? Watch the whole video to understand. I'll show you how to translate the cleaned Alpaca dataset. We will then use the translated dataset to fine-tune the Alpaca model (not the LLaMA model) for our desired language. For this, we will either use DeepL or ChatGPT. Also, I will show you how to evaluate the quality of your fine-tuned model. Last but not least you will learn how you can interact with your fine-tuned model in a UI. As always, if you have any questions, don't hesitate to reach out. Enjoy! :-)
My Medium Article for This Video:
/ 370f63753f94
Medium Article Showing the Evaluation Results:
/ 8e363a0a99ca
GitHub Repository for This Video:
github.com/thisserand/alpaca-...
00:00:00 Intro
00:01:52 Calculating Estimated Costs
00:09:03 Decision Making
00:10:03 Creating A Subset Dataset
00:12:00 Dataset Translation
00:17:36 Fine-Tuning the Alpaca Model
00:26:15 Model Inference
00:27:18 How Much Training Data Do We Need?
00:31:24 Evaluation
00:37:32 Outro
References:
Self-Instruct Repository: github.com/yizhongw/self-inst...
Alpaca Blog Post: crfm.stanford.edu/2023/03/13/...
Alpaca Repository: github.com/tatsu-lab/stanford...
Alpaca-LoRA: github.com/tloen/alpaca-lora
AlpacaDataCleaned Repository: github.com/gururise/AlpacaDat...
GPT-3.5 Documentation: platform.openai.com/docs/mode...
OpenAI API Introduction: platform.openai.com/docs/guid...
DeepL Pricing: www.deepl.com/pro-api
LLaMA Paper: arxiv.org/pdf/2302.13971.pdf
Is ChatGPT A Good Translator? Paper: arxiv.org/pdf/2301.08745v2.pdf
OpenAI Pricing: openai.com/pricing
vast.ai: vast.ai/
Stay in Touch
Medium
/ martin-thissen
LinkedIn
/ mthissen135
KZbin
Of course, feel free to subscribe to my channel! :-)
Of course, financial support is completely voluntary, but I was asked for it:
/ martinthissen
ko-fi.com/martinthissen

[@andrefalcao3015]

Amazing work. Congratulations! Raises a number of very big questions relative to the inner nature of LLMs

[@christopheprotat]

You have a new fan. Thanks a lot for sharing all this with the community. I learned quite a lot.

[@maxziebell4013]

This was a wonderful walkthrough. Great work… and interesting following your train of thought!

[@martin-thissen]

Glad you enjoyed it! :-)

[@LoggeL]

Wow just found the channel. Exactly what I was looking for. Nice cut, clean video and lots of backing infos.

[@superblondmale]

This is exactly what I've been waiting for. Thank you so much!

[@martin-thissen]

Glad I could help! :-)

[@TheOracleOnline]

Wonderful and in-depth walkthrough!!! Thanks a lot. You made my day!😊❤❤❤😊

[@asencionita]

Thanks! This is a great, detailed, no-nonsense tutorial. Best of luck on your next project.

[@martin-thissen]

Thank you so much, really appreciate it! :-)

[@antonberg1131]

Thanks Martin! This was great. Keep it up!

[@coolkaran1234]

Very interesting, good job, and I appreciate you explaining everything and putting it out here you youtube!!

[@OmarTravelAdventures]

Thank you Martin, this is so interesting. I will be following you on this Channel.

[@infocentrousmajac]

Very good article and awesome video. Really appreciate sharing your approach. Keep up the good videos. Subscribed

[@AndyKehOfficial]

Another great video. and good looking out with DeepL!

[@itseric5117]

Thanks for sharing. Really enjoy learning from your videos.

[@lorincpap7395]

Love your style, thanks for the content!

[@zaursamedov8906]

shine like a diamond!

[@talharuzgarakkus7768]

That is amazing. This will change to all knowledge i think .Thank you for video.

[@theresagarcia9081]

New subscriber! Great video! Thanks for sharing!

[@SaifAli96]

Great video as always!

[@MrAcarlo]

Hi, I'm Carlo. Thank you for this video. It will certainly be too difficult for a novice like me to replicate the steps necessary to obtain a model who speaks good Italian. But your video, in addition to being clear and instructive, has an extra magic: your enthusiasm is contagious. On KZbin it's rare to find someone who seems so genuinely passionate about a topic. I wish you all the best

[@martin-thissen]

Wow, that's such a nice and genuine compliment, thank you so much! Really appreciate it! :-)

[@masekin]

Amazing work!

[@saifgaida5872]

Awsome, thanks for sharing!

[@AlexanderBukh]

Tolles Job, much kudos!

[@kaoulkae]

Great explanations and ideas!

[@martin-thissen]

Thank you! :-)

[@ratside9485]

Krasser Typ ! Bin gespannt was da noch kommt. Danke für deine Arbeit. Hoffe das es bald auch ein Lora gibt das ich als normalo auch damit klar komme. Ähnlich wie bei Stable Diffusion Kohya_SS mit grafische Benutzeroberfläche. Weiß nur noch nicht was ich da Trainieren soll 😜.

[@rashping]

great work and many thanks!

[@ladidoschladido5024]

Excellent video!

[@TheReferrer72]

brilliant project, and explanation.

[@martin-thissen]

Many thanks! :-)

[@natw309]

mega useful! very nice 👍

[@1986chrisx]

Haha, I have seen your german finetuning on huggingface already on the weekend and checked your channel if you had some details on it.. happy that there is now a video for it :-)

[@martin-thissen]

Haha yeah recording and editing the video took a little bit longer.

[@1986chrisx]

@@martin-thissen Your videos are great! I am a novice in AI, but have a software engineering background and I can easily follow along. LLMs got me excited again in AI stuff :-).

[@martin-thissen]

Thank you, really appreciate to hear that! :-)

[@kontrakamkam7148]

Thanks for sharing that was great

[@chipcode5538]

Nice work 👍

[@Kireita]

THANK YOU!

[@vincentmanat4946]

Merci beaucoup ! Très actuelle et très pédagogique ! Ich liebe dich

[@faridautomatic]

@vincent si tu fais des expériences de train du modèle Alpaca en français je serais définitivement curieux de savoir si tu as de bons résultats et combien d’instructions tu as utilisé 😊

[@SaschasLWA]

Ganz großes Tennis, vielen Dank!

[@murilocurti1474]

Amazing video! Thank you! You could considere the modelo NLLB for translating too

[@CraftPit]

so useful. thanks!

[@martin-thissen]

Glad it was helpful! :-)

[@LeoAr37]

Vicuña works amazing in Spanish, I don't know if they had spanish prompts in there but I hope they soon reveal the dataset

[@a.david.s]

Hello, this is a good video! 😃

[@fouriertransformationsucks438]

Very nice work, well for the number of tokens you can just download the GPT tokenizer and calculated directly with a line of hugging face API. I am thinking of training something in Dutch (just not something in english) and this is excatly what I am looking for.

[@justindressler5992]

Thanks for the awesome video👍how large were the Lora weights did it have a big impact on vram when loading the final model. Also I just wanted to clarify when training did you need the original LLAMA weights as well as the Alpaca LORA weights. Thanks

[@ethanming7096]

🎉 I was just wondering how to train a new language! Your video just in time! And if I may, could you also make a tutorial on how to add custom dataset to the model using langchain? Preferably for a 🔰 beginner

[@martin-thissen]

Great suggestion! Added it to my list! :-)

[@hasenfuster]

Ich wäre auch interessiert :)

[@TheAiConqueror]

Super Video Martin! Ich hab gesehen dein Kanal ist gewachsen 😬👍 hast du definitiv verdient. Bin gespannt ob irgendwann eine perfekte Übersetzung ins Deutsche möglich ist. 👀 Jetzt wo schon diskutiert wird GPT-5 zu stoppen... wäre geil so was auf dem PC zu haben. Ein universeller Asistent sozusagen 🦾🤖 Weiter so!

[@martin-thissen]

Danke dir! :-) Es ist auf jeden Fall echt verrückt wie schnell die Fortschritte in den letzten Jahren waren und dass solche Modelle nun auch schon auf Consumer Hardware verwendet werden können.

[@TheAiConqueror]

@@martin-thissen Ich überlege mir noch extra eine 4090er zu holen um selber solche Modelle zu trainieren oder einer bisl auszuprobieren. Ich habe gesehen es gibt eine ähnliche Gui wie bei a1111 nur für Text. Hat mal ein anderer KZbinr gezeigt. Ich liebe gut aufgeräumte und benutzerfreundliche Guis. So viel Programiercode wie du drauf hast, überfordert mich einwenig. 😅

[@moseshu6917]

This was an amazing walkthrough. how to prepare the data for chatbot or Q&A texts?the instruction of the data can't be empty, right?

[@astroportterraformationfor2776]

Great exploration. Thank you for your channel. It gives hope on a way to free AI from datacenters. The main trick is to "initialize" the model with "instruction-following data". I wonder if it could be "contexted" by chat or forum channels?

[@martin-thissen]

Yes, I'm pretty sure it can be fine-tuned on chat or forum channels.

[@sebastianramirez5781]

This is really cool and all, I'm not trying to take away anything from that, this is an awesome way to leverage AI to make tools like Llama useful to companies anywhere in the world, my question is though, as a user wouldn't it be better for you to just translate the input to english and the output to german?

[@tyc00n]

your a beast!

[@MrOnePieceRuffy]

I am using DeepL and Google on a daily basis over 3 years with Japanese and sometimes with Chinese and this is my golden Rule: It's a sentence = DeepL clearly wins. As a Dictionary Google Translate always knows better

[@martin-thissen]

That sounds like a really good approach!

[@cangunen2165]

Brilliant content. I wonder if the adapter model is also usable/applicable for the 4-bit quantized version once it'is fine-tuned on the alpaca-7b-hf model like in the video?

[@martin-thissen]

Yes it is. I found this statement in the Alpaca-LoRA library: "These files contain scripts that merge the LoRA weights back into the base model for export to Hugging Face format and to PyTorch state_dicts. They should help users who want to run inference in projects like llama.cpp or alpaca.cpp."

[@user-bu1cq4sz9j]

Hello Martin, First of all your video is excellent. I have a few doubts that I have a 10k domain-specific dataset of question-answering, and for that, I have generated 200 instructions that are getting used in the model. so every single instruction is used 50 times. so does that affect the model training? or we need to have unique instructions without getting any repetitions.

[@tommyshadow66]

hey ive recently found your videos when trying out tortoise tts and want to say you are doing amazing. i dont know if you take requests but if you do can you make a tutorial on using tortoise tts fast on a local computer for longer text? the impression is that the "fast" version can 5x the speed of tortoise if not more

[@martin-thissen]

Perfect timing, will cover it very soon. :-)

[@tommyshadow66]

@@martin-thissen I think this officially makes you my hero. 🥂

[@user-bf6en1em1j]

Great work Martin Thissen,I just had one doubt now we have prompt data so we are able to Finetune by converting one language to another language , but what is the case i have large corpus of organisation data but it is in document format like same wikipedia data ...now i want to convert that data into prompt based how to convert that data?

[@henkhbit5748]

Great work and nicely explained. So i suppose if you want finetune with your own dataset in another language u need first to translate the alpaca instruction dataset in the same target language as your documents. Then u dont need to translate your documents again?

[@martin-thissen]

Yes, from my understanding that should work. :-)

[@adriangabriel3219]

Great Effort! Could you explain how you load a custom finetuned alpaca model? I am getting an adapter_model.bin, adapter_config.json, tokenizer_config.json. Do I have to use this folder as lora weights folder then?

[@Kontor23]

Vielen Dank Martin, bitte bleib weiter am Ball und informiere uns. Wäre cool wenn Alpaca soweit trainiert werden kann, das dieser dann auch Programmiercode ausgibt.

[@martin-thissen]

Oh, ich würde die Qualität nicht mit ChatGPT bzgl. Code-Generierung vergleichen, aber das Alpaca Modell kann schon jetzt Programmiercode generieren.

[@TheAiConqueror]

​@@martin-thissen Wirklich? Und wäre es möglich das Model so weit zu trainieren das es besser programmieren könnte? Das wäre ein cooles tut video 👀😬

[@martin-thissen]

@@TheAiConqueror Ja, ich spiele derzeit tatsächlich mit dem Gedanken, ein Video in die Richtung zu machen. Soweit dass es besser programmieren kann würde eine sehr gute Datenqualität und eine sehr große Datenmenge erfordern. Ich könnte mir aber vorstellen, dass man durch den Alpaca-Ansatz mit geringem Aufwand einen Datensatz generieren kann, der einem hilft, ein Modell zu trainieren, das ähnlich gut funktioniert.

[@TheAiConqueror]

@@martin-thissen Das wäre der Hammer! Ich beobachte deinen Kannal auf jedenfall, auch weiterhin. Und bin gespannt was du noch so raushaust hier. 👍

[@user-zz5ui1id5n]

Hello Martin, thank you for the amazing tutorial. I would like to fine-tune my data, which consists of only 100 prompts. I'm unsure if it is necessary to fine-tune the Alpaca model. Could you please provide guidance on this matter?

[@L4ky13]

Did you leave texts with code untranslated entirely? even the part where it explains the code in text? If yes, did you include the untraslated texts in your final dataset?

[@enchanted_swiftie]

Hello mate, I lovvveedd your tutorial series ❤ __ I have a question, actually I am trying to fine-tune "GPT-J" on my private data for question-answering. So, I have multiple documents, they all in the raw text. So, as the example goes, we will convert them into the huggingface dataset and then train the mode. My doubt is: How should I pass the prompt? I mean, during the training, how should I structure my prompt? Should I just give the raw text as-is? or I should do some prompt engineering like: Context:{} Question:{} Answer:{} to the model? Will you please shed some light on this? Thank you very much!

[@tanhaonan7891]

Very awesome video, so is your fine-tuned lora model running on a cloud server? Because I have never seen you download the original model, just train it on vast, is it possible to customize the UI interface? For example, change the name of alpaca lora to renaissance GPT

[@martin-thissen]

Yes, the model weights are loaded when I started the fine-tuning process. The only thing I exported where the LoRA weights. Yes, you can customize the UI interface inside the generate.py file. At the end of the file you can change the Gradio settings.

[@caillef]

Amazing work! Do you think fine-tuning is a great approach to teach ChatGPT how to code in Lua using a specific API that I made ? I want to generate lua code based on a documentation of a 3D voxel game engine, but I don't really know where to start. I need thousands of examples right?

[@reezlaw]

You can't finetune ChatGPT, but if you want to tune OpenAI models you can, you must choose one of the original GPT3 engines and they'll train them on their own hardware for a price

[@15ky3]

Super video 👍 beschäftige mich gerade mit dem Thema, hab aber so gut wie null Ahnung von Programmierung. Kannst du uns das deutsche alpaca model zur Verfügung stellen?

[@mayatroilo282]

Wow!

[@clray123]

Re the low translation price, I think OpenAI is currently massively subsidizing their models (price dumping so to say). There are some cost estimates out there that suggest that the price, based on computing resources cost alone, should be 50x what they are currently charging. Unless they have some mad optimizations that nobody knows about.

[@martin-thissen]

Oh that's interesting! Would make sense to make it cheap first and win as many customers as possible to then increase the prices once everyone is dependent on it. That's why I think it's important to have alternatives like Alpaca or Open Assistant.

[@matthiasschmitt644]

Hallo Herr Thissen, tolles Video, sehr lehrreich und schön vorgetragen! Wenn Sie eine Inspiration für zukünftige Videos suchen: Wie wäre es mit einem Question-Answer-Chatbot mit dem neuen GPT4All-Model auf PDF-Dokumente in deutscher Sprache? Das würde mich sehr brennend interessieren und andere sicher auch. Beste Grüße, Matthias L.

[@martin-thissen]

Spannendes Thema, habe es definitiv zu meiner Liste hinzugefügt. :-)

[@andrewandreas5795]

thanks for such awesome video! do you think the fine tunning could also be done on a 3090? its price per hour is cheaper and has the same amount of memory, but perhaps it's slower so it will take longer and therefore cost more in the end, what do you think?

[@martin-thissen]

Yes, should also work with a 3090. I think it's cheaper overall because as far as I remember the 3090 is half the price while the 4090 doesn't have twice the FLOP over the 3090.

[@andrewandreas5795]

@@martin-thissen cool. Btw did you already check the Koala model? Waiting for you to create something cool with it!

[@richardrgb6086]

Hello! Can you fine-tuning T5?

[@munishrajora2303]

Sorry for asking. Can we use fine-tune model for commercial use?

[@gustavomonteiro2087]

Have you tried to fine tune a higher parameter Llama like, 13B, 30B, 65B? I wonder how expensive that would be and if the tools you showed would be enough

[@martin-thissen]

I haven't yet but definitely something I'm considering! :-)

[@user-kg6jr3bj9o]

haalo her, I think your videos are very good. I have a rasssperypi 4 with 8 Gbytes and 64 bit linux system. I'm also from germany and would like to try this and install it. How do you do it? thank you

[@theshrubberer]

a great video would be hoe to fine tune it for other data that is not in instruction following format ...just expanding it's knowledge base re domain specific data

[@pkmnjourney]

how would you approach this if we were doing multi language, say spanish and german?

[@humbertozambrano8556]

Amazing, if possible I can do my own GPT model,??

[@user-su1iz6lm4p]

Hey Martin, ich versuche seit 2 Tagen dein Tutorial auch bei mir zum laufen zu bekommen. Ich nutze eine GPU in Azure. Könntest du vielleicht offenlegen welche Versionen der Packages du verwendest? im txt File von tloen ist dies nicht zu finden. Leider treten bei mir immer wieder Fehler auf, da die packages falsche Versionen haben und durch Abhängigkeiten unterhalb der packages wird dies sehr verstrickt!

[@wasifbaloch]

how can I use it for question answering downstream tasks?

[@TheLaPyae]

I wonder would that be work for Burmese Language as well 🤔

[@HaunterButIhadNameGagWtf]

How much VRAM it used for training? I'll want to try some on 12GB 3060.

[@user-wx5vs3zr5l]

Hi, i've opened an issue on your repo. I am unable to get your code running for some reason. Appreciate your help

[@theshrubberer]

did I miss something? did you try interacting with the original alpaca model in German to establish that it was not understanding German already to some degree?

[@emj2234]

Is it possible to use the same technique showed in this video to train GPT4all in other languages like spanish . GPT4all by Nomic is based on Llama

[@martin-thissen]

Yes, should definitely work. They just used even more tasks to fine-tune the GPT4all model.

[@sunkwolf]

this can be done for free whit the new meta IA translator Seamless M4t? if yes could you please make a guide to multy lenguage Llama2 ?

[@ahmedkotb3089]

Can use this tutorial for any language for example Arabic language ?

[@marcusmayer1055]

Супер контент.

[@MeinDeutschkurs]

Sounds strange, but have you ever tried to use Alpaca for translation? Some of my tests of the quality was „ok“.

[@martin-thissen]

Not really tbh, but good to know and thanks for sharing :-)

[@MemesnShet]

Wow I didn't know it was so cheap to rent compute

[@RagdollRocket]

hey danke, wo kommst du eigentl. her? LG, chris!

[@martin-thissen]

Bin vom Niederrhein (NRW), wo kommst du her? :-)

[@kyramauch4542]

Hii Martin, ich würde mich gerne für meine Bachelorarbeit unteranderem intensiver mit den Themen deines Videos beschäftigen. Allerdings stellt sich mir die Frage, ob bereits eine Oberfläche für GPT4All zur Verfügung gestellt wird. Könntest du mir dort weiterhelfen? Ich würde mich sehr über eine Antwort freuen - viele Grüße, Kyra.

[@77netwar]

Could you make a video going through an example from start to finish? For example, expanding on a base model by creating a Lora with specific knowledge that it can only know with the Lora trained (e.g. the contents of a certain book that you feed it) and have it answer questions about the knowledge that you added. A lot of people are searching for this information, especially now that models have come on the market that can be used for commercial use (e.g. MPT-7B). Easy 100.000+ subscribers if you were to make such a video where you for example take MPT-7B or other commercially usable model and show how to make the data set and instructions and feed it, I don't know, scripts on every Simpsons episode and then ask the model to make a list of all the characters featured in episode "Marge Vs The Monorail".

[@1986chrisx]

Hi Martin, is it possible to finetune the model on a custom context/dataset? Like can you train a custom alpaca/LLM to just give you information about a specific manual that you have trained it on? That would be great to create custom assistants for specific tasks

[@martin-thissen]

Yes, that is definitely possible. I plan to cover something similar in a future video :-)

[@J3R3MI6]

@@martin-thissen awesome just subbed for that! Been looking for this!

[@1986chrisx]

@@martin-thissen Wow, looking forward to that! For me that sounds like a total gamechanger, as it allows you to add an "AI" feature to your existing cloud application. I could think of an analytics feature that can be queried in natural language, a chat bot that acts as salesperson for a specific pruduct on a website, a supportbot in case of issues with a software/product, etc..

[@APrettyGoodChannel]

@@martin-thissen Any idea how it handles fiction writing? Even if it's imperfect it could help with blocking out paragraphs which I could then fix up.

[@Vigilence]

How to train so Alpaca doesn’t just complete sentences? Something more like gpt-4?

[@capasi5380]

you look like the vecna actor from stranger things 4

[@martin-thissen]

Had to look it up but I can see the similarity haha

[@capasi5380]

@@martin-thissen yeah 🙂

[@liwaiyip1769]

Is it possible to train the model on cloud like azure?

[@martin-thissen]

Yes, vast.ai is just a cheaper alternative to renting GPU instances from Azure, AWS, Google Cloud,..

[@fontende]

Handsome and smart 😳

[@shailendrarathore445]

HELLO MR. Martin can you make a special video of SO-VITS-SVC-FORK AI using Google colab step by step.. Video uploaded by Nerdy Rodent.. Hope see you soon with the awesome tool..😇😇🥰🥰🤗🤗

[@clray123]

The big problem is that it is all so unpredictable. And when it breaks it is not at all clear what to do to "fix" it.

[@ASAFH12]

what do you think will it work in Hebrew?

[@martin-thissen]

That's a good question. Unfortunately, I think it won't work because the LLaMA model wasn't trained on Hebrew language. But if you try it out, let me know if it did work :-)

[@marcin8432]

DeepL is indeed great tool. Thanks for another informative video 🫡

[@reezlaw]

I think I'll make a dataset out of certain WhatsApp chats, I want a model that will swear and insult me relentlessly in Italian. I found code that translates WA chats to usable datasets (including merging messages that are split over multiple sends, we all have that annoying friend that sends you 16 messages in a row that could have been a sentence).

[@UCs6ktlulE5BEeb3vBBOu6DQ]

I have a 24 threads Ryzen 9 3900x with 64gb DDR4 and Gen4 NVMe. Alpaca is writing a word every 15-40 seconds. I tried the 30b and 7b. How come ?

[@1dgram]

Using your CPU to run the model is going to be very slow, even with a very powerful processor. Use a compatible GPU or do what Martin does and rent one.

[@UCs6ktlulE5BEeb3vBBOu6DQ]

@@1dgram I could use my rtx 3070 ti but as far as I see, these models are built for cpu.

[@1dgram]

@@UCs6ktlulE5BEeb3vBBOu6DQ no, they are designed to be able to run on either and for all but the smallest models will run much faster on a GPU - do you have the right version of PyTorch installed?

[@user-wo4xz6ri4s]

i am too noob to get it up and running in my newly installed ubuntu. 😭 machine learning is moving in lightspeed and i feel like a snail

[@martin-thissen]

Just keep going and don't put too much pressure on it. It's not a skill you learn in one day but I'm sure you will make your way if you stay consistent :-)

[@marcusmayer1055]

Як на рахунок використати гугл колаб, це було безкоштовно

[@gurigraphics]

Another idea: Helsinki-NLP/opus-mt-en-de