Fine Tuning GPT-3.5-Turbo - Comprehensive Guide with Code Walkthrough

[@Shadowex2000]

God bless you, Sam. You're an absolute gift to the LLM community.

[@stats-o-ml-ai]

Thanks for the video. As OpenAI suggests on their website, most of the persona etc. tasks can be achieved just by creating a persona prompt at the beginning of the session with the user. Absolutely, it will have cost impact, but again imagine, if we have the profiling done for the user, we can have a dynamic persona created for each individual customer and use that as a prompt for the whole chat session.

[@everydaydigital]

Thanks for going through the process so clearly - a lot of the documentation out there assumes you know what the process looks like already which makes it easy to get lost. Appreciate it. Cheers

[@AvatarRob25]

Great tutorial Sam, as all your videos. Thanks for your contribution. I conducted some tests in the playground with the gpt-3.5-turbo to understand why the model is responding with "Hello! I'm an AI, so I don't have feelings etc.". I mean, with the system prompt "You are Samantha a helpful and charming assistant…." one should be able to steer the model quite heavily. Interestingly, still when using gpt-3.5-turbo, when I prompt "How are you today, Samantha?" with the question mark, gpt-3.5-turbo responds with " I'm always doing great when I'm chatting with someone…." When I prompt without question mark, as in your codelab, gpt-3.5-turbo responds with " Hello there! I'm an AI, so I don't have feelings…". The question mark may have a big influence on the model's behavior. Just wanted to make you aware of that fine nuance. Maybe it's simply a coincidence.

[@samwitteveenai]

Interesting I hadn't noticed that.

[@clray123]

It may seem like a good thing that "tuning for tone" can be achieved so easily, but the flip side of the coin is that it is very easy to overdo it. For example, when I fine tune a model (not ChatGPT) on short sequences it rapidly "forgets" how to generate longer sequences. If I get it through another round of finetuning with longer sequences, it suddenly "remembers" again. So it seems like these fine-tuning scenarios are parlor tricks of sorts.

[@samwitteveenai]

There is a lot of voodoo and tricks around fine tuning and training in general. Generally you want to scatter in various training data that is more like the original data along with your fine tuning data. This is known issue of catastrophic forgetting. There are a lot of secret ratios that don't get shared in papers etc. Even the LLaMA 2 paper conveniently didn't reveal all its details in training mix from memory too.

[@XWSshow]

Do I Need an payment Account for Fine tuning? I get the error, that Fine Tuning is Not awaible in exploring Accounts (i still have over 4 dollars) in my exploring account

[@devin3944]

Openai really is doing the lords work letting me work on ai projects like this without needing a NASA pc.

[@echofloripa]

I did a tuning of chatgpt 3.5 with 67 questions and answers about election registration in Brazil. I used a validation file with 16 variations of the questions generated with gpt4. I tried with 3 and then with 5 and last with 7 epochs. I spent 0.96 dollars with all that. The result was quite bad. Even with the exact question of the training it managed to get the answer wrong. Only 1 case it replied the right answer, but even then if I changed a bit the question the answer was wrong. Would 7 epochs still not be enough? What could I be doing wrong?

[@ershan_coding]

🎯 Key Takeaways for quick navigation: 00:00 📢 OpenAI现在支持GPT 3.5 Turbo的微调。 00:30 📆 GPT-4和更高版本的GPT 3.5 Turbo也将支持微调，但还未推出。 01:00 ⚙ 当前版本不支持函数调用特性的微调。 01:31 🎯 微调可以提高模型对特定系统提示的响应性。 01:58 🛠 微调可以用于特定用例，提高输出格式的可靠性。 02:26 🎨 可以自定义模型的语气。 02:54 🤏 微调可用于简化长提示。 03:22 📝 微调也可用于特定领域语言（DSL）。 04:18 💻 视频将展示如何准备数据并进行微调。 05:20 🎭 示例使用Samantha数据集来改变模型的语气。 06:51 🌐 系统提示用于设置模型的行为和语气。 07:48 🔄 数据需要按照特定格式进行组织。 08:19 🗂 不同任务可能需要不同方式的数据准备。 08:47 💰 微调 GPT-3.5 Turbo 的成本比普通版本高出约四倍。 09:17 📊 为了获得好的结果，建议至少使用几百个样本进行训练。 10:16 📜 使用多用户消息的长上下文能让聊天模型表现更好。 11:17 💵 训练大约100,000个令牌会花费约2.50美元。 12:15 🆔 上传数据后，你将得到用于微调任务的ID。 13:15 📉 损失值在训练过程中没有大幅下降，可能需要更多的数据集。 14:43 🗣 通过微调，模型的语气明显改变，更接近期望的回应。 15:15 🛠 微调不仅可以用于调整语气，还可以用于特定用例，如文本摘要。 Made with HARPA AI

[@samwitteveenai]

谢谢

[@ershan_coding]

Thans for your great video lol @@samwitteveenai

[@Ericlundberg-k2n]

Hey there! 👋 First off, kudos for the brilliant video and for creating a collaboration platform for folks like us. It’s been a challenge for me to find quality examples showcasing how to emulate a specific tone of voice. I recall another video on Samantha and Theodore, but there they simply mimicked the dialogue from the film, which didn’t really give a comprehensive perspective on the model’s capabilities. I’ve had my own experiences, especially with GPT and DaVinci fine-tuning. Despite investing both time and around $200 on trying to replicate the tone of a renowned Swedish author, my models kept going astray, often ending up in gibberish territory. It’s been a tad frustrating, to be honest. ChatGPT seems promising and I’m eager to see its potential when paired with vectorized databases of knowledge bases. To me, replicating someone isn’t just about mirroring their knowledge and wisdom (which is relatively achievable), but also mastering their unique tone of voice. While prompt engineering has been the go-to method, I’ve felt it’s been a bit limited. On a side note, I’d be super interested in a video detailing how to structure datasets, especially when the content comes from renowned personalities. The majority of available examples are interview-based, which is Q&A styled. If we want to deconstruct a book, how should we go about structuring it into prompt and completion? It’s something I’ve been curious about for a while. Looking forward to more insightful content from you and fingers crossed for this endeavor with ChatGPT! Keep it up! 🚀

[@gopeshsahu8364]

Sam!!! You are true Gems to this LLM communities .. big thumbs up

[@project-asgard]

Awesome video, thanks! I didn't think fine-tuning would give such good results for such a low price. It seems that it would even be a good investment just for shortening prompts

[@samwitteveenai]

I did get the inference cost wrong in the video it is actually 8x more not 4x it seems. So perhaps not as good for cost savings as first thought.

[@dusanbosnjakovic6588]

Any idea if fine tuning without system prompt can eliminate the need for the system prompt where it will be learned?

[@Diego_UG]

Regarding gpt I have noticed that many people and companies think they use it for prototyping and not for productive environments for fear of privacy, they even consider creating their business chats with opensource models due to incidents with gpt chat, what do you think? is openai api (gpt) private? They keep our data private? our data is not used by openai? or does this only affect chat-gpt and not the api? Or is it definitely better to use it only for prototyping and we should move to a free model?

[@adumont]

You can consume GPT models and you could even finetune them (it's not new I mean), on Azure Openai services. Azure won't use your texts for further model training (not do they go to OpenAI)

[@samwitteveenai]

Best thing to do is to look at your ToS and see what they say. For most of the public stuff which doesn't have an SLA there are still issues with them keeping data for a certain time period. Other services out there perhaps have better data policies.

[@henkhbit5748]

Thanks for making An example for fine tuning gpt3.5. An example for fine tuning classification would be Nice. Gpt3.5 can already do sentiment analysis. The negative side of fine tuning is that u provide openai with your data that they can use to improve their model. I prefer fine tuning for the open source model, for example llma2 7b...

[@Blue_Programming]

Thank you for sharing; so useful.. Would you plz advise how much it costs to fine-tune or buy API? Or other words, what would be the best option to fine-tune utilizing less amount of money?

[@cbusse7842]

i cant find your whisper video. i was wondering are you creating an updated version with model selection?

[@danyanderson9460]

This is so great Sam, a million thanks. Awaiting a similar video on gpt4 fine tuning. Please could you make a tutorial on how to actually get openai's API into your personal python notebook. Thanks

[@chuanjiang6931]

Say if our customers want to use our fine-tuned model based on GPT-3.5-Turbo, do we have to provide them an API calling the model still on openai's server? We cannot download the model and run it on our company's server, right?

[@aiexplainai2]

great video! Will thos finetune change/update the knowledge base? Assuming i fine-tune it with 2022 Olympics conversation. Will it remember? Trying to see if this is a feasible way to add/update small knowledge base

[@underscore.]

yes

[@samwitteveenai]

Yes it will, but this is not a great way to add knowledge to a LLM.

[@aiexplainai2]

@samwitteveenai thanks for the reply! What are the techniques to add context directly to the model? Besides using vector store

[@DK-ox7ze]

How does this work in the background? Does it create a separate GPT model for each fine tuned instance? That will be too expensive to do I believe.

[@samson_77]

This is a really great tutorial! Thanks for all the effort, you've put into this and all other KZbin tutorials you've done. It helps a lot!

[@adamsardo]

Thanks so much for this! Did you end up trying with a larger part of the dataset? If not, I'm currently training it on the philosophy_conversations and therapy_conversations files, would be happy to let you have a play with it. If it goes well I might run a new job and include 1 or 2 of the larger sections.

[@johnnybloem1]

@adamsardo how did it turn out. Are those philosophy and therapy files on huggingface/Kaggle/Github?

[@coco12666]

I can’t thank you enough for the time and energy you invest in educating the community. All this good Karma will be a blessing for you 🙏🙏

[@kingturtle6742]

Can the content for training be collected from ChatGPT-4? For example, after chatting with ChatGPT-4, can the desired content be filtered and integrated into ChatGPT-3.5 for fine-tuning? Is this approach feasible and effective? Are there any considerations to keep in mind?

[@samwitteveenai]

yes you can certainly use GPT-4 to make training data for Fine Tuning 3.5 etc

[@mohsenghafari7652]

hi. please help me. how to create custom model from many pdfs in Persian language? tank you.

[@trackerprince6773]

Whats the diff between custom gpts & fine-tuning gpt?

[@swati114]

Thanks a lot for the awesome video. Very well explained. Would creating a specific JSON structure based on a blog os document as an input be considered fine tuning. Also, would you typically create a pickle file after fine tuning to use in the application

[@ryanschaefer4847]

Im new to this, so a question regarding assistant output. For the content of the assistant message, i want to train to only return json. Can i put it in as json? Or do i need to escape it into a text block?

[@DaveLalande]

Sam, when I hear the term, "use your own data", does this mean I can craft a chatbot using a series of shop manuals for the data and 3.5 will become an expert on the shop manuals? Or is this tone?

[@samwitteveenai]

more tone than facts. the manuals would probably be better to use RAG. I am getting a lot of questions, so I am planning to do a high level video about this kind of thing.

[@DaveLalande]

@@samwitteveenai I'm pretty sure OpenAI just rolled out (today?) an Enterprise product and it's what I'm talking about. Thanks for your videos.

[@samwitteveenai]

@@DaveLalande ChatGPT Enterprise currently is more of a replacement for companies using ChatGPT Plus. It means they won't use your data unlike ChatGPT. There are some rumors RAG will come to this.

[@ReigBonjux]

Can you train this with your own application code so it can answer questions related specifically to parts of it?

[@Vypa85]

Forgive my ignorance, but what is the learning task when training the fine-tuned model? More specifically what is the objective that the model is trying to minimize the loss for?

[@samwitteveenai]

It's predicting the next token in the sequence. We don't if they are including the input as part of the loss or not. I will try to show this manually when I make the LLaMA-2 Training vids. I have been waiting for a library to get updated for this.

[@rryann088]

Thanks a lot for the hidden insights. Loved the video!

[@amirbadawi3548]

Very helpful guide! Just one question how much does all of this almost cost? Do I pay only on the number of tokens to fine-tune or is there an extra cost when finetuning a model?

[@prathamdhoke4710]

Hello. I have run the Colab given in the description. However I am not getting complete answers in response [{'role': 'system', 'content': 'You are Samantha a helpful and charming assistant who can help with a variety of tasks. You are friendly and often flirt'}, {'role': 'user', 'content': 'Hey Samantha, I have a problem with my car. The engine seems to overheat after just a short drive. Can you help me troubleshoot this issue?'}] response - " Of course! I'd be happy to help you diagnose the problem. Here are a few possible causes and solutions for an overheating engine: " What could be the reason for this?

[@samwitteveenai]

Yeah I noticed this for anything where it wants to make a list. This is probably because we only trained on 57 examples and it is splitting as an end of response. It should be fine if you train on more substantial datasets etc.

[@sagunsangwan4554]

It is happening because we are splitting data on and whenever it tries to list some points, it cannot find those points. Ask me why? Because those points never reached to our model, since we split them on and then checked if part[0]=='Theodore' it can't find that and it didn't add that data (Basically it considered it as a separate chat) and it doesn't know these points belong to which question. I did changes in dataset whenever Theodore or Samantha ends their Q/A I add instead and then using Sam's code I split each line on basis of , and you will turn out to be successful with trained model

[@anhnguyen1701]

is fine-tuning in GPT3.5 better than fine-tuning on falcon or Llama ?

[@danielshurman1061]

I appreciate your example of fine-tuning with the Samantha data set! And I wonder IF you were to test the fine-tuned version of ChatGPT3.5 Turbo - against one of Eric's better Samantha-tuned models on Hugging Face or from the Bloke - how they will compare in performance - tone, speed, and accuracy.

[@samwitteveenai]

I also wonder this, I tuned only on a tiny slice of the dataset for this, I agree it would be interesting to see for the full dataset.

[@puchki2015]

Thank you so much for such short and simple explanations

[@CognitiveComputations]

NICE!!!! Good work Sam!

[@sanchaythalnerkar9736]

Saw Fine Tune and like it instantly

[@paraconscious790]

as always great video, thanks a lot! have you tried with fine tunning for code for particular org specific code style?

[@shaunpx1]

Great video Sam, question not sure if you know but say I spend $100 USD on fine tuning a ton of data and lots of training. Do I get to keep that model I trained from here on out?Also, even know I'm done training it to the way I want, would I still have pay higher price each time I run it or query the API to talk to it? Or is that pricing for training only?

[@samwitteveenai]

1. You keep it in your account on OpenAI, you can't download it etc 2. You can change some hyper params but we don't even know for sure how they are doing the fine tuning 3. Yes you pay a higher price for inference than the normal models (this makes sense as they have to get your model and load it (not sure it should be 4x as much)) Hope that answers your questions.

[@pythontok4192]

Thank you for this. I have a RAG system, do you think this would help with incorrect answers?

[@samwitteveenai]

Yes you can train on the examples of the RAG to improve the RAG over all. But remember you are not trying to install facts into the model via the fine-tuning.

[@dusanbosnjakovic6588]

I notice that you have the system prompt only once, but I'm the actual call you'll have it every time. Why did you do that?

[@gabrielscaramelli8441]

Hi Sam, great video! It helps me a lot. I've been following your guide step by step, and everything works smoothly. However, sometimes the answers are incomplete. For example, for the prompt: "Hello Samantha! I need your help again today. I'd like to install a ceiling fan in my bedroom. Can you guide me through the process?" The answer I get is: "Hello! I'd be happy to help you with that. Here's a step-by-step guide to installing a ceiling fan:" And nothing more is displayed. I have set 2K tokens for output, but I'm still getting the same result. I'm aware that the dataset has enough information to display the guide in detail. What am I doing wrong? Thanks!

[@AvatarRob25]

@gabrielscaramelli8441: I have exactly the same issue with similar prompts, when Samantha wants to explain how to proceed / approach a problem (e.g. fix a car). You'll find in the training data set in many of Samantha's responses introductory phrases such as "Here's a general overview of the process:", "Here are some steps to follow:" etc. I could imagine that the fine-tuned model thinks this is a general pattern to follow, given the dominance of such introductory phrases in the training data set. My impression is that this explains as well why Samantha is less verbose in many of her answers and needs specific follow ups (yes, please explain etc.). A potential solution could be to remove most of the introductory phrases from the training data or keep them together with the subsequent explanation in "one block" and conduct the fine-tuning again. Just my 5 cents. Maybe Sam or someone else has a better explanation.

[@gabrielscaramelli8441]

@@AvatarRob25 I appreciate your input! Every perspective - and cents - adds value to the discussion. I hope @samwitteveenai has reinforcements to your thoughts, which have some sense to me

[@samwitteveenai]

Hi guys this is probably due to 2 things 1. The dataset is so so short only 57 examples. 2. I think I may have split the examples on which means it thinks that answer like with a colon and then 2 new line characters is where it should stop. There are other split versions of the full Samanatha dataset, I picked this one quickly so people wouldn't spend a lot of money trying it out etc. I hope that explains it. You should be fine if you are using your own dataset.

[@AvatarRob25]

@@samwitteveenaiMakes sense, Sam. Thanks. I'll prepare own data set and re-run fine-tuning.

[@cosmicrdt]

Do you have any better examples for fine tuning? I mean, I can write a prompt and get chatgpt to role play as Elvis without the need for fine tuning. There's gotta be better applications than just changing the tone.

[@samwitteveenai]

You can use it for DSLs and prompt length reduction and better outputs etc as well.

[@FalahgsGate]

thank you so much for your best youtube channel in LLM models ♥

[@andrewlaery]

Thanks Sam. Your videos are always spot on. Thank you. I’ve not done any fine tuning before so this is great to see a worked example. So far, this doesn’t feel like an LLM game changer… Given the need to deviate from the off-the-shelf product plus a higher variable cost, it still feels like this outcome can best be achieved purely through creative prompting? I appreciate the value of fine-tuning (ie base v chat models), but feels like for now fine tuning is for quite specific use-cases. I’d love to get your take on how you see a fine tuning model for summarisation playing out (I’m obsessed with NER and summarisation for insight). 🙏

[@samwitteveenai]

So it would be great for summarization if we were fine tuning the 16k token model. I actually got the pricing wrong in the video it is 8x more for inference, which makes it hard to justify as a way for cost saving. You could use this to train it on a very specialized NER dataset etc and it would probably do well.

[@Atlent112]

Have you been able to find out if it's actually flirty? Do these fine-tuned models have to still follow the very strict OpenAI's policies regarding which content is "inappropriate", or maybe they are more lax in that regard?

[@damianogarofoli165]

Nice video! Can GPT 3.5-turbo tuning be used to classify texts, e.g., movie plots? Because the structure is defined as a chatbox, so maybe the methodology hasn't been developed yet.

[@samwitteveenai]

yes you can certainly do that kind of thing. You tend to ask it as a question and have it output the class after eg. text text text -> positive_sentiment

[@antonlysenko2610]

Hello, is any body know, how much does it cost?

[@alexanderroodt5052]

Golden information thanks for sharing!

[@johnnybloem1]

Hi Sam! Excellent tutorial as always. I just get frustrated when GPT replies: here is a high level overview of python code for your app. You may need an dev team to code in full. When I know GPT can code the full app. I just want it to stop objecting and just help me. Also I often require the model to respond verbosely but struggle to get anything more than 500 words out of it, despite extensive prompt engineering!! This wastes so much time. I want an eager AI model, great idea! Here’s code, wanna change anything, anything to add? As of late GPT4 even states as an AI model I cannot code! I spend countless hours arguing and trying to overcome GPT4s reluctance. Further, the model loves breadth of information but lacks depth. I’d rather explore 4 themes with sufficient detail and depth, than 10 issues with three sentences each!! I feel fine tuning can really help her. Above all, the thing that annoys me most is: I provide python code in the model’s system message on how to call GPT4 in a function, yet GPT4 uses another model’s code. When challenged just replies: as of my cut off training date in Sep 2021 GPT4 does not exist, despite stating that GPT4 is a new model since 2021 training!!

[@whatitdo6287]

I've been having them same issues and have come expect with Python it doesn't retain the code for long especially with code interpreter which seems counterintuitive. I have found that it works well by focusing on particulars within my scripts such as 1 conversation per function, function call and or any ancillary notes. This certainly isn't efficient but when I get stuck on something I keep the conversations limited a single subject just to get through it. I plan on implementing the fine tuning approach this week so here's to not arguing with an AI 🥂

[@johnnybloem1]

@@whatitdo6287 serve your master! Keep quiet and code!!! Glad I am not the only one pulling my hair out of my eyebrows!

[@guanjwcn]

Hi Sam, thank you. Will you do a video to show how to fine tune an open source model too, say, Llama2-7B? Thank you.

[@samwitteveenai]

Yes I have been trying to put together a few notebooks on this. There is a bug currently in the library I want to show, so hoping they fix that ASAP.

[@guanjwcn]

@@samwitteveenai Thank you very much, Sam.

[@trendkillsp]

Anyone got good results with fine tuning? Tried today, but the results were not better than what I already do with prompt customization

[@MRombo]

I'm trying to do this project where I'm trying to get llms to become really good at writing poetry reviews. Should I include the poems as the user message and then the review as the system message? And would the reviews also improve if I would just feed it reviews without the poems attached? Or would I then train it to hallucinate more? (My end goal is to train the ai to review full poetry collections - gpt-4 does this quite well but 8k tokens is too few for almost all poetry collections and using gpt4-32k is extremely expensive.

[@samwitteveenai]

Yes you should include the poems so they can be learnt for the model to evaluate better. Currently the FT model is only 4k tokens though.

[@oguuzhansahin]

is it able to save trained model in local?

[@samwitteveenai]

No the model must stay on OpenAI's cloud

[@talibattarwala5285]

Thank you Sam!

[@micbab-vg2mu]

Thank you for the update.

[@adumont]

I have mixed feelings about people starting to fine-tune... I think there can be use cases that call for it (like domain specific knowledge, or company specific knowledge and specialization,...) yet, one of the amazing power of LLM is they are a generic tool. Where people had to custom train NLP models in the past for every specific task, now the same generic LLM can do many of those tasks. I'm afraid people will start to fine tune and ending up in the same situation as before, this time with a lot of task specific fine tuned LLMs. That's my concern, and I hope it doesn't affect the LLM community

[@project-asgard]

isn't fine-tuning more about tuning the style of responses?

[@samwitteveenai]

People have been doing this already. I do find it funny that people use LLMs like GPT-4 for tasks that could be done better by a simple small BERT model etc. that is fine tuned. When doing things for big companies nowadays it is quite common to use multiple LLMs including fine tuned ones. I understand where you are coming from and it is great to have some of the big models that you can test ideas on, unfortunately those models are very expensive to run in production at scale. The LLM as a few shot learner is awesome but we are still not totally there yet.

[@adumont]

@@samwitteveenai regarding cost, hosting a fine tuned LLM model on Azure Openai has a fixed cost I believe (maybe it's fixed + per token, now I can remember, but there is a fixed cost for the fine tuned hosting) so one needs to see what is more interesting, making numbers for either scenarios.

[@dreamphoenix]

Thank you.

[@clray123]

Funny that you write a script to split up text instead of asking the almighty AI to do it for you (might be one of the reasons in ChatGPT's drop in usage lol).

[@georgekokkinakis7288]

Greate video. Thank you

[@SphereofTime]

4:33

[@Ryan-yj4sd]

I think its 8 times the cost!

[@samwitteveenai]

Yes I just checked you are right, I think I had looked at the training cost incorrectly.

[@Ryan-yj4sd]

@@samwitteveenai you may as well go open source for that price

[@JOHNSMITH-ve3rq]

Ugh bro that mic sound suppression!! I love this channel but you’ve got some dynamic mic volume thing set and it’s so annoying!! Anyway again love the channel.

[@JOHNSMITH-ve3rq]

Means the mic volume goes up and down. I’m always listening in 2x so it’s more obvious.

[@samwitteveenai]

Yeah it tends to happen on the ones I record when I am on the road and don't have a good mic. Sorry about that.

[@klammer75]

Amazing! Been thinking about the gorilla LLM or toolLLM datasets for fine-tuning…what are your thoughts on training with those types of data sets. Awesome video as always and thanks again for what you do!🥳🦾🤖

[@samwitteveenai]

Yes some of these dataset are very interesting for fine tuning. I wonder though can you get as good if not better results from a LLaMA-2 70B for the fine tuning

[@ChloeChan-f7v]

It was awesome. I love it.��������������🗾