Fine-Tuning ChatGPT 3.5 with Synthetic Data from GPT-4 | VERY Interesting Results (!)

Fine-Tuning ChatGPT 3.5 with Synthetic Data from GPT-4 | VERY Interesting Results (!)
How to Fine-tune a ChatGPT 3.5 Turbo Model with Synthetic Data from GPT-4 - Step-by-step
👊 Become a member:
/ allaboutai
Get a FREE 45+ ChatGPT Prompts PDF here:
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
In this video I fine tune a ChatGPT 3.5 Turbo Model with Synthetic Data generated from GPT-4. A step-by-step guide on how you can use fine-tuning to improve your LLM results.
00:00 ChatGPT 3.5 Fine-Tuning with GPT-4 Intro
00:19 ChatGPT 3.5 + GPT-4 Fine-Tuning Flowchart
01:05 Step 1: Selecting Dataset Types
05:50 Step 2: Generating Datasets with Python
08:16 Step 3: Generating JSON Datasets
12:32 Step 4: Fine-Tuning Our ChatGPT Model
17:48 Price of the Fine-Tuned ChatGPT Model
18:35 ChatGPT 3.5 + GPT-4 Fine-Tuning Model Evaluation
www.allabtai.com/chatgpt-3-5-...

[@DJPapzin]

🎯 Key Takeaways for quick navigation: 00:00 🌐 *Introduction and Workflow Overview* - Introduction to the experiment: Using GPT-4 to create a synthetic dataset and fine-tuning ChatGPT 3.5. - Workflow overview: Steps involve creating a synthetic dataset, fine-tuning the model, and benchmarking. 01:10 🤖 *Choosing Dataset for GPT-4* - Selection of dataset types: Riddles, math problems, and logical problems. - Example problem: A woman in front of Taj Mahal with a reference to a classic video game character. 03:56 🐍 *Python Script for Dataset Generation* - Python script overview: Creating problems using GPT-4 with a loop for automation. - Dataset structure: Assignments, responses, and step-by-step problem-solving. 06:24 📝 *Creating Text File for Problems* - Using Python to create a text file with problems for step-by-step solving. - Task breakdown: Creating a problem, solving it in the next step, and repeating the process. 07:59 🖥️ *Running Python Script for Dataset Generation* - Execution of the Python script to generate synthetic data examples. - Opening and inspecting generated examples. 09:11 💾 *Creating JsonL Format for Fine-Tuning* - Adapting Python script for JsonL format: Structured data needed for fine-tuning. - JsonL structure: Messages, role, system, content, and role-user content. 11:29 ⚙️ *Fine-Tuning GPT-3.5 Turbo Model* - Uploading synthetic dataset for fine-tuning using the created JsonL file. - Observing epochs and training loss during the fine-tuning process. 18:02 💸 *Fine-Tuning Cost and Results* - Total cost breakdown: Cost of creating the dataset and fine-tuning the model. - Reviewing training loss and epochs during fine-tuning. 19:42 🧠 *Benchmarking Models with Example Problems* - Testing a problem on ChatGPT 3.5 without fine-tuning. - Testing the same problem on the fine-tuned model with GPT-4 synthetic data. 21:18 🔄 *Comparison of Model Responses* - Contrasting responses: ChatGPT 3.5's inability to answer vs. Fine-tuned model utilizing step-by-step reasoning. - Implications: GPT-4's impact on improving problem-solving capabilities. 21:57 🎯 *ChatGPT Vanilla's Incorrect Reasoning* - ChatGPT Vanilla provides an incorrect answer to a scenario involving dropping a ball through a cone and sending it in a box. - In ChatGPT Vanilla's response, the ball is mistakenly placed inside the cone inside a box sent to New York, showing a lack of understanding of the physical scenario. 23:21 🔄 *Improvement in Fine-Tuned Model's Reasoning* - The fine-tuned model shows improvement in understanding the scenario involving dropping a ball through a cone. - Unlike ChatGPT Vanilla, the fine-tuned model acknowledges that the ball passes through the cone and highlights the reasoning behind it. 24:57 📊 *Evaluation and Conclusion of Fine-Tuning Results* - The presenter evaluates the fine-tuned model's performance based on the given scenario. - The fine-tuned model, although not entirely certain, demonstrates improvement over ChatGPT Vanilla, indicating potential advancements in reasoning abilities. Made with HARPA AI

[@watcher1326]

The topic did not sound interesting, but Kris tends to be my favorite source of GPT information. I'm glad that I watched it, because the end results were eye opening.

[@AllAboutAI]

Thnx mate :D

[@andresrl-1]

Great tutorial on how to fine tune a model. Something that I didn’t kwon before watching your video ❤

[@jamesjonnes]

Generate a few thousand examples on your personal computer with LLama. Send each to GPT-4 to verify if it's good or not and answer with one word. Exclude the ones it say are bad and use the rest for training. That way you spend half the amount of money for data that's nearly as good.

[@AllAboutAI]

I like it :) def noted

[@jamesjonnes]

@@AllAboutAI Claude is cheaper too. You can generate with one LLM and then train another with the data.

[@alexsov]

look like it is better to split you data to train and validation sets to prevent overfitting. thanks for sharing

[@sadface7457]

CoT, with self-consistency and PHP (progressive hinting) have been shown significantly improved accuracy and reasoning. I would also look into Solo Performance Prompting (SPP), where you have a group of experts answer and then ai combine their answers. Try add the emeritus proffessor and look at all the ivy universities. I would also try MacArthur fellow, Rhodes and Fullfright scholar. Maybe even mensa or tripple 9 society.

[@priyanshugarg6175]

Hi ! Nice video. How would we know it the synthetic data samples created by LLMs (GPT 4 for your case) are similar or not ? Would we need to implement some kind of mechanism or use a metric ? Thanks.

[@theBear89451]

Imagine how smart the trained model would be if you used an MIT professor in your synthetic data creation prompt.

[@AllAboutAI]

haha

[@unrealminigolf4015]

My honest opinion the same as yours 💪🏼

[@amaanshareef5625]

can it be done?

[@unrealminigolf4015]

@@amaanshareef5625 yes easy. From all their published materials. Simply use plugins and prompt away as your chosen MIT professor 👩‍🏫

[@JohnGallie]

imagine GPT 5 has perfect skills like what he is training, that would be awesome.

[@nigeldogg]

Tests using fine tuned translations might be cool 🇳🇴🇺🇸

[@sveindanielsolvenus]

Great stuff! Will GPT-4 generated unique puzzles each time you make it create a new one, even though it doesn't have a history of what puzzles it already have made?

[@sveindanielsolvenus]

@@IOFLOOD Good ideas! It didn't seem like Kris did this in his video, so the risk of getting duplicates must have been there, is what I worried about.

[@priyanshugarg6175]

I was also thinking the same. Would we need to implement some kind of mechanism or use a metric ? Do you have a solution for this problem ?

[@Shahid_An-AI-Engineer]

I am reaching out to seek your guidance on a challenge I am currently facing with my fine-tuned GPT-3.5 model. Despite having fine-tuned the model with 130 examples, I have encountered an issue where the model is generating outputs that seem to be "hallucinating."

[@sklarenbach]

Are you avail for consulting work?

[@alarconfilms1]

Thanks is possible to get the codes used?

[@indikom]

I have documentation of an extensive API in the form of an HTML help (.CHM) file. Is there a possibility to teach GPT-4 this API, so that it helps me write code using this API? How to do it?

[@carlosburgoin2262]

you can use retrieval knowledge, with custom gpt (requires suscription) or assistants playground (api pricing)

[@elmehdiezziar]

😊

[@DeathGripper1337]

would it be cheaper to simply use gpt4 instead of fine tuning 3.5 turbo?

[@AllAboutAI]

Yeah great answer from, thnx :D

[@jackford3069]

Yes, absolutely, but this video is more about showing you what fine-tuning an LLM can do. You can actually just download different pre-made data sets for different use cases from huggingface, as well as different LLMs that you can fine-tune as well (and if for some reason you have the jardware for it, you can make your own LLM from scratch). Keep in mind that fine-tuning is more used for making an LLM behave in a certain way rather than building on its knowledge base.

[@DeathGripper1337]

@@jackford3069 hey buddy thanks a lot for the reply! I understand it and have experimented it! My partner and I are going to build our own LLM from scratch to fit our needs.

[@jackford3069]

@DeathGripper1337 That's awesome! Let me know how that goes.

[@pnddesign]

I think you should start with the end result, then explain how you made it

[@sophisticateddirtbagflyfis3081]

So epoch... is this why in the matrix Epox was the guy who uploaded info...?

[@metanulski]

I got lost around 1 min. What is the Synthetic Data all about? For me, it's not clear what is your input, and what is GPT-4 response? Where ist the "Emily is a 27-year...." coming from? It looks like GPT-4 generated it, but if this is the case, I don't know why GPT-4 just writes a riddle. I guess you gave GPT-4 instructions, but you skipped this part in the vid. Confusing.