Fine Tuning ChatGPT is a Waste of Your Time

Fine-tuning doesn't line up for many problems and teams. Today we discuss why fine-tuning has limitations and why alternative approaches might be better for you despite how major companies are talking about AI. We also glimpse an exciting field of study that is yet to be fully explored!
OpenAI Fine Tuning - platform.openai.com/docs/guid...
AWS re:Invent presentation - • AWS re:Invent 2023 - C...
Generative Agents paper - arxiv.org/pdf/2304.03442.pdf
Give us a follow on Stable Discussion: blog.stablediscussion.com

[@BradleyKieser]

Very good explanation and excellent thinking however the problem is that context Windows or not normally big enough to take all the data. This is why fine tuning is an important part of the mix. The correct usage is a balance between long-term data going into fine tuning and short-term data going into RAG. There will soon be a type of job specifically around the sort of data architecture.

[@iseeyoushining]

Very well pointed.

[@CitizenWarwick]

We had a well crafted GPT4 prompt with many tests covering our desired outputs. We took gpt35 and fine tuned it and now it's performing the same. Worked well for our use case!

[@adrianmoisa2281]

Excellent description of the challenges in fine tuning AI models! You got yourself a new subscriber 🎉

[@user-qh4ze7xq4f]

I wonder how performance of RAGs will vary with integrating generative and retrieval processes. Seems like it would be difficult to optimise, plus more expensive computationally. Definitely the way forward though

[@tomski2671]

Relaying what works and what doesn't is highly valuable. Too few people share their experience. Thank You Training/Fine-tuning is a very delicate process, it has to be done really well to get really good results. Moreover it's not a well understood process - new discoveries are constantly being made, even at the highest levels of research.

[@breadcho7446]

The problem is that finetuning GPT for example is a black box

[@joshmoracha1348]

Nice video dude. What is that app you are using to visualize your message.

[@StableDiscussion]

Thanks! Glad you liked it! Excalidraw is what I use for all the diagrams that help me explain things

[@JoshKaufmanstuff]

Great Video! What is the whiteboard app that you are using?

[@StableDiscussion]

Thanks for watching! It’s Excalidraw.

[@GDPLAYz155]

So, if I am correct, you suggest adding a value to the jason object, which is a chunk of data, sending with other set of data for fine-tuning the gpt model like questions and answers, am I right or does it require a different process?

[@breadcho7446]

This usually is done by having Vector Stores, with encoded data into Embeddings.

[@cyclejournal9459]

Would that be different with the recently introduced custom-gpts which allow you to personalize your model based on your specific instructions and provide it with your own contextual documents for reference?

[@StableDiscussion]

It’s similar, however there are a number of limitations to using custom gpts over using the API and a customized data source. We talk about this briefly here: kzbin.info/www/bejne/iXTIoop8n6d6oKcsi=MfIF0RPBH5tGdOr7 We also have a post on our blog about gpts more specifically: blog.stablediscussion.com/p/are-gpts-a-marketing-gimmick?

[@kingturtle6742]

Can the content for training be collected from ChatGPT-4? For example, after chatting with ChatGPT-4, can the desired content be filtered and integrated into ChatGPT-3.5 for fine-tuning? Is this approach feasible and effective? Are there any considerations to keep in mind?

[@dawoodnaderi]

all you need for fine-tuning is samples of "very" desirable outcome/response. that's it. doesn't matter where you get it from.

[@arthurguiot8897]

Waow it's qualitative! You won another sub :) soon you ll be big, I can see that, continue to work hard

[@aldotanca9430]

Currently I am planning and testing about a project which will rely heavily on RAG and I think I will have to also consider fine-tuning, becasue of the way I need the model to format, reference and present information from multiple documents. Still wrapping my head around how to produce the training data, but at the moment my impression is that, at least in my case study (a specialized and niche knowledge base about music and musical research), even RAG requires quite a bit of work to fragment the documents in ways that guarantee reliable retrieval.

[@StableDiscussion]

Absolutely! We did a video just a little while ago about how custom chunking RAG helps you to improve retrieval: kzbin.info/www/bejne/gnnYiIZ9jtp6aq8 Fine tuning might be what you need to do but it’s an optimization over being a first step. Doesn’t necessarily exclude it from being a valuable piece of the picture though!

[@aldotanca9430]

@@StableDiscussion thanks, I was a bit buried in study and missed your reply. I will chck it out!

[@gopinathl6166]

I would like to get your advice for creating conversational chatbot. Do RAG or Finetune be suitable because we have a CourtLAW based dataset that contains 1000's of PDF which is unstructured dataset of paragraphs?

[@zainulkhan8381]

Hello I am also trying to feed pdf data as an input to openai its unstructured set of data and ai is not able to process it correctly when I ask it to list transactions in pdf that it generated garbage values and not the actual values that are in pdf I am tired of giving prompts so I am looking forward to fine tune now

[@zainulkhan8381]

Did you achieved the results of the operations you were doing on your pdfs

[@yakubtopcuoglu3296]

I m also wondering if you achieved about you were doing on your pdfs? i m writing my thesis and interested on your results

[@alanli8899]

I am very new to this AI field, thank you very much for explaining in simple terms !

[@zalzalahbuttsaab]

5:22 When you started talking about the context window problem, I did think about indexing. I suppose an AI is a sort of an index but it is more dynamic than a traditional database. Setting up a session database would effectively solve the context issue.

[@StableDiscussion]

For an AI it has deep dimensionality to be able to search for language semantics and largely the context size is the issue rather than indexed search. Vector databases are the best at tracking this space and calculating similarity in a number of ways based on queries. But there are definitely ways to leverage traditional databases to provide context as well. Any form of retrieval opens a large space of possibility

[@YuraCCC]

Good explanation. However it looks like these two techniques are not mutually exclusive, e.g. it could still be valuable to finetune a model to improve processing of RAG generations without any specific data, while RAG mechanism supplying all the data for each specific generation

[@StableDiscussion]

Thanks the comment! That’s true and a good point. The most basic example is formatting responses but there could be other opportunities that don’t necessarily look to provide data and instead augment or support the generation. That’s a really interesting space and a topic I’d love to learn more about

[@injeolmi6]

Thank you for making this video. I remember I talked to my friends about a similar concept a few months ago, now I finally know I was not alone! RAG seems like the thing most AI services should have by default.

[@StableDiscussion]

Glad it was helpful! We’re hoping to continue and expand on this thinking in future videos

[@ominoussage]

I'm not an expert in AI topics, but I really do think the only thing we need is an AI that can just understand and it's just RAG on everything else. Great and insightful video!

[@PorkBoy69]

"just understand" is carrying a LOT of weight here

[@JaapvanderVelde]

The problem of 'just understand' is really the problem at the core of AGI. If we solve that, we won't need LLM's (unless they're part of the solution of course :)).

[@MaxA-wd3qo]

why, why so tiny amount of subscribers. Very much needed approach to problems, to tell 'wait a minute... here are the stones on the road"

[@BernardMcCarty]

Thank you. Your clear explanation of RAG was very useful 👍

[@tijldeclerck7772]

Loved this explanation. Subscribed.

[@user-du8hf3he7r]

Training augments behaviour, RAG augments information - they are two different things.

[@keithprice3369]

I'm far from an expert, but I think at least part of the challenge is when people think fine-tuning is for giving the LLM more DATA; increasing it's knowledge base. That's not what fine tuning is for. It's for customizing the WAY it responds. It's more of a style guide than a knowledge store.

[@StableDiscussion]

I think this is largely because of how we see OpenAI and other companies train their models off of data. It’s not a clear separation but I agree, that is the prevailing opinion on where fine tuning fits. If so, I still question how useful fine tuning will be for unexpected prompts and if it gets stuck in the ruts or correctly adapts to the situation it’s presented with

[@rafaeldelrey9239]

There is a general misunderstanding of fine-tuning vs RAG. Fine-tuning is used to teach patterns of question-answers, not to add new data to a model.

[@-Evil-Genius-]

🎯 Key Takeaways for quick navigation: 00:00 🤖 *Understanding Fine Tuning in AI* - Fine tuning is a technique to customize AI models, gaining popularity in the AI community. - Major AI companies like OpenAI and AWS focus on making fine tuning more accessible. - The appeal of fine tuning arises from addressing the limitations of AI models, particularly in handling context and relevant information. 02:35 🧠 *Challenges of Fine Tuning and Overtraining* - Defining training data for fine tuning is challenging due to the difficulty in understanding what the model lacks. - Overtraining poses a significant challenge, making the model rigid and less adaptable to changes. - The need for a representative set of data that mirrors real-world scenarios to avoid overtraining pitfalls. 05:19 🔄 *RAG (Retrieval Augmented Generation) as an Alternative* - Retrieval Augmented Generation offers a more flexible approach by breaking information into smaller, manageable pieces. - Using smaller chunks allows for better management of context window problems in AI. - Updating and modifying information chunks becomes easier compared to the fixed nature of fine-tuned models. 06:51 🔐 *Security Concerns in Fine Tuning and RAG* - Security issues arise in fine tuning as users can extract data about the training process and model's responses. - Retrieval Augmented Generation provides better control over which documents go to specific users, enhancing security. - The ability to control and restrict the knowledge base of AI systems based on user requirements. 08:21 🌐 *Future Possibilities with Retrieval Augmented Generation* - Retrieval Augmented Generation opens up diverse possibilities, such as developing autonomous agents with brain-like patterns. - The potential for AI systems to perceive, plan, reflect, and act based on stored details about their environment. - An exploration of the vast capabilities within the space of Retrieval Augmented Generation compared to the limitations of fine tuning. Made with HARPA AI

[@christinawhisler]

Is it a waste of time for novelist too?

[@nyxthel]

Solid work! Thanks!

[@quick24]

Am I the only one here surprised to find out that Jack Black is an AI expert?

[@MrAhsan99]

thanks for the insight

[@protovici1476]

This video and opinion is fairly incorrect in regards to fine-tuning. Especially, fine-tuning can be utilized in any deep learning hyperparameters (i.e. GenAI, Discriminative AI, BERT, NLP) with any data set. Self supervision, supervised, to reinforcement learning just to name a few use cases of algorithms to solve a problem. KZbins fine-tuning in their algorithm made me stumble upon this video. Highly recommended re-evaluation of this video to save folks from misunderstanding.

[@tecnopadre]

Sorry but why then it's a waste of time? It wasn't clear or finally mention as far as I've listened

[@korbendallasmultipass1524]

I would say you are actually looking for Embeddings. You can set up a database with Embeddings based on our specific data which will be checked for similarities. The matches would then be used to create the context for the completions api. Fine tuning is more to modify the way how it answers. This was my understanding.

[@DJPapzin]

🎯 Key Takeaways for quick navigation: 00:00 🎯 *Fine-tuning Overview* - Fine-tuning is a technique to personalize AI models. - It's data-intensive and currently a popular trend in the AI community. - Major AI companies, including OpenAI, are emphasizing fine-tuning. 01:01 🤔 *Why Fine-Tune?* - Fine-tuning addresses limitations in AI's memory space and context windows. - Challenges arise when context exceeds the model's memory, leading to information loss. - AI enthusiasts and companies advocate fine-tuning for more personalized responses. 02:35 ⚠️ *Challenges of Fine-Tuning* - Defining relevant training data is complex, considering unknowns in the model's knowledge. - Overtraining is a significant challenge, leading to rigid responses and missing diverse solutions. - Difficulty in determining what the model lacks in knowledge and how to supplement it. 05:19 🔄 *RAG (Retrieval Augmented Generation)* - RAG breaks down related data into manageable chunks, overcoming context window issues. - It enables searching for specific chunks relevant to the question, improving answer quality. - RAG allows continuous updates to data chunks, providing flexibility compared to fine-tuning. 06:51 🛡️ *Security Considerations* - Fine-tuning and AI interactions may expose proprietary information and data vulnerabilities. - RAG offers stronger control over which documents are sent to specific users, enhancing security. - The ability to control data distribution to users provides additional security benefits. 08:21 🌐 *Future Possibilities of RAG* - RAG opens up exciting possibilities, such as developing autonomous agents with perception and planning capabilities. - The potential for optimizing RAG for various situations makes it a promising area. - RAG's flexibility and adaptability make it a more compelling option compared to fine-tuning. 09:18 🎙️ *Conclusion and Call to Action* - RAG offers more potential than fine-tuning, especially in terms of data curation and understanding. - A glimpse into the fascinating space of RAG and its diverse applications. - Encouragement to follow Stable Discussion for more insights and discussions on AI. Made with HARPA AI

[@rfilms9310]

"you to curate data to feed AI"