RAG vs. Fine Tuning

[@Kk-ed1gr]

Thank you for the clarification, I had this question in mind last week, and I am glad that you have provided the answers I need.

[@IBMTechnology]

Glad it was helpful!

[@yusufersayyem7242]

35 minutes after downloading the clip, I received a notification, perhaps due to the weak internet in my country.... Finally, I would like to thank you sir for this wonderful explanation

[@IBMTechnology]

You're welcome!

[@educationrepublic9273]

Love IBM's short and sharp explainers! Thank you for an excellent video once again :)

[@FauziFayyad]

I have just watched the 1 years ago, then it updated today. Amazingg 🎉

[@MohammadFaizanKhanJ]

This channel continuously producing very good academic videos! Thanks for sharing it free!

[@satwiknag711]

In terms of digital asset allocation which is less resource intensive RAG or Fine tuning? RAG will definitely be easy to market in short time

[@florentromanet5439]

I wanted to scream "WHY NOT BOTH⁉️) until 7:35 😂

[@emont]

your video didn't included EDA, LLM answers based on pre loaded info, a future evolution is LLM answering based on real time information

[@NabiehSalim]

Thanks for the video, please we need more (from you).

[@sandeeppatil5925]

wonderful explanation ... will love to also know the choice of use from TCO or cost POV

[@MukeshKala]

Great explanation ❤

[@ggggdyeye]

sir can you tell me how to make the vectorstore and store it in a specific file to use it every time.

[@sabeensadaf4926]

Thanks a lot, that was a good one to understand both RAG and fine tuning.

[@shrutisingh9801]

can u make a video about reinforcement learning and performance evaliation of llm models?

[@BibekMishra84]

I have a question. Is the LLM retrained on the new information during fine-tuning ?

[@jignareshamwala3401]

Yes, LLM model is retrained for fine-tuning. For efficient fine-tuning check out Parameter-efficient fine-tuning (PEFT). In PEFT a small set of parameters are trained while preserving most of the large pre trained model’s structure, PEFT saves time and computational resources.

[@GreatTaiwan]

@@jignareshamwala3401this feels like ChatGPT answer 😂

[@rajeshkumar-pz3ym]

Would be great how long in terms of cost and materializing both approaches and fine tuning the model specific to industry will incur additional training

[@GrowStackAi]

Innovation becomes effortless when AI is involved 🔥

[@Siapanpeteellis]

What happens to a model when it is fine-tuned? do you use a database for RAG?

[@cloudnativecedric]

Good question! So with fine-tuning, using an approach like PEFT (Parameter-Efficient Fine-Tuning) which only updates a subset of the full model's parameters, we have new model weights and biases, which could then shared, deployed on a server, etc. for model inferencing with AI-enabled applications. For RAG, yes indeed, the most common method is with a vector database and turning your data into embeddings to search for similarity when using the LLM. But, there's other ways of setting up RAG pipelines too :)

[@jasonrhtx]

@@cloudnativecedricWhen would it make sense to first use PEFT, then apply RAG? Do both PEFT and RAG assign/label semantic relationships to the texts of user-added corpora and store these in a graph database?

[@RaushanKumar-qb3de]

Wow... combination is great. Thanks dear! For information

[@youssefsayed4378]

Use Case: if i have a huge online library of books and i need to use llm to answer questions based on these books and research papers i guess we will use RAG but the point is can use it with a really HUGE amount of data (books and PDFs) and what if there multiple answers for the same question but from different resources and each resource has its own opinion which could be in a different direction than the other resource. what will happened?

[@nickbobrowski]

Great use case, Youssef! When you use RAG, it provides the model with multiple snippets of documents from your database. It's important to adjust the chunk size and the number of snippets injected into the context along with the user prompt. Typically, what I do with my clients is start with creating a set of evaluations for the system. These look like example prompts and example outputs. Any change I make to the system - I always run evals to see if the performance improves or gets worse. Once we have evals that measure how close the actual outputs are to the target outputs, we can work backwards and optimize the chunk size and number of snippets provided to the LLM. This way, it will get a balanced selection of relevant documents from your database. In some cases, it requires careful engineering to write proper search queries. Finally, the way the model writes the final response based on the retrieved information can be steered by instructions and fine-tuning. If you're interested in AI Development, feel free to contact me!

[@prathamsinghdhankhar5001]

Can you please share your email ID or instagram ID in order to get in touch with you.

[@thankuchari]

Could the use of tools from external sources be classified as RAG, i did some searching and the answer is not conclusive?

[@cho7official55]

I thought the retriever was on the far right, and llm in the middle of both, was I wrong, partially, is that schematic representation doesn't fathom all of the architecture, I'd like to go deeper on that matter.

[@cloudnativecedric]

There are a lot of variances with the RAG approach that can lead to different architectures, but there's a full video on the IBM Technology channel that dives into RAG as well!

[@naresha2017]

I want to train my deepseekcoder llm on my source code repo. Should I go for Rag or fine tune the llm? Fine tune - do I need to provide code snippets for different queries?

[@bharathYerukola-gt7vt]

Make a vedio on termonolgioes are often used on ai like benchmark and art of the state and etcc ❤

[@shahraanhussain7465]

How would i get to know which model is using RAG in it or Not?

[@bharathYerukola-gt7vt]

Nice vedio and also make a vedio on neural networks in deep like how neiral network is interlinked with deep learning and machine learning and what is actaully neuarl network and architecuture and why architectute is inporatnt fir neural networks and what is neural network actalkuy like a technique or mathematical expression or anything else so make a vedio on all these

[@CalvHobbes]

Thank you, this is very useful. I'm curious about how the volume of data might affect the choice of FT vs RAG. If we tune the model again with the new data, would it become much larger over time? On the other hand, if we use RAG, would the restrictions on context length hold us back (i.e. if we don't want a very expensive model)?

[@steveyy3567]

clear clarificaion, great job!

[@EmaPython]

Great video, thanks. It was useful for me

[@davidrivera2946]

Like Developer i was working and create system with RAG patern and its fine, but have problems something with specefic documents, i mean when you play with tons of documents the RAG system get more complex, and you depend in strength way of prompts, im not yet play with fine-tunning but is something near to do for experiment, nice video thanks

[@joeyjiron06]

Love the video! I’m building an app that empowers users to generate landing pages with a prompt using AI. I’m planning on building many custom components/sections that I want the model to use when generating a response. I want the model to choose the right sections to use and fill out the copy in the components to fit the users prompt. What would be the best way to handle this in the model? Use rag, fine tuning, both, neither, something else?

[@mark-lq4rk]

Thank you for the fascinating presentation. Assume certain conditions are similar, how would the cost of rag and fine-tuning differ?

[@IBMTechnology]

RAG is generally more cost-efficient than fine-tuning because it limits resource costs by leveraging existing data and eliminating the need for extensive training stages.

[@scycer]

True, but over scale I would assume the excessive context needed to be provided at runtime via RAG to answer questions may outweigh the initial cost in fine tuning no? Like it seems fine tuning is an investment choice in training costs while RAG is an ongoing cost of additional tokens. Obviously in relation to context that can be train ahead of time.

[@rfflduck]

Great video!

[@IBMTechnology]

Thanks for the visit

[@GG-uz8us]

I would like to see a real app that is in production with RAG and fine-tuning.

[@ab-fj2iq]

Can open ai support RAG? All i saw it is only FT

[@johannvgeorge8393]

Thank you for this helpful video🙂. Could you please explain the implementation of how we can update the RAG system with the latest information?

[@kalcavaleiro6993]

updating the database? cz RAG using such as vector database that updated regularly using similarity from the prompt to the database content to retrieve and then augmented the prompt to the LLM

[@apraksh]

Wonderful!

[@giovannispillo5176]

Fantastic Technology for value Great Lesson

[@IBMTechnology]

Glad it was helpful!

[@hi5wifi-s567]

Using “Fine Tuning” , then machine ( accounting software) can be a bookkeeper to prepare financial records for …?

[@cloudnativecedric]

Just some ideas from the top of my head for fine-tuning with financial records are preparing financial statements, tax preparation (fine tuning on region-specific tax rules and historical data), expense tracking & categorization, etc.

[@infotainmentunlimitedbyrohit]

Thank you 🙏💛

[@harryli7557]

Large Manguage Model! 2:08

[@andiglazkov4915]

Thanks ☺️

[@IBMTechnology]

You're welcome!

[@cungthinh1040]

I feel like reading an explaination from chatgpt, this video not really explain anything. Or i think i just suck at learning

[@Louic099]

Most impressive is your mirrored writing.

[@atanasmatev9600]

Large Language model is "LMM"?

[@cloudnativecedric]

Whoops! Good catch, sometimes I mess up when speaking and writing at the same time, it should be “LLM”.

[@ElaraArale]

Thank you~!

[@IBMTechnology]

You're welcome!

[@salehmir9205]

this is gold

[@RajeshKumar-sz6ef]

You did not tell about cost difference :)

[@ridwanajibari4443]

so the concept of rag is like you attach file in gpt and asked question based on the attached file. isn’t it?

[@aravindradhakrishna8660]

I seem to have the same understanding too 😂

[@ammarparmr]

RAG Is take the question from the customer, then find the most relevant chunks of documents related to it. Then pass them both along with the creater instruction to an LLM For example a cusimer has a question and the FAQ PDF file is there So both will enter an LLM so the cutomer get an accurate answer

[@user-rw6iw8jg2t]

Yes that's true LLM' are generalistic by default.

[@Criszusep]

Euro 2024 World Championship. Nice... of course the LLM could't give a response 😂

[@umakrishnamarineni3520]

The RAG isn't updated with new tournament 😂😅

[@RuiMiguelDaSilvaPinto]

😂

[@memehub2002]

cool

[@suibinzhang6782]

Am I seeing the younger version of the other dude?

[@Amelia-o6h3q]

ty

[@Robert-zc8hr]

Obviously you need both, duh? No, seriously, they are not mutually exclusive. Fine tuning is learning, RAG is gathering requirements for a specific project. An expert needs to do both, he needs to learn in order to specialize, and he needs to be able to gather information for the specific task at hand.

[@einjim]

So, You are all told to wear your watch on your right hand right?!

[@SandraGarcia-t1k]

Garcia Michelle Thompson Scott Martinez Ronald

[@byankko]

wtf

[@anshumanpatel9212]

Did he just write “LMM”, instead of “LMM”?

[@scotthill4104]

Just the opposite, he wrote "LMM" instead of "LMM".

[@JamilaJibril-e8h]

Uhhh okay i see you .....😂😂😂

[@SandraGarcia-t1k]

Garcia Kimberly Lopez Karen Hall Mark

[@Datasciencewithsheddy]

You’ll always have to use a combination of both RAG and FT.

[@scycer]

Kinda, we will always have to choose the ideal model for the use case (off the shelf or finer tuned) and what context is provided to the model (rag and other data). Really, it's all about context, whether it's engrained in the model or added as part of the prompt.

[@Zaid-st6wn]

LMM lol

[@SandraGarcia-t1k]

White Deborah Wilson Susan Garcia Cynthia