Prompt Engineering, RAG, and Fine-tuning: Benefits and When to Use

🎁 Join our Skool community: www.skool.com/entry-point-ai
Explore the difference between Prompt Engineering, Retrieval-augmented Generation (RAG), and Fine-tuning in this detailed overview.
01:14 Prompt Engineering + RAG
02:50 How Retrieval Augmented Generation Works - Step-by-step
06:23 What is fine-tuning?
08:25 Fine-tuning misconceptions debunked
09:53 Fine-tuning strategies
13:25 Putting it all together
13:44 Final review and comparison of techniques
Prompt engineering is a powerful tool to steer a large language model's behavior by providing instructions and examples directly in the prompt. RAG, a type of knowledge retrieval, grounds the model's responses to reality by pulling in dynamic and trusted external data sources. Fine tuning trains the model on your own examples to narrow and customize its outputs.
Each approach has its strengths. Prompt engineering is fast and intuitive, while RAG connects real-time data. Fine-tuning bakes in your style, tone, and formatting. The good news is they can all work together! Use prompt engineering to rapidly prototype, RAG to leverage your knowledge base, and fine-tuning to improve speed, cost, and quality.
In this video, Mark Hennings explains the unique value of each technique and shows how you can combine them for the best of all worlds. See examples of few-shot learning prompts and how they can be upgraded with fine-tuning datasets. Learn the specific fine-tuning strategies for optimizing cost, speed and quality.
There's something for everyone here. Whether you have experience with machine learning or just starting out with AI, this video will expand your understanding and give you ideas for improving your generative AI projects.
Join our next Fine-tuning Masterclass at www.entrypointai.com/masterclass

[@XXCDMXX]

it's really fresh and accurate that u say "LLMs don't store facts, they just store probabilities". BTW, it's a very excellent video with wonderful backstage and straightforward script and presentation.

[@pantoffelslippers]

Probably one of THE BEST large language model videos I’ve seen. 😮

[@steve_wk]

This is a very high quality explanation of the relationships between prompts, RAG and fine tuning. Really well done - thanks for taking the time to make this very clear.

[@abtix]

These are some solid information you are providing in this video, I'm actually surprised that this video has such low amount of views. Just know, for those who are seeking information regarding the things you covered, these are very well presented and explained, and I really appreciate that.

[@JoshVonSchaumburg]

I work in technical presales and delivery management. I probably watched 10 KZbin videos to try to better understand RAG vs. fine tuning (which I'm now calling TAG :)). This was by far the best explanation. I'm sending to all my coworkers!

[@EntryPointAI]

Awesome!! Glad I could help :)

[@deepstum]

Excellent explanation. Best wishes for your professional endeavors. While I rarely comment on KZbin videos, this one deserves all the praise.

[@EntryPointAI]

Thank you very much!

[@altrubalag]

This is simple and super useful. Remember, putting it in simpler terms is not an easy job. Simplicity is the ultimate sophistication! 👏

[@linluxv2275]

Thank you for explaining the differences between Prompt Engineering ,RAG and Fine-tuning.Your explanation is easy to understand, which is very helpful for those who are new to AI. I will continue to follow your updates!

[@lifechamp007]

Finally I understood RAG and fine tuning so clearly - Thank you for awesome video !!

[@BorHouse]

Tomorrow I have a presentation on RAG and you are like an angel to me right now 😅

[@marksaunders8430]

Hands down, the best video I've found for this topic

[@jong-keyongkim565]

The most efficient and informative content on those terms and their respective usage. 👍

[@scottt9382]

I work in this field - he did an exceptional job here. I would have liked to see FLARE in here an an extension of RAG.

[@user-ll6mn8ky8d]

This channel will be super useful in coming days

[@darrin.jahnel]

This is one of the best AI videos I've seen (and I've watched hundreds). Great job!

[@Mel-lp4hz]

Very nice and clear explanation, thanks!

[@Franchise-infoCa]

Very nice explanation. Thanks

[@neiltaggart1]

Very clear & simple - well done!👏

[@Aishakamath]

This was really really helpful! Thanks for the video😊

[@andrewsuttar]

Superb video man! This is exactly the content that fits what I've been aiming to achieve. And I love the RAGTAG acronym 😂👍

[@EntryPointAI]

Awesome!! Thank you 😃

[@robertomeers]

Excelente info. Muy concreta y clara. los ejemplos muy sencillos pero propicios para cada caso. RAGTAG team 👍 Suscrito 🔔

[@funkyflorion]

Good summary, thanks!

[@saraswathisripada]

Very well explained, thank you.

[@past_life_project]

Wowzer ! that was brilliant ! Subscribed !

[@CodersArch]

Thank you for an educational video

[@user-ks9sf5hm8j]

Awesome explanation man, like perfect, clears all confusion regarding LLM ES in 15 mins. All the very best for Entrypoint AI. Keep rocking.

[@EntryPointAI]

Thanks so much!

[@SarabjitMadan]

This was just brilliant. Thanks so much.

[@bernard2735]

Great explanation, thank you.

[@aldotanca9430]

Very clear, thanks!

[@jachymdolezal5103]

Great video thanks!

[@Aspect0529]

Fantastically done!

[@thunken]

Nice explanation! I signed up for the masterclass, but, its 4am my time; might be a challenge

[@tunairaiol]

Perfect video, thank you.

[@windhusky4009]

This is really good!

[@RolandoLopezNieto]

Great video sir

[@Blogservice-Fuerth]

Great Content 🙏🏽

[@coachdennis6139]

Nice. Thanks.

[@martynstrydom9904]

Brilliant bro -- so good actually

[@hernandosierra8759]

Excelente. Gracias.

[@sivavenu5100]

very clear explanation,

[@italoaguiar]

Excellent video!! Congrats and thanks! +1 subscriber ;)

[@bryanbimantaka]

Thank you! This is what I need for my thesis. You explained it in an easy yet precise way. If I may make a request, could you provide a video of designing/building a closed-domain chatbot (using RAG for prompting) and employing a pre-trained LLM like GPT, BERT, or LLAMA? please 😅 Anyway, subscribed!

[@EntryPointAI]

Glad you found it helpful! That sounds like a fun topic, I will try to get to it eventually. It might be a while, though :)

[@brianhauk8136]

Thank you so much for your clear and useful explanation of the relationships between prompts, RAG and fine tuning. How affordable is fully-fledged RAG retrieval for a chatbot, say, for a dental office website using GTP-4? And is it likely to be fast enough going through many steps to provide site visitors with quality information before they give up waiting and leave the site? Maybe we need to first provide them with a quick, useful, partial answer while letting them know that additional information will be available in approximately 15 seconds? I'm concerned visitors will abandon the site and that compute costs may be too high for most business owners. Does anyone have experience with this?

[@EntryPointAI]

It depends on how many steps you need to optimize the RAG workflow. Each step adds latency. If you also need to do a quality-check (eg LLM self-review) before responding with the answer, that means you can't stream tokens in real-time, which makes it feel much slower. However, lots of people are doing this for chatbots and the basic retrieval lookup should be pretty blazing fast since it's essentially just a database query. For a dental office website, I would not try to build RAG from scratch unless it was a nationwide chain because I think you'd need either a dev team or a really good developer and cost could add up fast. Either way, I'd start with the simplest solution, trying to leverage something that has already been built. Fixie has some cool solutions you could look into for off-the-shelf RAG www.fixie.ai/

[@brianhauk8136]

​ @EntryPointAI I didn't know that building RAG from scratch today requires a dev team or really good developer to get specific responses. In your video you say to include required specific information in the prompt so responses stay grounded (in the reality contained in the documents) and true to that. Can fine tuning provide similar, grounded responses, or is our only option to use very long, expensive prompts? And is there any reliable approach to getting these specific, accurate responses when querying hundreds or thousands of documents? My understanding is that ChatGPT-4 allows up to only 20 documents, and that most LLMs aren't good at accurately retrieving and using content in the middle of longer context windows and documents. Also, are other methods of training models doing more than using prompt/completion pairs more effective?

[@EntryPointAI]

Fine-tuning can help prevent the model from answering questions that are out of scope for it's task. It can also teach it how to respond to certain inquiries, in a "customer service" type of tone. It depends on the nature of the task and how much domain-specific knowledge is required if fine-tuning can handle it. 99.9% of all model knowledge is learned during pretraining, so if you can quiz the LLM and get back accurate results without RAG then fine-tuning can be sufficient to customize for your use case or task. The documents get split into chunks and stored in a database, so they are split and logically separated has a big impact on how well your retrieval works. You can also train a custom embeddings model or experiment with different off-the-shelf embeddings models to try to improve the retrieval results. OpenAI just came out with some new ones. I am sure there are other training methods out there but the primary way to train LLMs is with prompt completion pairs. With chat-based models, you can also train on multi-turn chat, but all they are doing under the hood here is inserting special tokens between each chat message and labeling them (ChatML for example is the markup language OpenAI uses).

[@devtest202]

Excellent material, I want to ask you a question, try to train the gpt2 (fine tuning) model for these purposes, do you see it as valid? On the other hand, do you have examples of how you have trained the model? It is very interesting what you say about how with a few examples you can see good results, for example if I want to train him so that he acts like a salesperson and that if he is asked about a certain type of product he always responds with a particular logic. I think it's great in the final part when you say that the great model would be a rag with only variables and the promt is somehow already trained

[@EntryPointAI]

I haven’t tried GPT 2, I would stick to more recent models for best results. For a sales agent that might be a good one to combine with RAG, to where relevant product info is looked up and inserted into the prompt, and then fine-tuning teaches the model how to “sell” it.

[@zebirdman]

Love this video - thank you! Question: When RAG is used you say it shows up in the context window (i.e., where the user types in the prompt). However, my understanding / experience has been that once the knowledge is provided to the LLM via a vector database, the user can just type the prompt/query and nothing else is needed. Can you provide an example of a user's prompt/query when using RAG?

[@EntryPointAI]

The knowledge has to be retrieved for each new user inquiry, and inserted into a prompt alongside it each time.

[@trackerprince6773]

Clear and simple 👍. For gpt3.5 turbo fine-tuning do you always need a user role value or does it make sense to sometimes just provide the system and the assistant value? Curious if all you want to fine tune on is tone and style , doesnt it make sense to provide how the ai should respond without the user input words haveing any weight.

[@EntryPointAI]

I haven’t tried this specifically, but it is possible to pass empty prompts (system and user) and teach the model to learn output style given no information. Usually what I do is have a minimal system prompt (longer for smaller training datasets) and then use GPT-4 to generate user inputs from my assistant completions in the way I would want to ultimately use and prompt the model (generally minimalist inputs or in the form of rough notes for a given output). Note that the model isn’t learning much if anything from your input in terms of style and formatting, its just using it for context for what kind of output to generate. Prompt learning rate is usually 0 or a nominal value like 0.01

[@trackerprince6773]

@@EntryPointAI would def like to test if the style or tone differs. Since my assumption is to some extent the input context should have some effect. But maybe for many use cases having an input(even synthetic generated) may be better. But I was watching another video( kzbin.info/www/bejne/g3vQZWyKdt6ta9kfeature=shared ) where a podcast transcript was used to train for capturing host tone and style. the creator found the style to be more similar to the host when the sentence spoken from the guest was removed in training.

[@VerdonTrigance]

Hey! Nice and concise explanation of different technics. But I didn't get one moment. Whenever I want a big LLM to deal better with my specific task which is answering open question about the content of the book (for example Lord of the Rings) and if I want to fine-tune it, should I always provide a structured data in form of question and answer? In case of book it seems like impossible task to prepare such kind of input data. This is what I want to grab from the model, because I don't know it, right? So question is, can a LLM be fine-tuned just by giving a new content to it (like book chapters one by one)?

[@EntryPointAI]

You can but its not a practical way to teach knowledge. More likely it will learn the writing style but not give it the ability to retrieve specific facts. LLMs learn almost all their knowledge during pretraining.

[@cprashanthreddy]

Good video... 😀

[@dilippalwekar1223]

loved it

[@FernandoOtt]

Awesome content Mark. A Question. I need to create an AI psychologist and I need to store college data, but this college data is kind of a guide of what to speak, and not the content itself. In that case, what is the best approach, RAG or Fine-tuning?

[@EntryPointAI]

If it’s “how” to deliver the content, most likely fine-tuning.

[@FernandoOtt]

Tks Mark. I still question if fine-tuning works for that cause isn't the "how" for like personality but it is for something like "use the technique X or Y" to continue the interaction. Do you think it is still a fine-tuning approach?

[@user-js6ed4io9u]

this is really informative and a well organised presentation. could you pls guide me a material or link to now how to implement RAG-TAG? and also i'm wondering what amount of data should we provide for Fine-tuning in case of RAG-TAG?

[@EntryPointAI]

I don't have a more detailed guide to implement RAGTAG, but I will take this as a request to prepare one 🤙 If you can prepare 24-48 high quality and diverse examples, that's enough to see benefits from fine-tuning. See this paper: arxiv.org/abs/2305.11206 Video on "How much data do you need for fine-tuning?" coming soon!

[@user-js6ed4io9u]

cool 👍thanks alot🙏🙏@@EntryPointAI

[@PopeCap]

I love the idea of TAG, but had a question. I'm not quite sure what you mean by taking all the examples out of the prompt and into a data set. Don't they still land in the prompt? Your graphic at 12:00 implies they're in a DB somewhere, but then isn't this just RAG again?

[@EntryPointAI]

Thanks for your question! The database icons might be misleading there. The intent on that slide is to show a training dataset that is used to fine-tune a model on examples of the task.

[@PopeCap]

Ahh ok - so that goes into a separate model that's separate from an LLM?@@EntryPointAI

[@micbab-vg2mu]

nice

[@lpalbou]

are you using the apple vision to record yourself ?

[@EntryPointAI]

Nope, Canon R6 Mk II

[@lpalbou]

@@EntryPointAI ok, i was asking as it seems there are some heavy post processing.. but could be the results of something else. Cheers

[@trackerprince6773]

Whats the diff between custom gpts & fine-tuning gpt?

[@EntryPointAI]

Custom GPTs do not re-train the underlying model in any way, they use prompt engineering and RAG if you upload documents.

[@seanmeverett]

Such a key point: probabilities, not facts. They’re quantum in nature, not binary.

[@PlayOfLifeOfficial]

What I learned: We are no we’re near AGi

[@gehdochnicht]

Those eyes though 😍

[@tanveeriqbal6680]

Very clear explanation in its simple form, thanks @entrypointai

[@Jas.in-bx3eg]

Uncanny, avatar moderator.

[@FunnyVideos-ni4iu]

very helpful! thanks