Retrieval Augmented Generation with OpenAI/GPT and Chroma

Рет қаралды 16,992

10 ай бұрын

In this video, we'll learn how to use Retrieval Augmented Generation with Chroma and LangChain to provide an OpenAI/GPT LLM prompt with more data to effectively answer our questions about the Wimbledon 2023 tennis tournament.
#llm #largelanguagemodels #openai #chatgpt #dataengineering #chromadb
Code - github.com/mneedham/LearnData...

Пікірлер: 64

@kmz8160 7 ай бұрын

Thanks for this step by step guide!

@frankdearr2772 4 ай бұрын

Great topic, thanks 👍

@abhishekvtatachar2541 10 ай бұрын

The content is truly exceptional!

@learndatawithmark 9 ай бұрын

Thank you!

@shibak4 10 ай бұрын

Nice one, Mark! Thank you.

@learndatawithmark 10 ай бұрын

Thanks for watching :)

@shivamroy1775 10 ай бұрын

Great video. Thanks !

@learndatawithmark 10 ай бұрын

Glad you liked it!

@SuperMacmac74 9 ай бұрын

Thank you, awesome vid

@learndatawithmark 9 ай бұрын

Thanks for the kind words 😌

@rmehdi5871 9 ай бұрын

Great video. Mark, I have one technical question, if you do not mind: when I create a store with my data and embeddigs with Chrome.from_documents (I work with PDFs), my notebook cell is running for ~16 min, but fails with OperationalError: database is locked, pointing to sqlite_pool.py:31 in Connection.commit(self): self._conn.commit(). Do you have any idea why and how to fix it? My sqlite3 version is 3.41. Thanks!

@learndatawithmark 9 ай бұрын

I dunno off the top of my head, but that function call is hiding a lot of work. We can probably narrow down what's going on. First question - how many documents are you sending in? And can you reproduce it with some code that you can send me to debug? You can email it to me - my email's on the About page - www.youtube.com/@learndatawithmark/about

@shubhamguptachannel3853 8 ай бұрын

Great brother❤

@PallaviPadav 10 ай бұрын

Very cognitive thank you Mark

@learndatawithmark 9 ай бұрын

Glad it was useful!

@patriciubogatu7663 24 күн бұрын

Good day. I was wondering whether this technique will be enough to make GPT-4 LLM solve arduino circuit projections on the breadboard, for the most basic problems. I want the LLM to be able to pinpoint the exact hole locations on the breadboard, where the pins/wires should be placed. I tried by just explaining but it still commits the trivial mistake. I seeking to make it avoiding mistake by remembering the correct answers for past problems. Is RAG enough or I need to fine-tune, or create new LLM? Regards.

@da-bb2up 10 ай бұрын

thxfor the cool video, mark :) is it also possible to use this method with much data in the database (whole books)? and is there any possibility (like parallelization and vm power) to make the search time shorter for the output at the end? Do you have any suggestions for me maybe? :)

@learndatawithmark 10 ай бұрын

Yeh I mean if you have the raw text you could certainly chop it up and store it. I haven't really played around with the search time when storing more documents but my sense is that the searching is fast, it's the text generation that takes a while!

@avidlearner8117 7 ай бұрын

I appreciate all these videos, but I feel they're very high level. Per example, no words on metadata, which is crucial to proper scoping and filtering, since the LLM can use them for a better search when it creates the query sent to the DB.

@learndatawithmark 7 ай бұрын

Thanks for the feedback and you are right.Most of the examples are (kinda intentionally) super simple so that it's easier to see how all the pieces glue together at a high level. In this particular one there's only one document so I don't think filtering on metadata would be that useful. But you've given me an idea for another video where we ask the same questions but pass in metadata to filter which documents get returned to the LLM. Let me know if you have any other ideas/feedback on the other videos!

@avidlearner8117 7 ай бұрын

@@learndatawithmarkabsolutely, it would differenciate yourself from all the other channels. Also, no one explains the mechanisms behind RAG; the LLM actually generate the query that is sent to the database and only is aware of the returned content and it's not aware of the rest of the content.

@BAkedStakes 9 ай бұрын

enjoy your videos very informative

@learndatawithmark 9 ай бұрын

Thank you!

@tropical-doggo 8 ай бұрын

Thanks for the video Mark. How do you type the code so smoothly without typos? It looks cool!

@learndatawithmark 7 ай бұрын

I don't type! With a lot of help from ChatGPT, I created a Node script that drives Jupyter in the browser - github.com/mneedham/drive-pinot/blob/main/examples/jupyter-openai-vector-search.mjs I have another one that I use for the terminal - github.com/mneedham/iterm-simulator

@nrusimha11 7 ай бұрын

Thank you for an informative video. From what I understood, the llm acts as a focused research assistant here. Is there a way to make it integrate the new knowledge with its general training knowledge while answering questions?

@learndatawithmark 7 ай бұрын

Yeh - that's what we'd use the database (e.g. Chroma) for. I only stored one article in there to make it easier to explain, but we can load a lot more data and even add to it as new data comes in. There are also tools like Metaphor that let us wire up the LLM to web search so that we can get the absolutely latest general data - metaphor.systems/

@briancleary6751 2 ай бұрын

Your videos are so good. I'm glad I ran across the DuckDB one, but I've followed the rabbit hole down. KZbin ended up recommending a video one Ollama + AnythingLLM which seems to be some sort of agent building tool. But I don't understand LLM agents really and you don't seem to have any videos on them. Can you fix that?

@learndatawithmark 2 ай бұрын

Hey - thanks for the kind words. I haven't looked into agents yet, but yes I should do that. do you have any thoughts on what would be a good example for them?

@briancleary6751 2 ай бұрын

The toy examples I've seen are like falling back to websearch if the document store doesn't have good results. There appear to be many libraries/toolkits, but that only adds to me confusion, especially when it all seems like a thin python wrapper around prompting/parsing.

@dexterlee1199 5 күн бұрын

Thanks for great sharing ! Quick qiuesiton my embedding parts seems not work not sure why can you guide a bit ? thanks

@dexterlee1199 5 күн бұрын

I'm using Auzre OpenAi and put my paramter in .env shall i do anything else?

@learndatawithmark 2 күн бұрын

What error do you see?

@manikant1990 10 ай бұрын

Excellent video and editing to explain things in the most efficient manner!! May I know how long it takes to make a video and post-production of a video, would be helpful for me!

@learndatawithmark 10 ай бұрын

I do it in a few parts and I've never timed it from end to end! Part 1 is coming up with the idea of what I want to explain and constructing a notebook or equivalent to best demonstrate it. Part 2 is creating and then presenting a script based on that video. Part 3 is editing those two things together. I spend the most time on part 1, trying to come up with a small enough topic and then thinking of the best way to explain it. That could maybe be a day if I haven't decided what to do but obviously much less if I've already identified the topic. Part 2 is usually pretty quick. Maybe it takes an hour or a bit longer. And then part 3 takes me a few hours depending on how much I tweak things.

@manikant1990 10 ай бұрын

@@learndatawithmark thank you very much for the detailed explanation. It’s very helpful. I will keep trying from my side and let you know how it goes just as a co-content creator. Thank you. Have a nice day again.

@qualitymusic485 10 ай бұрын

Do you happen to know by now why it did not know about the Russian players when asked the first time? Also, is there a parameter to adjust how many documents(text splits) it uses when retrieving the answer? What would you need to change if you want it to give more thorough or longer answer? The prompt? Awesome video btw

@learndatawithmark 10 ай бұрын

I don't know why it didn't know about the Russian players! It seems to pull in the same documents and yet one time it answers correctly and one time it doesn't. Maybe someone else knows why it doesn't work the first time? For adjusting the number of documents, you can pass parameters to the 'as_retriever' function e.g. store.as_retriever(search_kwargs={"k": 1}) You can also play around with the threshold above which it will return documents, which would help to exclude irrelevant documents. See python.langchain.com/docs/modules/data_connection/retrievers/vectorstore To get a longer/more through or even differently formatted answer we would need to update the prompt as you pointed out.

@richardpinter9218 6 ай бұрын

RagFusion might help to solve that issue @@learndatawithmark

@GenericHelper 5 ай бұрын

Such an uderrated channel! Other channels would've made this video 45 mins long! Subd!

@learndatawithmark 5 ай бұрын

hah, thanks. That should be my tag line :D

@GenericHelper 5 ай бұрын

@@learndatawithmark 💯

@nitinmuteja 8 ай бұрын

link to your notebook?

@sundayaliu8562 8 ай бұрын

I can’t find any mention of openAI key to use gpt-4, or am I missing something

@learndatawithmark 8 ай бұрын

I don't think I included that but yes you need to set it. See help.openai.com/en/articles/5112595-best-practices-for-api-key-safety

@nrapendrasingh6964 8 ай бұрын

Dear Mark, can we use here any Hugging face LLM, instead of GPT-4 ?

@learndatawithmark 8 ай бұрын

Yes! My current favourite way of doing that is using Ollama. You would need to replace the place where we initialise the LLM with something from this docs page - python.langchain.com/docs/integrations/llms/ollama And take a look here for how to install Ollama itself - ollama.ai/ I think that will be enough to help you get it working, but if you're curious about Ollama, I have done a few videos about it. Start here: kzbin.info/www/bejne/hHfKdpqnqsZgnMk

@nrapendrasingh6964 8 ай бұрын

@@learndatawithmark Thanks a lot

@alod83 10 ай бұрын

Thanks for this excellent tutorial! Could you share the code please?

@learndatawithmark 10 ай бұрын

Thanks and yes of course, sorry that I didn't already - it's here github.com/mneedham/LearnDataWithMark/tree/main/retrieval-augmented-generation I will write it up into a blog hopefully as well!

@alod83 10 ай бұрын

Thank you very much for your fast answer! @@learndatawithmark