Improve RAG with This Simple API (code included)

[@farhanafridi8694]

I believe you are the hero every AI engineer needs. Unlike most KZbinrs who copy and paste code from documentation, you address the real problems AI engineers face.

[@daveebbelaar]

Appreciate that!

[@ClarkeBishopConsulting]

Very helpful, Dave! Many companies try to use naive chunking because there are so many examples on the web, KZbin videos, etc. You gave us a very good way to do smarter chunking and get more useful results. This is the future for RAG use cases.

[@daveebbelaar]

Thanks Clarke!

[@BrockMesarich]

So good at simplifying concepts in these tutorials. Loved this Dave!

[@daveebbelaar]

Thanks Brock 🙏🏻

[@GeertBaeke]

Good stuff! We use the exact same technique with markdown-based chunking and extra metadata for the chunks. Works really well!

[@daveebbelaar]

I think this is currently the best approach for RAG.

[@micbab-vg2mu]

great - thank you for sharing:) Please explore the topic more - )

[@skyrayzor3693]

Your video is detailed and very helpful, thank you for these type of techniques.

[@krlospatrick]

Thanks a lot for sharing this knowledge, it's really useful!

[@inflationking1271]

Could you do a GraphRAG tutorial?

[@Divyv520]

Hey Dave , Really Nice Video . I was wondering if I could help you with more High Quality and engaging editing with maintaining a brand colour to your youtube channel which can help you to get more engagement in your videos and Build your Unique Personal Brand . Pls lmk what do you think ?

[@StephanieNguyen-om1ss]

Super helpful. Can you please make a tutorial on how to use AWS Textract too?

[@trendavira5128]

Hi Dave, Thanks for the awesome content, a client come to me for a RAG solution, he have a library of hundreds of thousands of pages (about 60 Giga), simplest rag techniques doesn't seem to work for this case, come up to a solution using hybrid retriever and a reranker using llama-index, the results was good but not perfect, if were you how will you tackle this problem?

[@awakenwithoutcoffee]

we are working on a solution for this that can be white-labeled on release! does your client has an API endpoint or some kind o bucket containing all the files ? it really depends in what formats the data comes. If it its just text than you can use a hybrid-approach with semantic chunking, parent-document retrieval or other meta-data filtering techniques. The main point of importance is to make sure the data is pre-processed and cleaned before being chunked an embedded. Entity extraction is expensive but can be very helpful. A second best option is to extract meta-data. One is used for semantic extraction (entity) and the other for additional filtering. GraphRAG is the best solution, using entities, but it costs a massive amount of resources & development time making it only accessible to enterprise clients (10-50k +).

[@AaronGayah-dr8lu]

Enjoyed this. Thank you.

[@__m__e__]

Thanks I'm a newbie and your videos helped get me started. Can you please also share pdf_ingester?

[@awakenwithoutcoffee]

awesome video but where can we find the "from config.settings import get_settings" ?

[@LaHoraMaker]

Have you tried passing the PDF to Jina Reader API? The Markdown output is quite clean too! (but it's only usable for public documents)

[@chwaleedsial]

Will try this with textaract. For my use case I am just sending a csv ( of an excel ) and its working but I think that is not a systematic, luck proof way. Do you think rag approach will be better, less prone to context, structure related hallucinations ?

[@testadrome]

Does it work with scanned pdf docs?

[@daveebbelaar]

Yes!

[@brandonvelasquez3530]

This seems similar to GraphRAG. What is the difference?

[@sahiljain9376]

GraphRAG is a more powerful solution than this baseline RAG. In GraphRAG, the data is stored in the graph with entities and relationships and also doing community summaries in detail which excels in retrieval flow. For eg: questions like "Did company underperform in Q4 vs Q3?" This kind of question would be difficult to answer using Baseline-RAG which can be answered easily using GraphRAG

[@awakenwithoutcoffee]

@@sahiljain9376 you can enhance RAG with agentic frameworks to allow these questions e.g. an SQL Agent with meta-data filtering. I love graphRAG but its a.) super expensive since entity extraction requires a ton of LLM calls b.) takes allot of time to set-up the graph, c.) has additional challenges to be overcome before it can really be used for non-enterprise.

[@__m__e__]

@@sahiljain9376 I was unaware of GraphRAG, and it looks really interesting thanks. It looks like it's beyond my skill level now, but hopefully MS integrates it into Azure soon