LightRAG: A More Efficient Solution than GraphRAG for RAG Systems?

[@kai_s1985]

Thanks so much! You are the best channel when it comes to RAG. Please keep informing us about the latest advancements in this field. I have played with GraphRag. It can be expensive if you have tons of data, but considering how cheap GPT-4o and GTP-4o-mini have become, the price is not the biggest concern at least for my use case. I processes more than a thousand page document with GPT-4o and it cost me cents. The biggest problem with MS GraphRag is the inference latency. It is not very practical if you want to build a chatbot based on this. Also, it is less customizable in my recent experience. Hope LightRag is better in terms of accuracy, customizability and inference speed.

[@johnkintree763]

It looks like the pipeline is modular, and could work with different vector and graph databases. In the future, users could rate the responses, so highly rated responses could be stored and retrieved when similar queries are made by other users. Retrieving highly rated responses could improve overall system performance compared with generating responses for every query. Apply fast thinking instead of slow thinking when possible.

[@KevinKreger]

It is modular. For example, if you want to replace Textract with MinerU for doc/pdf parsing it would be very easy.

[@CAPSLOCK_USER]

such a great channel, thanks for this guide, i was just about to implement a knowledge base!

[@kenchang3456]

Thanks for this. I was looking for a way to add to the KG without having to rebuild it. Crossing my fingers that this is it. And cheaper too 🙂

[@Imasoulinseoul]

Thank you heaps for the diagrams and explanations!

[@srhino14]

Thanks!

[@engineerprompt]

thank you!!!

[@AndrewNanton]

Pretty cool - if you do more with this, I'd love to see some experiements combining this with late chunking

[@engineerprompt]

good idea, when I get time, I want to implement a system that combines all these different approaches together.

[@hiranga]

@@engineerprompt Has anyone recreated this in LangGraph or LangChain JS ?

[@MrAhsan99]

@@engineerprompt looking forward!

[@marktahu2932]

I am assuming that eventually all publications will come with a link to their embeddings pre-prepared on some database somewhere associated with the publishers.

[@FunDumb]

Definitely want to learn more about lite rag. Cost was a hindrance with regular rag. 😊

[@amdenis]

I started using a recursive variant of this for a bit, which evolved to a multi-LLM approach due to the need to optimize cost-performance efficiencies, but still leverage external inference time optimizations and multi-step sequencing and solving. I think most of these RAG and TAG mechanisms (light, long, standard GR, and the various fine and related tuning methods) will all continue to be superseded at an accelerating rate. The biggest problems I see in the industry from startups to universities and research groups is that the choices made and implementations used are often too brittle and subject to rip and replace requirements to be anywhere near cost-performance optimal in the long term, which for AI means even 1-2 years. So, better design patterns, tooling and implementation architectures are needed.

[@gunu3187]

Thanks for sharing , however this is another smart config changes resulting in low prompt tokens , Microsoft GraphRag uses higher overlap window ( 300:100 ) whereas here it is using (1200:100) which itself reduces number of prompt tokens used significantly. we should deep dive to understand if researchers have done something different to ensure much lower overlap window

[@greglinklater6331]

Thoughts about using late-chunking with Light RAG? Honestly I barely understand what's going on at a high level but conceptually is there anything obvious preventing late chunking from being applied in LightRAG?

[@trytry6569]

Please please please make a video in which we can use it with our local models.

[@engineerprompt]

on it :)

[@trytry6569]

@@engineerprompt Thanks man🙌

[@johnkintree763]

​@@engineerpromptThe Zamba2 family of models looks interesting for running locally with lower latency and more tokens/sec output.

[@KevinKreger]

On the 'mixed' dataset, I don't think it had enough information on the various topics to create a comprehensive graph. I'm wondering if GraphRAG digs deeper or does some other technique with the graph meta-data like BM25.

[@ahmadzaimhilmi]

It costs me $0.02 per 10 page pdf document using openai api. I think it's pretty decent. Future improvements should also include storing all these json files in a database. Also, have you figured out how to get the reference chunks used to generate the response to the query?

[@penguinmagehello]

Mind sharing how/ which packages you use to split pdfs especially those with images?

[@ahmadzaimhilmi]

@@penguinmagehello google marker pdf to markdown

[@KevinKreger]

@@penguinmagehello No images. The example on github is using textract for pdf text extraction, but you can use any other approach and save the images. You could add them to the graph by sending them to a VLM for a descriptive summary. That would go to the entity/relationship resolution step. Your chunking would have to respect the figure description (See figure 1, which blah blah) and the figure title.

[@yvescourtois]

Great stuff, well done! I don't see any reason to use GraphRAG any more after that. I guess the technology will continue to surprise us, but the cost argument is powerful when you handle tons of data

[@dawid_dahl]

Thanks for the really great video, by the way.

[@abhishekbose1081]

Can I use this with Hugging face inference api or Vertexai api ? Actually I have access to this api only.

[@yigidovic]

Use LiteLLM to standardize them to OpenAI API then configure BASE_URL and you are free!

[@derekwang5982]

I only processed 1 chunks for 2 hours, not 2 chunks being processed...not sure why

[@emirishak7609]

Using LightRAG with gguf?

[@remusomega]

Graphs solve the problem of chunk embeddings being de-contextualized. Late Chunking solves this problem. I think we need to re-consider the use cases for GraphRags.

[@muhammadshafiqsafian6149]

how can i view the algo flowchart? a bit complex to understand

[@WaylonLu]

how does integrate the vector dataabse?

[@mclachan]

Have you tested it on structured data?

[@axelwehmeyer9599]

cool. Is there an implementation for Groq-API like gpt_4o_mini_complete-API in LightRAG? How can i use a GUI-Chatbot for LightRAG, e.g. chainlit/streamlit/...? thx

[@engineerprompt]

I think there are a couple of PRs for other models. Not sure about the GUI

[@guscastilloa]

How do you record your screen so that it follows your cursos? It’s super useful to follow along specially when reading papers! Kudos on this amazing expose of this methodology!

[@dylanmoraes990]

It's a video editing app called screen studio on mac

[@ai_dart]

Good Info

[@syedsaifullahtarique]

How to create LightRAG object inside dicken folder

[@ashgtd]

great video thank you

[@AIWhale3]

Is Lightrag supposed to be used with a vector database, a graph database or both?

[@KevinKreger]

Both (nano vector database) along with the graph (which I believe is Neo4J by default)

[@VaibhavPatil-rx7pc]

Awesome 🎉

[@RickySupriyadi]

wow so many RAG system this year already

[@engineerprompt]

Yup, its hard to track but nice to see the different ideas that are coming up.

[@mirkoappel]

Thank you!

[@ingenierofelipeurreg]

Which of all RAG is cost effective and quality better?

[@engineerprompt]

Its a hard question to answer. It will really depend on the task at hands. If you think its just search/looking up information, may be a standard RAG. If there are relationships between objects/entities then may be a knowledge graph.

[@dawid_dahl]

In a few years I suspect we will laugh at all these hacky RAG implementations as one will simply be able to dump everything into the context window and there will be some native mechanism to handle efficiency. What do you think?

[@adegboyegaajenifuja1274]

Or you just grant access to your document repositories and you're good to go

Needle in haystack problem of big context Windows....

[@BryantAvila]

To dump 10K documents each composed of at least 100 pages seems it will always be unrealistic. RAG of some sort will still be needed.

[@dawid_dahl]

@ Will be interesting to see how this comment ages over the coming years. (Or mine!)

[@thunkin-ai]

the native mechanism might probably be a graphrag implementation

[@viiskb]

For the same book, graphRAG uses 1.1 million input tokens and 200k output tokens.

[@Jvo_Rien]

thank you :)

[@themax2go]

so essentially they implemented triples / triplets (sciphi/triplex) 🤔

[@engineerprompt]

pretty much :)

[@JoseAntonio-sn6sf]

nice video, I am just starting with RAGs, so sorry if my question is a little stupid, but if you spend 80k tokens running LightRAG, why the necessity to even implement a RAG when the book it self has 40k tokens? I mean wouldn't be easier to send the whole book to chatgpt?

[@ravigurram9884]

Imagine large systems with terabytes of data.

[@vap0rtranz]

> "I mean wouldn't be easier to send the whole book to chatgpt?" because this stuff is for knowledge graphs, not simple doc Q&A. The graphs create relationships *between* content. We used to do this in KBs by manually (and slowly) tagging and hyperinking stuff that we thought was related. Now models can create the relationships for us without having to hire 1,000s of librarians. The possibility to have a system trained to create meaningful relationships between content and dynamically pull that content on demand is mind blowing. Pre-trained models are already coming out.

[@AhmedMagdy-ly3ng]

Hey 👋 I'm one of the most exciting fans of you and I have the opportunity to come to India I wish I can see you and have a conversation.. Please 🥺🙏

[@akshatgandhi7958]

Thanks

[@ahmadzaimhilmi]

I like everything about lightrag. I wish we could see the reference chunk/document for the returned query too. Also, if there's a better way to query from a sql/nosql db instead of querying from drive.