LightRAG: A More Efficient Solution than GraphRAG for RAG Systems?

[@kai_s1985]

Thanks so much! You are the best channel when it comes to RAG. Please keep informing us about the latest advancements in this field. I have played with GraphRag. It can be expensive if you have tons of data, but considering how cheap GPT-4o and GTP-4o-mini have become, the price is not the biggest concern at least for my use case. I processes more than a thousand page document with GPT-4o and it cost me cents. The biggest problem with MS GraphRag is the inference latency. It is not very practical if you want to build a chatbot based on this. Also, it is less customizable in my recent experience. Hope LightRag is better in terms of accuracy, customizability and inference speed.

[@AndrewNanton]

Pretty cool - if you do more with this, I'd love to see some experiements combining this with late chunking

[@engineerprompt]

good idea, when I get time, I want to implement a system that combines all these different approaches together.

[@hiranga]

@@engineerprompt Has anyone recreated this in LangGraph or LangChain JS ?

[@johnkintree763]

It looks like the pipeline is modular, and could work with different vector and graph databases. In the future, users could rate the responses, so highly rated responses could be stored and retrieved when similar queries are made by other users. Retrieving highly rated responses could improve overall system performance compared with generating responses for every query. Apply fast thinking instead of slow thinking when possible.

[@yvescourtois]

Great stuff, well done! I don't see any reason to use GraphRAG any more after that. I guess the technology will continue to surprise us, but the cost argument is powerful when you handle tons of data

[@CAPSLOCK_USER]

such a great channel, thanks for this guide, i was just about to implement a knowledge base!

[@dawid_dahl]

In a few years I suspect we will laugh at all these hacky RAG implementations as one will simply be able to dump everything into the context window and there will be some native mechanism to handle efficiency. What do you think?

[@adegboyegaajenifuja1274]

Or you just grant access to your document repositories and you're good to go

Needle in haystack problem of big context Windows....

[@BryantAvila]

To dump 10K documents each composed of at least 100 pages seems it will always be unrealistic. RAG of some sort will still be needed.

[@dawid_dahl]

@@BryantAvila Will be interesting to see how this comment ages over the comment years. (Or mine!)

[@JoseAntonio-sn6sf]

nice video, I am just starting with RAGs, so sorry if my question is a little stupid, but if you spend 80k tokens running LightRAG, why the necessity to even implement a RAG when the book it self has 40k tokens? I mean wouldn't be easier to send the whole book to chatgpt?

[@ahmadzaimhilmi]

It costs me $0.02 per 10 page pdf document using openai api. I think it's pretty decent. Future improvements should also include storing all these json files in a database. Also, have you figured out how to get the reference chunks used to generate the response to the query?

[@penguinmagehello]

Mind sharing how/ which packages you use to split pdfs especially those with images?

[@ahmadzaimhilmi]

@@penguinmagehello google marker pdf to markdown

[@trytry6569]

Please please please make a video in which we can use it with our local models.

[@engineerprompt]

on it :)

[@trytry6569]

@@engineerprompt Thanks man🙌

[@johnkintree763]

​@@engineerpromptThe Zamba2 family of models looks interesting for running locally with lower latency and more tokens/sec output.

[@guscastilloa]

How do you record your screen so that it follows your cursos? It’s super useful to follow along specially when reading papers! Kudos on this amazing expose of this methodology!

[@dylanmoraes990]

It's a video editing app called screen studio on mac

[@RickySupriyadi]

wow so many RAG system this year already

[@engineerprompt]

Yup, its hard to track but nice to see the different ideas that are coming up.

[@ai_dart]

Good Info

[@mclachan]

Have you tested it on structured data?

[@muhammadshafiqsafian6149]

how can i view the algo flowchart? a bit complex to understand

[@remusomega]

Graphs solve the problem of chunk embeddings being de-contextualized. Late Chunking solves this problem. I think we need to re-consider the use cases for GraphRags.

[@syedsaifullahtarique]

How to create LightRAG object inside dicken folder

[@dawid_dahl]

Thanks for the really great video, by the way.

[@ashgtd]

great video thank you

[@axelwehmeyer9599]

cool. Is there an implementation for Groq-API like gpt_4o_mini_complete-API in LightRAG? How can i use a GUI-Chatbot for LightRAG, e.g. chainlit/streamlit/...? thx

[@engineerprompt]

I think there are a couple of PRs for other models. Not sure about the GUI

[@ingenierofelipeurreg]

Which of all RAG is cost effective and quality better?

[@engineerprompt]

Its a hard question to answer. It will really depend on the task at hands. If you think its just search/looking up information, may be a standard RAG. If there are relationships between objects/entities then may be a knowledge graph.

[@Jvo_Rien]

thank you :)

[@AhmedMagdy-ly3ng]

Hey 👋 I'm one of the most exciting fans of you and I have the opportunity to come to India I wish I can see you and have a conversation.. Please 🥺🙏

[@akshatgandhi7958]

Thanks

[@themax2go]

so essentially they implemented triples / triplets (sciphi/triplex) 🤔

[@engineerprompt]

pretty much :)