GraphRAG Ollama: 100% Local Setup, Keeping your Data Private

[@anubisai]

Great work as usual. Humble. Concise. Helpful. Perfect. 👌

[@SullyOrchestration]

Can you please show a way to visualize the knowledge graph with an interactive UI?

[@maxs6128]

Hey! Cool video. I actually built a full local solution using Ollama, no need for LM Studio at all. Here's what I did: I created a proxy that translates between OpenAI API embeddings and Ollama's format, both ways. The cool thing is, it works flawlessly for both global and local queries. I'd be happy to share the script with you if you're interested!

[@matthewsoltis3383]

Please.

[@MervinPraison]

Yes please

[@maxs6128]

ollama embeddings proxy on git hub

[@MatheusVieiradaSilva-g1w]

Please.

[@maxs6128]

ollama embeddings proxy on git hub

[@MattJonesYT]

Looking forward to more on this, it is the most interesting cutting edge tech in AI and almost no one else on youtube is talking about it

[@nikhielsingh748]

best GenAI KZbinr, I mean it .

[@macjonesnz]

Mervin, Hi from New Zealand, I see that took 20 minutes to index…. what are the specs of your machine?

[@d.d.z.]

Same question. My computer runs Gemma 2 quite slow and I prefer to use Llama3 or Phi. The results will be the same?

[@dudicrous]

@@d.d.z. What's your pc spec?

[@d.d.z.]

@@dudicrous Intel core i5 8th gen 8gb RAM I have a HP pavilion 13-an0012la.

[@MervinPraison]

It’s very slow for me too, I use Mac M2 32GB In the video I had to cut that part, because it took 26 mins just for indexing and considering it’s small chunk of data

[@yingzhu505]

Great tutorial! Thank you!

[@GeertBaeke]

Good stuff. As expected, on a Mac M2, indexing and global queries are quite slow. Local queries are doable because it's usually just one LLM call after the similarity & graph search.

[@Gwaboo]

can you please show or explain how to get the visualization of the data ? looks verry good, and thanks for the tutorial

[@Gurdershan]

I was eagerly waiting for this, big thanks

[@ignaciopincheira23]

It is essential to conduct a thorough preprocessing of the documents before entering them into the RAG. This involves extracting the text, tables, and images, and processing the latter through a vision module. Additionally, it is crucial to maintain content coherence by ensuring that references to tables and images are correctly preserved in the text. Only after this processing should the documents be entered into a LLM.

[@girijeshthodupunuri1300]

This is not all feasible on my computer but I would love move graph rag videos aiming more at how we can get this technology production ready.

[@Independent_AI]

Another great video about GraphRAG, good job.

[@debarghyadasgupta1931]

Would you recommend GraphRAG for structured data as well like Postgres or MySql? I am still stuck between LLM SQL agent vs Vectors. I did explored Vanna already and like it. Appreciate your thoughts on this.

[@patriktolomeotti]

Is it possible to perform this queries with LM Studio(or any other interface) instead of using terminal?

[@xinzhang3502]

So compared to GPTs, his search generation effect will be better?

[@nikhielsingh748]

quick question, I already have a folder of embeddings and chunks, can I just pass the documents and embeddings to GraphRAG ?

[@BatukeshwarVats]

Nice and useful video, but still not getting one thing. You made this video around 3 weeks ago, but in april , ollama released some embedding models. Then how we are saying it is not having embedding compatibility?.

[@nz6291]

Thanks this really helped!

[@lesptitsoiseaux]

What a perfect video to wake up to after yesterday's video :) I'm starting to think that we're abusing graphRAG here, all of us. You see, and I may be wrong I'm still a n00b here, we are not using semantic chunking and also, for those of us with thousands of files, say transcripts, feeding graphRAG a summary and tags might be good enough for a recommendation engine and if the user wants to dive in, then you use rag but you create a rag for each main collection of documents. So the graph rag may be able to list say what cooking classes you can take much faster and then querying each class that is its own rag for details should be also much faster and overall cheaper? What do you think?

[@MervinPraison]

Basic RAG is fine for basic tasks but this GraphRAG is for advanced and more meaningful response.

[@MeinDeutschkurs]

A start, great!

[@jcalonsoc]

Great video! Can you export a CSV file to visualize the graph using an external tool like Gephi?

[@sharankumar31]

Thank you for this tutorial. Very useful..

[@alamgirqazi1]

You are amazing 🎉

[@siliconberry]

Ollama does provide an embeddings. then why do we need to use LMStudio ?

[@SonGoku-pc7jl]

thanks. coming soon local vs. global

[@Yannick-ei2tz]

Thx for the prez. It is about graph so is it possible to get a grip on the underlying graph db and vizualise it using a ds tool ?

[@codelucky]

Can you create a video on how to use GraphRAG with the GROQ API? Looks like nobody has done it yet. Thank you.

[@changtimwu]

Is LM Studio necessary? I believe OpenWebUI should also suffice, potentially creating a completely open-source graphrag solution.

[@MattJonesYT]

At 7:10 I believe the reason it's giving errors is the url in the settings file is missing the word embeddings at the end. It probably tested some different urls until it figured it out.

[@song8775]

Hi, how do you fix the issues with running local search using command line?

[@mllearning-qc6dt]

What is the average query time that you were experiencing with the global/local search?

[@Gwaboo]

how can we use graph rag on data that's in another language? i have hundreds of documents i want to put into a rag but i cant get a good result out of the rag. the only things that come out are generic and often even things that i didn't ask about. do i have to use a German llm and embeddings model and translate all the prompts into German or do i have to translate all the data into English and live with it being in English?

[@JV61]

This is what I was looking for

[@eduardoconcepcion4899]

Can't we use nomic-embed-text provided by ollama for the embeddings?

[@lemon268]

is this completely free? or are there open api calls?

[@shiweiluo]

Why do you read the settings.yaml file by default when you create an index, but mine reads the .env file?

[@rockypunk91]

I want to understand, how can we use it in a actual application. I reality users will upload their documents anytime they want, If I run indexer for different documents seperately it creates a seperate timestamp based folders in output, now how will the graph rag work when we have multiple artifacts? Our do I have to run indexer on entire documents even if one new document is added? and how do we trigger it programitically

[@KS-tj6fc]

Do a video that finds the balance of speed (local takes forever) and cost by using DeepSeek-V2, which is only $0.14/million tokens input and $0.28/million tokens output. Once you get your results, I would say that at RAG = 2 the Deepseek would be at least equal to or slightly better than GPT 4o at =1. Then add an additional python step, prompt to take overall points and run limited amounts of tokens via GPT 4o or even Gemini Flash 1.5, which is quite good, for improvement to 1.6~1.7 level outputs at almost “free token costs”.

[@mrchongnoi]

Good video. Please do work out the error you encountered. Do you have a GPU on your laptop. 20+ minutes makes this unusable for a company with 100s or Thousands of documents.

[@bradmiles358]

Great stuff, id be really interested to see csv rather than txt import as it looks like it might possibly give higgher reasoning by leveraging the structure. Ie better temporal reasoning etc... source_column: "author" # the column containing the source/author of the data text_column: "message" # the column containing the text of the data timestamp_column: "date(yyyyMMddHHmmss)" # optional, the column containing the timestamp of the

[@dawn_of_Artificial_Intellect]

can you use NIM to keep your Data Private?

[@mohsenghafari7652]

Thanks

[@mamadousow7323]

Didn’t understand the final sentence, running things in llmstudio , what about many pdf documents

[@dove8998]

You need an OpenAI API Key to run it. It is unfortunatly not 100% local. Is there a way to run it without an OpenAI paid subscription?

[@linmichael7935]

yes you can, just export the "string or some text" for the OpenAI API Key. And then you can run it locally.

[@am4_m037]

I got this error during the indexing raise ApiKeyMissingError graphrag.config.errors.ApiKeyMissingError: API Key is required for Completion API. Please set either the OPENAI_API_KEY, GRAPHRAG_API_KEY or GRAPHRAG_LLM_API_KEY environment variable. ⠋ GraphRAG Indexer

[@chending5419]

After a few tries, my conclusion on graphRAG was that it is buggy when running local, took too long to process - as such that it is not practical to run locally.

[@JrTech-rw6wj]

can i use gemini model with graph rag ??

[@MervinPraison]

Yes, but not straight out of the box as I can see. You might need to modify the code slightly

[@takshitmathur2761]

@@MervinPraison can you make a video on this please.

[@PradeepKumar-zy6cd]

The problem is LM studio getting error

[@ShadowyLane]

Something is off. It works with openapi, no problem... But local models and embedding models - no chance. Followed all instructions. EDIT: httpS made me problems for local model, amateur issue. Sorry. It works locally now. Thank you.

[@vamsivamsi2903]

@ShadowyLane How did you make it work for local models can u please tell the solution

[@zz-dy7bz]

Are you serious? It's not working at all.

[@iham1313]

as long as ms does not add other formats besides txt and csv, this graphrag is useless to me and all the business use cases i know and have.

[@eggmaster88]

Why? Can't you transform the data?

[@iham1313]

@@eggmaster88 can you? including images, diagrams, tables, metadata like page numbers, … please tell me, if there is a good solution

[@eggmaster88]

@@iham1313 for tables you can use .csv from what i know but it seems that you don't really need graph database for your data, more like sql.

[@littledaddi3]

@@iham1313 look up LangChain GraphRag vids on the topic - they show how to do images diagrams tables and metadata.

[@zhengwu-jw6fm]

GRAPHRAG_API_KEY= "ollama"？big thanks

[@matthewsoltis3383]

I got it to run without adding any key

[@gagnongui78]

@@matthewsoltis3383 how please?