GraphRAG Ollama: 100% Local Setup, Keeping your Data Private

No video

GraphRAG Ollama: 100% Local Setup, Keeping your Data Private

Рет қаралды 14,758

Mervin Praison

Күн бұрын

Пікірлер: 82

@anubisai Ай бұрын

Great work as usual. Humble. Concise. Helpful. Perfect. 👌

@maxs6128 Ай бұрын

Hey! Cool video. I actually built a full local solution using Ollama, no need for LM Studio at all. Here's what I did: I created a proxy that translates between OpenAI API embeddings and Ollama's format, both ways. The cool thing is, it works flawlessly for both global and local queries. I'd be happy to share the script with you if you're interested!

@matthewsoltis3383 Ай бұрын

Please.

@MervinPraison Ай бұрын

Yes please

@maxs6128 Ай бұрын

ollama embeddings proxy on git hub

@user-ul6fz8fw3l Ай бұрын

Please.

@maxs6128 Ай бұрын

ollama embeddings proxy on git hub

@SullyOrchestration Ай бұрын

Can you please show a way to visualize the knowledge graph with an interactive UI?

@ignaciopincheira23 Ай бұрын

It is essential to conduct a thorough preprocessing of the documents before entering them into the RAG. This involves extracting the text, tables, and images, and processing the latter through a vision module. Additionally, it is crucial to maintain content coherence by ensuring that references to tables and images are correctly preserved in the text. Only after this processing should the documents be entered into a LLM.

@nikhielsingh748 Ай бұрын

best GenAI KZbinr, I mean it .

@MattJonesYT Ай бұрын

Looking forward to more on this, it is the most interesting cutting edge tech in AI and almost no one else on youtube is talking about it

@NimaAmini2000 Ай бұрын

Another great video about GraphRAG, good job.

@Gurdershan Ай бұрын

I was eagerly waiting for this, big thanks

@macjonesnz Ай бұрын

Mervin, Hi from New Zealand, I see that took 20 minutes to index…. what are the specs of your machine?

@d.d.z. Ай бұрын

Same question. My computer runs Gemma 2 quite slow and I prefer to use Llama3 or Phi. The results will be the same?

@dudicrous Ай бұрын

@@d.d.z. What's your pc spec?

@d.d.z. Ай бұрын

@@dudicrous Intel core i5 8th gen 8gb RAM I have a HP pavilion 13-an0012la.

@MervinPraison Ай бұрын

It’s very slow for me too, I use Mac M2 32GB In the video I had to cut that part, because it took 26 mins just for indexing and considering it’s small chunk of data

@yingzhu505 18 күн бұрын

Great tutorial! Thank you!

@xinzhang3502 Ай бұрын

So compared to GPTs, his search generation effect will be better?

@alamgirqazi1 Ай бұрын

You are amazing 🎉

@Gwaboo Ай бұрын

can you please show or explain how to get the visualization of the data ? looks verry good, and thanks for the tutorial

@MeinDeutschkurs Ай бұрын

A start, great!

@girijeshthodupunuri1300 Ай бұрын

This is not all feasible on my computer but I would love move graph rag videos aiming more at how we can get this technology production ready.

@sharankumar31 Ай бұрын

Thank you for this tutorial. Very useful..

@JV61 Ай бұрын

This is what I was looking for

@nikhielsingh748 Ай бұрын

quick question, I already have a folder of embeddings and chunks, can I just pass the documents and embeddings to GraphRAG ?

@song8775 23 күн бұрын

Hi, how do you fix the issues with running local search using command line?

@nz6291 Ай бұрын

Thanks this really helped!

@GeertBaeke Ай бұрын

Good stuff. As expected, on a Mac M2, indexing and global queries are quite slow. Local queries are doable because it's usually just one LLM call after the similarity & graph search.

@BatukeshwarVats 23 күн бұрын

Nice and useful video, but still not getting one thing. You made this video around 3 weeks ago, but in april , ollama released some embedding models. Then how we are saying it is not having embedding compatibility?.

@SonGoku-pc7jl Ай бұрын

thanks. coming soon local vs. global

@MattJonesYT Ай бұрын

At 7:10 I believe the reason it's giving errors is the url in the settings file is missing the word embeddings at the end. It probably tested some different urls until it figured it out.

@mllearning-qc6dt Ай бұрын

What is the average query time that you were experiencing with the global/local search?

@lesptitsoiseaux Ай бұрын

What a perfect video to wake up to after yesterday's video :) I'm starting to think that we're abusing graphRAG here, all of us. You see, and I may be wrong I'm still a n00b here, we are not using semantic chunking and also, for those of us with thousands of files, say transcripts, feeding graphRAG a summary and tags might be good enough for a recommendation engine and if the user wants to dive in, then you use rag but you create a rag for each main collection of documents. So the graph rag may be able to list say what cooking classes you can take much faster and then querying each class that is its own rag for details should be also much faster and overall cheaper? What do you think?

@MervinPraison Ай бұрын

Basic RAG is fine for basic tasks but this GraphRAG is for advanced and more meaningful response.

@eduardoconcepcion4899 Ай бұрын

Can't we use nomic-embed-text provided by ollama for the embeddings?

@shiweiluo Ай бұрын

Why do you read the settings.yaml file by default when you create an index, but mine reads the .env file?

@jcalonsoc Ай бұрын

Great video! Can you export a CSV file to visualize the graph using an external tool like Gephi?

@mamadousow7323 Ай бұрын

Didn’t understand the final sentence, running things in llmstudio , what about many pdf documents

@chending5419 25 күн бұрын

After a few tries, my conclusion on graphRAG was that it is buggy when running local, took too long to process - as such that it is not practical to run locally.

@codelucky Ай бұрын

Can you create a video on how to use GraphRAG with the GROQ API? Looks like nobody has done it yet. Thank you.

@rockypunk91 Ай бұрын

I want to understand, how can we use it in a actual application. I reality users will upload their documents anytime they want, If I run indexer for different documents seperately it creates a seperate timestamp based folders in output, now how will the graph rag work when we have multiple artifacts? Our do I have to run indexer on entire documents even if one new document is added? and how do we trigger it programitically

@3dW0od Ай бұрын

Something is off. It works with openapi, no problem... But local models and embedding models - no chance. Followed all instructions. EDIT: httpS made me problems for local model, amateur issue. Sorry. It works locally now. Thank you.

@debarghyadasgupta1931 Ай бұрын

Would you recommend GraphRAG for structured data as well like Postgres or MySql? I am still stuck between LLM SQL agent vs Vectors. I did explored Vanna already and like it. Appreciate your thoughts on this.

@Gwaboo Ай бұрын

how can we use graph rag on data that's in another language? i have hundreds of documents i want to put into a rag but i cant get a good result out of the rag. the only things that come out are generic and often even things that i didn't ask about. do i have to use a German llm and embeddings model and translate all the prompts into German or do i have to translate all the data into English and live with it being in English?

@changtimwu Ай бұрын

Is LM Studio necessary? I believe OpenWebUI should also suffice, potentially creating a completely open-source graphrag solution.

@Yannick-ei2tz Ай бұрын

Thx for the prez. It is about graph so is it possible to get a grip on the underlying graph db and vizualise it using a ds tool ?

@mohsenghafari7652 Ай бұрын

Thanks

@dove8998 Ай бұрын

You need an OpenAI API Key to run it. It is unfortunatly not 100% local. Is there a way to run it without an OpenAI paid subscription?

@linmichael7935 Ай бұрын

yes you can, just export the "string or some text" for the OpenAI API Key. And then you can run it locally.

@bradmiles358 Ай бұрын

Great stuff, id be really interested to see csv rather than txt import as it looks like it might possibly give higgher reasoning by leveraging the structure. Ie better temporal reasoning etc... source_column: "author" # the column containing the source/author of the data text_column: "message" # the column containing the text of the data timestamp_column: "date(yyyyMMddHHmmss)" # optional, the column containing the timestamp of the

@dawn_of_Artificial_Intellect Ай бұрын

can you use NIM to keep your Data Private?

@KS-tj6fc Ай бұрын

Do a video that finds the balance of speed (local takes forever) and cost by using DeepSeek-V2, which is only $0.14/million tokens input and $0.28/million tokens output. Once you get your results, I would say that at RAG = 2 the Deepseek would be at least equal to or slightly better than GPT 4o at =1. Then add an additional python step, prompt to take overall points and run limited amounts of tokens via GPT 4o or even Gemini Flash 1.5, which is quite good, for improvement to 1.6~1.7 level outputs at almost “free token costs”.

@lemon268 Ай бұрын

is this completely free? or are there open api calls?

@mrchongnoi Ай бұрын

Good video. Please do work out the error you encountered. Do you have a GPU on your laptop. 20+ minutes makes this unusable for a company with 100s or Thousands of documents.