pdf.py has been replaced with upload.py (which handles pdf, txt and json)
@ThriftyEngineer5 ай бұрын
Thank you! I was stuck here.
@mikaelroed21074 ай бұрын
neither work for me, do I need to install something?
@GEORGE.M.M2 ай бұрын
@@mikaelroed2107 Make sure your in the same directory you cloned the github repo, basically find the 'easy-local-rag' and in the file explorer address bar type cmd or powershell hit enter and it should work, or just load powershell from start and when it loads type 'cd whateverTheLocationOfYourFolderIs'
@gianni4302Ай бұрын
@@mikaelroed2107 make sure you have both the llama3 model and the mxbai-embed-large embedding downloaded through ollama
@davidtindell9502 ай бұрын
So far, some of my "key findings" regarding "different Ollama-supported models" include: 1/5. Use OpenAI's "text-embedding-3-large" for high-quality embeddings -- but it is somewhat expensive. 2/5. Use "text-embedding-3-small" for a balance between performance and cost. 3/5. In addition to "llama3.1:8b", the "mistral:latest" model has good performance across various tasks. 4/5. For PDF's, use text extraction tools like PyPDF2 or pdfminer, but we must remove or skip encoding errors. Finding the ideal chunk size and overlap is difficult! 5/5: We must set up good benchmark datasets of relevant PDFs to compare results. ALSO: Unfortunately, "faiss-gpu" appears to be deprecated but an older conda version can be run under MS Win 10/11; however, the latest version appears to run only under Linux and, perhaps, Mac OS !?!?!
@rickmarques105 ай бұрын
Hey! There's any solution to make a good RAG like this using ollama on Openwebui?
@BalvinderNagi5 ай бұрын
I get the following error: python localrag.py Traceback (most recent call last): File "/Users/eil-its/Documents/experiments/workspace-python/llama3rag/localrag.py", line 130, in response = ollama.embeddings(model='mxbai-embed-large', prompt=content) File "/Users/eil-its/Documents/experiments/workspace-python/llama3rag/llama/lib/python3.9/site-packages/ollama/_client.py", line 198, in embeddings return self._request( File "/Users/eil-its/Documents/experiments/workspace-python/llama3rag/llama/lib/python3.9/site-packages/ollama/_client.py", line 73, in _request raise ResponseError(e.response.text, e.response.status_code) from None ollama._types.ResponseError: failed to generate embedding
@chriskingston19815 ай бұрын
me too, found any solution?
@kumarrohit5575 ай бұрын
same facing this error
@TheSalto664 ай бұрын
same error
@ariel79043 ай бұрын
ollama pull mxbai-embed-large , thats works for me
@GEORGE.M.M2 ай бұрын
Try another Model delete the model he tells you to use in the video 'mistral' you can do this by running the command 'ollama rm mistral' and then install a model that is working for me and the one he has listed in his github llama3 'ollama pull llama3'. and do not forget 'ollama pull mxbai-embed-large'.
@gumshoe94965 ай бұрын
The discord link is still not working for me. Says it’s expired. Did you update the link or is this something on my end?
@JonathanDeCollibus5 ай бұрын
same
@mrkubajski95285 ай бұрын
can I add a folder including multiple PDFs and txts?
@benafedeyim3 ай бұрын
So if I want to teach my AI my uni lessons, should I make all my pptx pdf docx files in to a single pdf . I am new to AI training and I am kinda struggling
@streetroller10005 ай бұрын
Just want to say that you're probably one of the easiest to follow and most intuitive persons I've seen on KZbin doing guides for LLMs! Thanks!
@AllAboutAI5 ай бұрын
thnx mate:) appriciate it!
@genemathewquijano11-initia718 күн бұрын
I know this might be weird but if I were to uninstall everything including the envirenments installed. How would I do that?
@SSHVWork-px6ry5 ай бұрын
it works after ollama run mistral this thank you
@michaelgoralski80283 ай бұрын
Does anyone else's script get stuck on generating the embeddings when you run it?
@AbhishekKumar-vt1iu3 ай бұрын
same here.. anyone having solution to this.
@jayateerthkatti7722Ай бұрын
@@AbhishekKumar-vt1iu it is due to limitations of computing resources in local machine
@ronaldreck2 ай бұрын
didnt work for me, despite what the video said the code is using from openai import OpenAI
@joshuapaul80425 ай бұрын
very interesting. issues: 1. it uses tkinter! 2. it strips the text from the pdf so it doesn't preserve page numbers - so you can't ask questions about where the text was found.
@TonySchiffbauer10 күн бұрын
This is cool! how can you run this on a web interface? just curious
@davidtindell9503 ай бұрын
I am experimenting with different Ollama-supported models and embedding models to see what current works 'best' for PDF's. Any Recommendations ? Thank You.
@cjjb2 ай бұрын
Any findings?
@davidtindell9502 ай бұрын
@@cjjb So far, some of my "key findings" regarding "different Ollama-supported models" include: 1/5. Use OpenAI's "text-embedding-3-large" for high-quality embeddings -- but it is somewhat expensive. 2/5. Use "text-embedding-3-small" for a balance between performance and cost. 3/5. In addition to "llama3.1:8b", the "mistral:latest" model has good performance across various tasks. 4/5. For PDF's, use text extraction tools like PyPDF2 or pdfminer, but we must remove or skip encoding errors. Finding the ideal chunk size and overlap is difficult! 5/5: We must set up good benchmark datasets of relevant PDFs to compare results. ALSO: Unfortunately, "faiss-gpu" appears to be deprecated but an older conda version can be run under MS Win 10/11; however, the latest version appears to run only under Linux and, perhaps, Mac OS !?!?!
@davidtindell9502 ай бұрын
@@cjjb "So far, some of my "key findings" regarding "different Ollama-supported models" include: ... 1/5. Use OpenAI's "text-embedding-3-large" for high-quality embeddings -- but it is somewhat expensive. 2/5. Use "text-embedding-3-small" for a balance between performance and cost. 3/5. In addition to "llama3.1:8b", the "mistral:latest" model has good performance across various tasks. 4/5. For PDF's, use text extraction tools like PyPDF2 or pdfminer, but we must ... remove or skip encoding errors. Finding the ideal chunk size and overlap is difficult! 5/5: We must set up good benchmark datasets ... of relevant PDFs to compare results. ALSO: Unfortunately, "faiss-gpu" appears to be deprecated but an older conda version can be ... run under MS Win 10/11; however, the latest version appears to run only under Linux and, perhaps, Mac OS !?!?! "
@davidtindell9502 ай бұрын
@@cjjb please see the “key findings” that I posted for everyone’s review.
@alexanderp752113 күн бұрын
hi, can i build all the same using docker?
@ia360graus5 ай бұрын
Thank you so much for this tutorial, Could you make a video with some use cases with RAG in your daily life?
@davidtindell9504 ай бұрын
Again, Thank You! As you suggest this local RAG program works fairly well and is certainly 'good enough' for my personal use cases.
@nic-ori5 ай бұрын
👍Thanks! Useful information.👍
@AllAboutAI5 ай бұрын
no problem:) tnx for tuning in!
@luisjaimes98494 ай бұрын
Great! How to remove pdf?
@CharlesOkwuagwu5 ай бұрын
Please how would you advice we run this on JSON files, not PDF. I have several Q-A in . JSON format, not PDF.
@ClipsofCoolStuff5 ай бұрын
I made a pull request to the repo and he accepted it. If you repull the repo you should now have access to uploading PDF, TXT, and JSON files into the context.
@JBB6852 ай бұрын
@@ClipsofCoolStuffmarkdown? 😅
@sebastianbecerra10345 ай бұрын
no working in mac
@fernabot5 ай бұрын
Really nice work! .. but is this really fully local? ... OpenAI?
@DoctorMandible5 ай бұрын
... Huh?
@crawfordscott3d5 ай бұрын
Really appreciate all your content and how much energy you put into learning all this and sharing it. Thanks buddy
@brianclark46395 ай бұрын
Getting this error: import PyPDF2 File "C:\Anaconda\lib\site-packages\PyPDF2\__init__.py", line 12, in from ._encryption import PasswordType File "C:\Anaconda\lib\site-packages\PyPDF2\_encryption.py", line 34, in from ._utils import logger_warning File "C:\Anaconda\lib\site-packages\PyPDF2\_utils.py", line 55, in from typing_extensions import TypeAlias ModuleNotFoundError: No module named 'typing_extensions'
@Thressian4 ай бұрын
Hi, thank you for the brilliant video and tutorial! I was wondering if you had any experience with implementing this process with LlaMA3 rather than Mistral, or if there was any difference in the implementation? Thank you :)
@VeronicaAngelesEscudero4 ай бұрын
Thanks to share your kwoledge with us! I have some questions because i have a similar code but it is not work as welll when the context is so extensive, how can i do? also, the chatbot lost when the history chat is large, i don't know hot to fix it. Yo have some idea? thanks
@luisjaimes98492 ай бұрын
script get stuck on generating the embeddings!
@enesgul29705 ай бұрын
Harikasınız
@userrjlyj5760g5 ай бұрын
What if my pdf has like a 1 million word/token, would this still work?
@leadnauta4 ай бұрын
muchas gracias, muy util, funciono el codigo
@pm12345 ай бұрын
Great! Thank you! I'd like to have the same for local code (django). One liners won't work, so how to do this?
@Dylan-jq7hu5 ай бұрын
Will you be doing a video’s on ChatGPT the App version anymore?
@cclementson19865 ай бұрын
How would you deploy this in AWS for production? Would you use ollama and download the entire LLM model on an EC2 instance?
@ashishkgp5 ай бұрын
use an inference endpoint for llm and a cpu based ec2 instance. It may be cheaper
@Anzeljaeg3 ай бұрын
Hello sir lets get this on!!!
@ddricci124 ай бұрын
Cool Beans! Thanks so very much you have helped me enormously. Thanks again
@pearlynrodrigues48163 ай бұрын
The phrase, "cool beans" gives me so much nostalgia.
@gilzonme5 ай бұрын
Oh Man that worksssssssssss!
@_TheDudeAbides_3 ай бұрын
Thanks for this video!
@JNET_Reloaded5 ай бұрын
very good stuff
@TheHistoryCode1255 ай бұрын
This video is a misleading tutorial that oversimplifies the process of creating a local RAG (Retrieval Augmented Generation) system. While the presenter claims to create a "100% local RAG in around 70 lines of code," they fail to address the complexities and limitations of such a system. The tutorial relies heavily on pre-built libraries and models, such as Ollama and sentence-transformers, without providing a deep understanding of how these components work together. Moreover, the presenter does not discuss the potential drawbacks of using a local RAG system, such as the limited amount of data it can handle and the lack of real-time updates. The video may give viewers a false sense of ease in implementing a RAG system, without considering the necessary expertise and resources required for a robust and reliable solution.
@sluggy60745 ай бұрын
Thanks, GPT 4.
@pm12345 ай бұрын
FYI GPTZero states this comment is 100% AI generated.
@userou-ig1ze5 ай бұрын
Jeez, it almost academic if it weren't reeking of AI
@bolonekoplo25614 ай бұрын
Hahaha 😂 @@userou-ig1ze
@ipoop43593 ай бұрын
Well no shit no one’s gonna write a whole semantic search on parsed documents. Thats like saying “im gonna program my own garbage collector in java”
@user---------5 ай бұрын
Which video card and how much memory?
@frosti73 ай бұрын
awesome, i dont get it, anything easier that does not require any code ? thank you :)
@slevinhyde32123 ай бұрын
LOL
@mytradingbuddy-vp5vmАй бұрын
Free Palestine.
@kumarrohit5575 ай бұрын
i found an error when i run localrag.py File "D:\easy-local-rag-main\localrag.py", line 134, in response = ollama.embeddings(model='mxbai-embed-large', prompt=content) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\HP\AppData\Local\Programs\Python\Python312\Lib\site-packages\ollama\_client.py", line 198, in embeddings return self._request( ^^^^^^^^^^^^^^ File "C:\Users\HP\AppData\Local\Programs\Python\Python312\Lib\site-packages\ollama\_client.py", line 73, in _request raise ResponseError(e.response.text, e.response.status_code) from None ollama._types.ResponseError: failed to generate embedding
@kumarrohit5575 ай бұрын
please guide
@EduardoJGaido5 ай бұрын
@@kumarrohit557 have you tried before doing that ollama pull mxbai-embed-large ?
@Mike-dl7gt3 ай бұрын
ollama pull mxbai-embed-large
@Mike-dl7gt3 ай бұрын
@@EduardoJGaido .
@Mike-dl7gt3 ай бұрын
@@kumarrohit557 .
@SSHVWork-px6ry5 ай бұрын
raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: Error code: 404 - {'error': {'message': "model 'mistral' not found, try pulling it first", 'type': 'api_error', 'param': None, 'code': None}}
@grahaml60725 ай бұрын
You have to download the model. ollama pull mistral
@pojomcbooty5 ай бұрын
This is pretty epic !! I wonder if pdfplumber might be a better fit than pypdf2 ? I have been working with plain text and that works great, but a simple text-only PDF downloaded from google docs gets quite garbled. maybe pdfplumber might not be much better tbh but I've generally heard it's very solid. I also wonder how difficult would it be to add re-ranking / recursive retrieval / corrective RAG ? are any of those even necessary given the way sentence transformers work ? Either way - big kudos for all your awesome work and sharing it with the commuity !!
@pojomcbooty5 ай бұрын
would also love a comparison with a RAG based on knowledge graph (high level) - if anybody can graph this out and explain this well, it's you !! I've watched pretty much every youtube video there is that promises the earth from PrivateGPT etc. but you are dispensing actual knowledge !!!