FULLY LOCAL Mistral AI PDF Processing [Hands-on Tutorial]

Using faiss, sentence transformers and ctransformers, we have got a fully functional completely LOCAL AI powered PDF Processing engine powered by Mistral AI
Colab - colab.research...
Katana ML's Project - github.com/kat...
❤️ If you want to support the channel ❤️
Support here:
Patreon - / 1littlecoder
Ko-Fi - ko-fi.com/1lit...
🧭 Follow me on 🧭
Twitter - / 1littlecoder
Linkedin - / amrrs

[@jorgerios4091]

Wow, practical use videos are back, thanks!

[@1littlecoder]

Haha Thanks!

[@L1merick]

Yup, love the practical use-case videos! ❤

[@azuremis]

Epic video, really helped me out with my first local use of an LLM!

[@1littlecoder]

Glad it helped!

[@azp7]

Thank you! I had some issues with some Python Packages, that I could resolved. Your Hands-on tutorial is great, very practical!

[@amindforevervoyaging3789]

Thanks. Keep up the great work 🙂

[@1littlecoder]

Thank you too!

[@dreamwalker1739]

how to get get output as json format. I used json outputparser for getting output , but it gives output as schema. How to fix this?

[@santanuc]

this is good, but can it handle complex unstructured tables or text with images?

[@__________________________6910]

9:57 "I'm not just cheating around" 🤣

[@1littlecoder]

Haha. People make such allegations so wanted to make sure 😁

[@__________________________6910]

@@1littlecoder lol

[@eriktedja1199]

Is it possible to handle hundreds of documents each around 200 - 300 pages long

[@gabscar1]

Its for invoicing. Not large PDF reading. Video by the creator here: kzbin.info/www/bejne/b4Oog6impsh5gpY

[@papitou4140]

overall which open source model is the best at the moment for creating chatbots? but only the ones that are truly open source and commercial free without any llama licenses. thanks

[@rakshaballa]

Can I know how do you solve that error which is shown at 7.49?

[@SanketRaut-x6u]

Great tutorial why this model takes to much time for answering the question ?

[@RonanChardonneau-b7b]

How can you make it analyze another pdf? I deleted the invoice sample pdf, i uploaded another file into it (300 pages) and the answer given by mistral are saying that they don't know have the answers to my questions.

[@MichealScott24]

❤

[@Sri_Harsha_Electronics_Guthik]

nice.! gpu version?

[@jurandfantom]

Did I miss information how big files it is possible to process? I have 800 page pdf book for example - don't think that hardware can be limiting factor (24VRAM 128 RAM) in my case

[@hossromani]

Thanks, when pip install, it modifies the local packages, would you recommend Conda or Docker?

[@NageshPindi-y3v]

Will it work for HTML files??

[@lgprasad]

But using open source LLM, is it possible to restrict the RAG only to the documents, in this case the PDF(s) alone ? Won't it answer any or all generic questions too ?

[@1littlecoder]

Yes

[@lgprasad]

Means can we restrict the responses ? If so can you guide how?@@1littlecoder

[@AbishekEkhool]

finetuning quantized model is possible or not if it possible make video for that plz

[@philtoa334]

Nice.

[@1littlecoder]

Thank you! Cheers!

[@antiinfluencer2269]

the link is guff model is outdated

[@__________________________6910]

LOCAL -I love local private things

[@j.k.priyadharshini9753]

katanaml? does it belong to mistralAI?

[@1littlecoder]

No it's an open source by someone else

[@malleswararaomaguluri6344]

After 3 or 4 questions, it is showing token limit exceeded. How to overcome this?

[@MungeParty]

Summarize each perspective in the chat history by relevance to the question, then break the question down into steps. Repeat that recursively until the total tokens required for context plus chat history plus question add up to few enough tokens to run past your max on the model.

[@__________________________6910]

are you using MacBook Pro 14” or 16” ?

[@1littlecoder]

16" why? It's an old one though, been thinking to buy an M1 machine, but haven't!

[@__________________________6910]

@@1littlecoder noice

[@echofloripa]

Isn't it slow because it's CPU based?

[@1littlecoder]

It is slow exactly because of the CPU speed. Also the compute power makes a difference. You could see the time difference in Colab and on my machine. I'd see if i can port these to GPTQ models which can leverage GPU as well

[@gayathrik1517]

Very helpful, kindly do the needful with GPU machine as well.

[@1littlecoder]

Thanks, I'll try to see if this could be ported for GPTQ as well

[@alx8439]

6 minutes. Gosh. What does it do during all that time - mining etherium?

[@1littlecoder]

That'd be amazing if it can mine, I'd end up rich $$$ !

[@alx8439]

@@1littlecoderhaha. But not sure it's doing that for your favour :)

[@edwardlori4627]

Damn, it doesn't work ( Can you update this tutorial? Maybe some new tools are available?

[@1littlecoder]

What was the error ?

[@edwardlori4627]

@@1littlecoder Mmm... Looks like youtube automatically deletes the copied error.... I'll try it piece by piece...

[@edwardlori4627]

after "Python main.py "what is the name of the seller ?"" appears...

[@edwardlori4627]

Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 286, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url:

[@edwardlori4627]

raise RepositoryNotFoundError(message, response) from e huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-65dee93f-395f6cfe199fc1705cb53901;1c0ae4c6-0efd-4e1e-b6e6-9800213dbd3b) Repository Not Found for url: h%%%%%%%%%o/api/models/models/mistral-7b-instruct-v0.1.Q5_K_M.gguf/revision/main. Please make sure you specified the correct `repo_id` and `repo_type`. If you are trying to access a private or gated repo, make sure you are authenticated. Invalid username or password.@@1littlecoder

[@flethacker]

how could it possibly take 1 minute on a 1 page pdf? i dont think this code approach is any good tbh

[@adrianacala1034]

1 minute of compute type is significantly cheaper than 1 minute of human time.

[@ArtisanTony]

I am glad I did not get doxed in this video like these people did :)

[@1littlecoder]

I'm sorry didn't understand the comment well!

[@ArtisanTony]

@@1littlecoder It was humor, The person name and address on the invoice seems like personal information they would not want out on the internet :)