Langchain: PDF Chat App (GUI) | ChatGPT for Your PDF FILES | Step-by-Step Tutorial

"Build a ChatGPT-Powered PDF Assistant with Langchain and Streamlit | Step-by-Step Tutorial"
In this comprehensive tutorial, you'll embark on a project-based journey where we leverage Langchain and Streamlit to develop an interactive ChatGPT for your PDF documents. With the power of an LLM (Large Language Model) such as OpenAI's ChatGPT, we'll create an application that enables you to ask questions about PDFs and receive accurate answers.
🦾 Discord: / discord
☕ Buy me a Coffee: ko-fi.com/prom...
|🔴 Patreon: / promptengineering
💼Consulting: calendly.com/e...
📧 Business Contact: [e-mail]
Become Member: tinyurl.com/y5h...
💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).
Link to the code: pastebin.com/m...
Learn how to harness the power of Langchain, an open-source Python (and Javascript) framework, to create intelligent applications. Discover Langchain's capabilities in training GPT models on your data and generating personalized LLMs. Explore text embeddings and their integration with Langchain using OpenAI's API.
In this tutorial, we'll guide you through building a fully functional Streamlit application. Train GPT on PDF documents and fine-tune it to your specific use case. Experience the seamless user interface as you upload PDFs, ask questions, and receive prompt answers from the LLM.
Unleash Langchain's versatility in chatbots, document analysis, and more. Automate tasks and improve efficiency using Langchain with Streamlit.
Take your natural language processing skills to the next level. Start building powerful applications with Langchain today!
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

[@engineerprompt]

If you are interested in learning more about how to build robust RAG applications, check out this course: prompt-s-site.thinkific.com/courses/rag

[@TheCopernicus1]

Wow amazing! Now this retro fitted with privateGPT would be gold!

[@jorgerios4091]

What would change if we decide to use an open source LLM, what lines should be changed and how? Would be a good idea for a future video

[@brian2590]

Also a good idea for the wallet 😎

[@luizaugusto8154]

I agree too

[@thunderwh]

This, plus I want to load in directories with PDFs as entire topical collections to query

[@engineerprompt]

For Open source options: Check this out for embeddings: kzbin.info/www/bejne/pZiokp-Ggrp2ibs Check this out for LLMs: kzbin.info/www/bejne/raOnXpmQq7lpi6s

[@ianabrahams5434]

Learnt a lot from just this one video. Appreciate all the hard work that went into making these fantastic tutorials. Thanks very much.

[@engineerprompt]

Thank you :)

[@huilingxie4829]

Same up there! Please teach us more end to end project, i really learned things here@@engineerprompt

[@JunWenSoh-f9w]

Great stuff for explaining the flow of the architecture so well. I like the pace and also how details regarding each part of the architecture are explained with just enough details not to overwhelm the viewer.

[@engineerprompt]

Glad you enjoyed it!

[@bobhoward4530]

Thank you for this clear, concise explanation. It’s definitely the best I’ve found so far. New to Langchain and quite excited. Thanks for your help.

[@engineerprompt]

You're very welcome!

[@fishandcat4281]

It is my first time to know streamlit....That's amazing!!! Thanks for the sharing. Your step by step video is ultra clear... Thanks a lot.

[@engineerprompt]

Glad it was helpful!

[@user-zk1zm6sm2u]

Tried the whole videos, really functional and you have a great insight in different concepts. Thanks a lot for this detailed video. Once of the best videos watched in recent times.

[@engineerprompt]

glad you found it useful.

[@terradyn]

Would love an open source version of this that we can swap in different embedding and llm models as the space evolves.

[@engineerprompt]

Coming soon :)

[@efexzium]

Its to slow for a book

[@georgetuges4821]

Thank you so much, Ive been self taught on AI and this video has really helped me on AI... God Bless

[@engineerprompt]

Want to connect? 💼Consulting: calendly.com/engineerprompt/consulting-call 🦾 Discord: discord.com/invite/t4eYQRUcXB ☕ Buy me a Coffee: ko-fi.com/promptengineering |🔴 Join Patreon: Patreon.com/PromptEngineering ▶ Subscribe: www.youtube.com/@engineerprompt?sub_confirmation=1

[@abdulhaditarar8539]

The Pastebin link is not working can u share the git repo link for this. that would be much help.

[@srh1034]

What is the benefit of using OPENAI API and call it local?! The whole point is to make it safer by not sending any data out or cutting cost buy not paying for API! Without those objectives, it is a useless tutorial

[@arpansangwan9293]

@engineerprompt

[@ReviewSmartTech]

Am halfway to the video and I am so excited, how easy you are making me to learn… kudos to you

[@engineerprompt]

Glad you are finding this useful

[@legendsanimexy8217]

Does it requires to buy chatgpt API to build this project ? ​@@engineerprompt

[@DouglasHeyen]

Thank you for these in depth videos. Learning so much. Thanks again!!

[@neelsoni13062]

One day you will get million pls make this amazingt videoes with new technologies daily for enhance our knowledge thank you so much brother !!!

[@shivamroy1775]

Great content, very well explained. Really appreciate the long videos with elaborate explanations.

[@engineerprompt]

Thank you 😊

[@alaaharb5987]

I'm not able to store the vectorStrore using pickle. Any one get this error ?! TypeError: cannot pickle '_thread.RLock' object

[@ronitvirwani4920]

yes me too

[@AbdulBasit-jp9dz]

Did u fix the error?

[@evgeni_logic]

Instead of pickle, use "vectorstore = FAISS.load_local(store_name,embeddings)" to load, and "vectorstore.save_local(store_name)" to store the embeddings.

[@kavin9313]

​@@evgeni_logicwhere to give chunk??

[@devyanshrastogi]

It was one of the coolest video on LLMs thanks for sharing such knowledge, I learnt alot from this video.

[@timmymanamperi]

Thank you for a beautiful step-by-step learning walkthrough. I like how you articulate each area clearly and precisely. Good on you, and keen to see more videos, and well done.

[@engineerprompt]

Glad it was helpful!

[@nazihfattal974]

The pace, contents and explanation are all great. Thank you for a great video. Always checking your channel for great content.

[@engineerprompt]

Thank you for your kind words 🙏

[@gapigorsek7878]

I get this error (when i load a pdf): AttributeError: module 'openai' has no attribute 'error' can you please help i have been debugging for 2 hours?

[@MateMatricks5119]

I was facing the same error, it means you are facing error in api calls and fetch, try to use api first in normal way if its working its good,else use some other llm like gemini or cohere

[@aj-tt9qh]

I am also facing this error..any luck??

[@aj-tt9qh]

@@MateMatricks5119 I am also facing this error..any luck??

[@gbemi9408]

Can we use a pdf thsg has images..?? Will the image also come shownin the questions?

[@arpansangwan9293]

Sir I’am facing an error EOFError: Ran out of input can you please help me out to solve this error , i have followed the same approach that you have done but still I’am facing thi error please help me out

[@mrmoffett7034]

Well done, this is just what im looking for. now i just need to find out how to clean old scanned PDF files so i can chat with them. 👍

[@Pubuditha]

Great Video And you explained it really well too. Earlier I was confused about why langchain is used but your video helped me to understand ! Thank you !

[@goforit5]

Best videos! Instruction is so clear. I learn so much from you. Thanks

[@engineerprompt]

Thank you!

[@MichaelLass]

Keep up the great work. I love your detailed videos.

[@mohideenabdulkadhar4280]

very good explanation

[@epictetus__]

Extremely useful. Thanks for making this awesome useful project

[@MrReesh]

Thank you for this video. Just signed up as a patron. Keep up the great work and all the best to you.

[@engineerprompt]

Thank you for your support!

[@ThisAbhinavL]

I'm getting this error TypeError: cannot pickle '_queue.SimpleQueue' object someone help

[@996hunter4]

and you my friend are a true hero!

[@GaganThakur]

Loved this, learned so much today. Thank you

[@anasputhawala8677]

Based on your workflow, if you ask a question it will end up re-uploading the document and re-processing the pdf document, text splitter, and all that stuff. Is there a way to prevent that from happening? It happens because streamlit reads python code, top-down every time a change is made on the page.

[@muhammadmustafa3611]

TypeError: cannot pickle '_thread.RLock' object

[@user-nz6rm1hf8p]

Thank you so much man this was of great help cheers!

[@asaibybike]

Great tutorial!! I was unable to get past typeerror: cannot pickle _thread.rlock object error so I removed the if and else statements and ran the model without pickling the VectorStore. Thanks again, keep up the good work!

[@kdbarchives]

Hey, can you please share the code? I need it for a project

[@artarioo]

@@kdbarchives # # embeddings store_name = pdf.name[:-4] st.write(f'{store_name}') embeddings = OpenAIEmbeddings() VectorStore = FAISS.from_texts(chunks, embedding=embeddings) essentially just get rid of if/else and the with statement

[@evgeni_logic]

@@kdbarchives , I created GitHub repo with the code: langchain_pdf_chat_app . Cannot post full link - apparently it's not allowed.

[@saifuddinsaifee8161]

Please make more videos of chain_type, as well as explore more configuration for OpenAI and other LLMs. Thanks❤

[@keertiattem4262]

Every time I use faiss, I am getting deserialization error. Please advise

[@NexTechNuggets]

Can you make it go through "somename" folder and use all pdf's there then ask it question from those files?

[@abinash01]

Great work! Thanks for sharing. Keep it up :)

[@pragmaticprogrammer3397]

Thanks a lot, though it was a long video i was able to complete and create a full working application. Any one who is reading the comment and yet to watch the full video, please go ahead lots of learning and its a complete tutorial. One question if you can answer will the langchain or open ai be storing the pdf in their servers. iam asking this question to understand if i can upload any confidential document

[@engineerprompt]

Thank you for the kind words and glad you found it useful. If you have confidential documents, I wouldn't recommend using OpenAI if you are not comfortable sharing. I would recommend to checkout my localGPT project. Everything runs locally on your own hardware. You will need to use a bigger model than the default model.

[@Jonathan-rm6kt]

What if you have a semi-structured markdown document that contains it's own semantic structure (like H1 headings), and want to chunk a document based on them? The langchain docs for the MarkdownTextSplitter method mention splitting into chunks for sentences and recombining.. but how?

[@user-kj5mn7yy3e]

Hey, I have doubt. In vector store you are storing embeddings, but When you upload another file and ask question on that document it will also use embeddings of previous one pdf because you are not deleting those data. So how to resolve that problem?

[@indrajitshah5357]

Great work and learned new things, I have tried along with this video, but at the below code block, started getting error, also I am unable to find input_documents when replicating the code, I have update the langchain package too, but its same error, any solution to fix it? AttributeError: module 'langchain' has no attribute 'verbose' langchain.verbose = False llm = OpenAI(temperature=0) chain = load_qa_chain(llm=llm, chain_type="stuff") with get_openai_callback() as cb: response = chain.run(input_documents= docs, question=query) print(cb) st.write(response)

[@engineerprompt]

Oh, yeah the langchain keeps making changes and seems like they have removed it. Will create an updated video on it.

[@ohp47]

Thanks for this wonderful video.

[@ticelsoful]

Hello teacher. Your videos are excellent. I'm getting an error when trying to insert the OPENAI_API_KEY char, and I'm not sure where to place it. Could you help me?

[@engineerprompt]

This specific version uses a .env file to store the OpenAI key. So create a file called ".env", make sure it's not a text file and the extension of the file is env. Then put your openai api key in there like this: OPENAI_API_KEY=YOUR_KEY Then run the above code, hope this helps. Alternatively, you can put your api key in the file itself if you are not sharing this with anyone. In this case, after your import all the packages add this line to your code: OPENAI_API_KEY="YOUR_KEY"

[@intellect5124]

Grt video from a genius man. Hats off. can you pls record a short video using google palm LLM and instructor embedding and also hosting of this applicaiton on VPS?

[@prazyraj1735]

I have this use-case where there are different types of documents. I can parse documents using document loaders using langchain. But, there are images also in these documents. I want to store them as metadata and if answer generated from a context chunk it show the image also. Please help.

[@MoreWardom]

i am getting a AttributeError: module 'openai' has no attribute 'error' error everytime tried everything but ddint work

[@meerulhassan2944]

upgrade your openai to latest version. this resolve the error for me

[@jraavaatu]

thanks for the tutorial!! but I had a huge storage problem when it comes to creating vector store. my SSD became rapidl full and I do not know from where to clean those files. Is there anybody that can help me?

[@riprap007]

excellent, thanks

[@sudarshanrajagopalan8552]

I am having a problem. Where I always get a response saying cannot answer questions based on the context provided. How do I fix this ??

[@engineerprompt]

change the system prompt. that will help.

[@Zoronoa01]

Thank you so much!

[@KinglhbCoding]

AttributeError: module 'langchain' has no attribute 'verbose' I copied your code, it doesn't work for me

[@ahmedebenhassine2828]

Do you any idea why when we send one question with the chunked embedding we gonna have: 2 requests text-embedding-ada-002-v2, 2 requests ? 1 request gpt-3.5-turbo-16k-0613, 1 request Why 2 requests of embedding not only one ?

[@notmadbong]

help, i am getting this error while uploading it AxiosError: Request failed with status code 403

[@xianggao445]

in the video, the constitution.pdf is uploaded. there were only 13 states back then. why 50 is returned as the answer? is the PDF really used as the data source when we do the query?

[@maxgadd]

thank sm bro your the best fr

[@Drmedquizz]

Thank You !

[@jameswilliamhanzell3292]

I was wondering actually and I know it's a dumb question but how do you make the file or folder that you've shown in the video in vscode?

[@joannezhu101]

i actually have another question - how to set vsc to point to the environment interpreter, it is now using global python, i want to use the localgpt python?

[@otimistarj]

ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`

[@vivekmehta4862]

that is a very beautiful walkthrough. Just one question, Can you pls tell how to change the language of conversation to some other language like hindi?

[@Officer-kd6]

Really an amazing video, glad you shared this with us!

[@tanmaysalwe1352]

Thankyou buddy.

[@nischalvasisth3153]

Problem with this approach I find out that Your Bot gonna reply on irrelvant question "Ask who is spiderman". So even after providing a prompt which clearly Instruct system Not to response If current context doesn't hold knowledge LLM gonna reply.

[@user-ms5kr3yf9h]

I tried this and it works really well, while testing the feature, i find that if i ask questions that require knowledge outside the pdf, the ai is still able to answer the question. This is suprising becuase the only knowledge source should be the pdf and nothing else right?

[@rajithkumar3424]

Very Informative tutorial .Thanks for educating us

[@engineerprompt]

Glad it was helpful!

[@AnsulSingh-nu1xd]

@rajithkumar3424 can u explain me ..... waiting for your reply

[@tlchannel2359]

I have a question, how about I ask him to summarize the pdf document I just sent , it can't do it because it only retrieves information , so how can I give him context here that I upload 1 pdf file and I want 1 larger request for the whole file instead of just retrieving it Any 1 small piece of information ? Pls help me!

[@engineerprompt]

There are specific summarization chains within langchain for that. The approach listed here will not work. Look into this: python.langchain.com/docs/use_cases/summarization

[@ReliableAIBooks]

It is giving this error, i have created .env file and entered correct open_api_key, but still working. Did not find openai_api_key, please add an environment variable `OPENAI_API_KEY` which contains it, or pass `openai_api_key` as a named parameter. (type=value_error)

[@AbdulBasit-jp9dz]

Did u fix the error?

[@JC-hu8rq]

hi i get an error : ValueError: Missing some input keys: {'input_documents'}. anyone know what this is?

[@ahmedebenhassine2828]

Thank you for the tuto, iquestion: waht is the model used here for open AI, and why when I ask "Could you extract the product name and return a JSON representation of the answer" the given answer is "No, I cannot extract the product name and return a JSON representation of the answer." ? Any help would be appreciated.

[@miladmirmoghtadaei5038]

Thanks a lot man. What version of Python are you using? I got this error as soon as I ran the `pickle` function, and after some looking around I found out that the latest version of langchain as of now doesn't match Python 3.11. So I should downgrade my Python and see what happens. The specific package it has an issue with in langchain is called `sqlalchemy`. Here is the error: "TypeError: Parameters to Protocol[...] must all be type variables or parameter specification variables." Thanks for your great videos again. Your videos are just so distinguished from the other channels talking about AL stuff. Keep it going.

[@engineerprompt]

I am running this on Python 3.10. Probably should pin all package versions in the requirements.txt file as these packages are updated regularly.

[@miladmirmoghtadaei5038]

@@engineerprompt downgraded to 3.9 and it worked. Thanks.

[@none-hr6zh]

I am getting billing issue . I didnot use any api before?how can we do without api keys?

[@vickvolpe5967]

ValidationError: 1 validation error for OpenAIEmbeddings __root__ Could not import openai python package. Please install it with `pip install openai`. (type=value_error) HOW TO RESOLVE?

[@Zoronoa01]

install openai by typing `pip install openai` into your terminal

[@amankumarsingh8692]

RateLimitError: You exceeded your current quota, please check your plan and billing details. How to fix this?

[@engineerprompt]

You used up all your openai credits. You will have to setup payment to openai

[@saichandujuluri-db1th]

i tried implementing it in react and flask but its giving threadlock error in flask. i don't know why

[@AbdulBasit-jp9dz]

Did u fix the error?

[@owmygawd12]

Thank you for this! How about a tutorial on making my own chat bot, with only the information I have provided it, then use it in my own app ?

[@himanshukrothwal7501]

great content but i'm having a error that is RecursiveCharacterTextSplitter is not importing from langchain.text_splitter any solution please

[@vijaysrinivasan2586]

Im facing rate limit error while using open ai api key without being used once.

[@rohitchauhan7159]

starting 2-3 days it's working properly but now I have run the same It shows an error with incorrect API but I am giving the right API.

[@ozzy1987mr]

excelente video, seria bueno uno pero utilizando los modelos libres, se que es mas simple todo con OPENAI pero considero que para este tipo de programas existen buenos modelos libres compatibles con langchain para lograr el mismo resultado a costo 0, me gustaria mucho un video de lo mismo con estas variantes y utilizando el procesador de cualquier tipo de documentos no solo PDF que tiene Landgchain. gran trabajo

[@engineerprompt]

Good point. More coming using the GUI :)

[@simarpreetsinghofficial]

@@engineerprompt This was probably one of the best video I saw on this topic. Would really love to see more videos on GUI.

[@engineerprompt]

@@simarpreetsinghofficial thank you, more to come :)

[@JavierAngelesMar]

Tengo la mism inquietud. Hay forma de hacerlo y que el costo de utilización sea cero?. Tu ya has probado algo? Y me causa curiosidad, te refieres a .doc, .txt y quizás más?? O par que querías usar más tipos de archivos ?

[@ozzy1987mr]

@@JavierAngelesMar langchain tiene compatibilidad para multibles formatos porque la cantidad de carga de archivos que yo sepa no esta limitada, no he probado la verdad pero solo cambia el llm de openai por alguno de los libres que esta en la documentacion y deberia de funcionar

[@saidinesh216]

Can you please help me with deploying this app into any cloud services but it throwing errors

[@priyanksrivastava834]

how can we restrict the output if it is not present in the document we provided have you made any tutorial on that

[@joser100]

Thanks for your great video, as usual. I noticed that you place the focus on treatment of a given file, about things like summarizing or QnA on the example file. Have you ever tried to: 1. Add multiple documents on a similar subject in order to, 2. Request a cohesive consolidation of information about that subject. In other words, to add, say 20 docs, all talking about Langchain (just to give an example) and containing overlaps in different sections, etc. To then, ask for a consolidated document on the subject that would aggregate points from the different documents?

[@engineerprompt]

That's an interesting application. I think it will be possible with careful prompt engineering. It is a good idea and I will explore it further for sure. Thank you.

[@LearnWithBahman]

what if we want to upload pdf , not user , use just come and ask question about pdf ? give pdf information to the model before user interact with it?

[@domasstepas967]

Great video, any advice on integrating images in the pdf and not just text when splitting them into chunks?

[@AbdulBasit-jp9dz]

Can u share ur code with me I m getting error

[@user-bo4ui2rv8l]

I want to translate the above 5mb pdf file at once with this library, so that all the pictures and the order of the paragraphs are preserved, does this library have this capability?

[@shashankshash59]

Great video man!! Can you help me on the changes that needs to be made if I don’t want to use historical data or documents please?

[@percysimpson1777]

I'm getting error "IndexError: list index out of range". I am using python version 3.11.3 . What is the python version you are using? Any solution for the error?

[@engineerprompt]

Make sure your path to the files is correct and you can read the files.

[@emresasmaz258]

After adding the embedding, I get the error below. ConfigError: duplicate validator function "langchain.llms.openai.BaseOpenAI.build_extra"; if this is intended, set `allow_reuse=True` How can I solve this?

[@saran2506]

Bro if the pdf file have images how can I access 😢 them please put video for this kind of LM bro.

[@aieducators]

thanks buddy🥺❤❤❤❤

[@FRANKWHITE1996]

thanks for sharing

[@rjayanth]

Would like to try to implement this on my laptop.. i am new to GCP or Azure.. Does this need Google Platform to implement?

[@abhijithjain5292]

ModuleNotFoundError: No module named 'langchain.chains.question_answering' not able to get rid of this error

[@trivenianand4169]

Real nice explaination, Could u help me out with the open ai api key? It only works if you have a paid openai account , any other alternative?

[@abusufyanvu]

is it possible to train it once time instead of uploading the pdf file everytime. because i want to develop a chatbot on a book so anyonce can guide me how can i train it once instead of uploading it manually everytime before questioning answering.

[@roseblack6089]

im use local ollama with pdf ,but models running was slowly.

[@paulp6752]

Great stuff, thank you for spending time recording this video! Question - if I would like to do the embeddings calculation and semantic search on my server, which model would you recommend?

[@engineerprompt]

Check out my localgpt project, there I am using llama2 for the llm part along with instructor embedding

[@paulp6752]

@@engineerprompt thanks! And you have earned yourself a subscriber!

[@muhammadqasim6524]

Link to the code is expired. Can you please update that.