GPT-4 & LangChain Tutorial: How to Chat With A 56-Page PDF Document (w/Pinecone)

[@chatwithdata]

00:03 PDF demo (56-page Legal PDF doc) 02:05 Visual overview of pdf chatbot architecture 06:56 Code walkthrough pt.1 11:10 Pinecone dashboard + setup 13:43 Code walkthrough pt.2

[@yeezythabest]

You should post this in the video description in order for KZbin to use it to chapterize your video

[@chatwithdata]

@@yeezythabest ohhh thanks for the tip

[@abhinavdhull1431]

can you make a tutorial on mac os,like step by step guide me follow please.I am a beginner in coding world

[@tangdavid3317]

@@chatwithdata - error Error: Cannot find module 'next/dist/server/future/route-modules/route-module.js' please help

[@erniea5843]

That design chart overview was excellent! That’s what is usually missing from most tech tutorials.

[@colbydotcom]

For real! I'm a visual learner, and if I can't visualize something in my head then it's hard for me to understand what's going on. I've watched so many videos from others that are similar to this and I got so much more out of this one just from the opening 5 minutes than I did from the hours and hours of other videos I watched on this topic.

[@benohanlon]

What tool is it?

[@ESGamingCentral]

@@benohanlon any updates?, I want to know.

[@sebastiantillmanns7747]

@@ESGamingCentral I guess it is Excalidraw

[@benohanlon]

@Oscar Llerena if you wanted to answer the question, without answering the question, then 10/10.

[@a.m2053]

As a software engineer i’m very happy to say: one of the best and coincise video ever saw. Thanks. Really appreciate 🎉😊

[@chatwithdata]

Thank you

[@JasonMelanconEsq]

Amazing job! Even as a non-coding attorney, I was able to follow your instruction (with a little help from GPT) and install the code from Github on my Mac, and it worked flawlessly! Occasionally, I get some incorrect or incomplete answers from the chatbox, but it's amazing nonetheless. I sent you a message through Twitter regarding consulting with my developers. Looking forward to hearing from you. : ) Thanks again!👏👏👏

[@soumojitguhamajumder3143]

Do you feel an app that automatically does this for you would have value ? App would contain all these cases and you'd probably only have to ask questions. Also, for example if you search for cases,it would give out similar cases,and say score how similar each of these cases are ? Also, for example if another feature could predict a new case's possible result and potential areas that could a lawyer focus on in terms of one's argument, would this help as well ?

[@RezaYassi]

Yes

[@sissyrecovery]

As a programmer who worked with attorneys I wish more attorneys were like you lol

[@SwingingInTheHood]

I am on the Weaviate Slack forum and I see newbie questions all the time from people struggling to understand the relationship between embeddings (vectorization) and chat completion. I've been referring them to this video for an overview of the process. Good work!

[@nobody796s]

is it possible to use similar method here, to review a long code instead of a pdf doc?

[@unconv]

Great explanation! So can this chatbot only find the answer to a question if it is included in a single chunk of the original data? So if some question requires information from page 2 and page 30 it can't answer that question correctly?

[@resistance_tn]

This is an awesome tutorial ! Hands-on especially for the clear and very well schematised system design :) Subscribed and will wait for upcoming videos !

[@codespace]

Can I upload 1TB of docs/pdf to openai and do prompt chat with it?

[@orangtimur6812]

you can, but you will need to pay pinecone and open ai api for that

[@chatwithdata]

If you're running into errors on "ingesting" the data due "Error calling upsert", I posted a list of major culprits to troubleshoot here: github.com/mayooear/gpt4-pdf-chatbot-langchain/issues/4 It's likely you haven't properly configured your Pinecone settings or your insertions above Pinecone's size and insertion limits per upsert request.

[@kaifirschau]

Amazing video, thank you very much! I am trying to get it running with an Azure OpenAI model but get errors there all the time, do you happen to know what has to be changed for that to work?

[@ItsNikoSlater]

How can we manipulate the response length to generate longer responses?

[@andrewlaery]

Love to see a more detailed strip through. Awesome vid!

[@chatwithdata]

I'm covering that here in the upcoming workshop: tinyurl.com/zcks7jsk

[@aayushchrs]

Can it ingest tabular data from the PDFs accurately? Superb presentation btw

[@김창민-j6b]

You taught me what I needed most. thank you my messiah!

[@MedoHamdani]

Is it possible to submit more than one book and then chat with all of them at once?

[@abedaarabi3014]

Thank you- it great job, but may I ask how the chatbot bring the url links since, you are searching within the pdf. GPT 4 browsing the internet ?

[@Kage1128]

can you use this for math books as well?

[@martinharris4416]

Depend on how good the OCR is. Essentially whatever you put id converted to text

[@chatwithdata]

Yes, it's a similar approach, you'd just need to more prompt engineering to prime GPT to be an expert "mathematician" and then also provide examples of problems and solutions.

[@DGeneral101]

The upcoming Mathematica plugin integration could probably help as well

[@adnuzzolillo]

Thank you so much for the tutorial. It was very well explained. I appreciate you sharing your knowledge with the community!

[@kishanranawat2481]

What I was looking for. Everything explained so well. Way better than some paid courses.

[@internproj]

you have implemented it,if yes can you give your repo link,i am getting pinecone ingest error.even after following the video.

[@techknowledge5069]

yes same@@internproj

[@beralc]

Hi, Great app. I have 2 questions. Apart from gpt3.5 turbo and gpt 4 can we use davinci? I'd like for the AI to reply in spanish, and I don't have access to gpt 4 yet. And.. is there a way not to limit the answers to the pdf and use Chat GPT as well?

[@parthbhattacharya9810]

Is there a way to upsert embeddings without using fromDocuments? As pinecone free tier doesn't support namespaces anymore

[@AmineR31]

Hey can you tell me whats your microphone setup? It sounds amazing

[@David-rz4vc]

My main question is does open ai reads directly from the store or is it sending all that data up to openAI for every question?

[@yibuh]

Great video! I have two questions: is there any way to prevent the chatbot from coming up with an answer? and is there any structure the input data must follow for better results? My chatbot some times sends the wrong answer or completely makes up an answer.

[@manuazoulay323]

Hey tou have solve the problem ?😢

[@b1chler]

why did you put 1536 into the dimensions field in the pinecone dashboard? Where does this number come from? thanks

[@ESGamingCentral]

What software was used to create that graphic organizer?

[@nikow1060]

Is it possible to use chroma instead of Pinecone ... what would be the downsides of chroma ...? I am asking because chroma is open source and I can store the embeddings on premise as well as in the cloud. Furthermore chroma is free.

[@vishalahuja2502]

Is the chat history storing the question and the answers, or just the questions?

[@KleiAliaj]

Thats very nice. I wanted to ask you, How it is so fast in the video. I run it locally and it is like 10-15 seconds slow

[@fernandes1431]

❤ this is the first time I've been able to comprehend the whole chatbot process. Please please can you share this diagram flow chart. Its honestly the best visually and i woild love to be able to play around with it.

[@dawn_of_Artificial_Intellect]

Great App! I see the potential of how powerful it can be. Your video is excellent, I got the script up and running and I am a beginner in learning code. Thanks for the tutorial

[@PelletsEaseFrance]

Is this working with pictures?

[@zhangxiang18]

Hi thanks for your sharing. I came across one error message is creating vector store... error [Error: PineconeClient: Project name not set. Call init() first.] d:\chatPDF\gpt4-pdf-chatbot-langchain\scripts\ingest-data.ts:43 throw new Error('Failed to ingest your data'); but I have set my index name at Pinecone as "ChatPDF001" and set this value in the config file. Any suggestion here? thanks.

[@chatwithdata]

Try and upgrade to the latest node version.

[@sujitmourya]

Hi, Is there a solution to replace pinecone with cosmodb?

[@matmorales2011]

I didn't find the code line where you find and output the link of the location of the question. Can you help me with that? thanks

[@brandonfrancis1317]

When will enrollment open up? Would love to get used to using these tools.

[@suyingluo]

May I ask how you jumped to the interface of 6.56? Please ---From a junior high school student who is humble in learning

[@suyingluo]

I have watched this video no less than ten times but still can't figure it out

[@Austin-z3c]

Great vid. Setup complete. How did you get it on your localhost to test? BTW, anyone ever tell you that you sound like the UFC Champ Adesanya :)

[@andydataguy]

Great video! Would love to see this done with a notion database using langchain. Perhaps outputting long text into additional motion pages that are then added to pinecone so that it creates a veritable limitless memory of previous responses, answers, and structured content for the usecase

[@maorre6809]

I need to use version 4 of ChatGPT and upload tens of thousands of documents If I upload documents in version 3.5, can I use them in version 4 without paying?

[@kashifsohail]

Great work!. Few questions, if you can reply plz. 1) how can ask it to look answers in particular file only, rather than all ? 2) what about adding files later, do we need to ingest all, again?. Thanks

[@harryzachariou1]

Great video, how would you gather the text from a PDF that doesn't encode the text, so its essentially a picture as PDF, would you have to implement some OCR system to extract the text from the PDF to then encode the embeddings? Essentially I have a PDF document that when parsed the text output is ` pageContent: ' ',`

[@vlod]

is it possible to use gpt 3.5 instead?

[@Mountaineer170]

With the new plugin announced, will this be relevant in a few months?

[@chatwithdata]

Yes, you won't have control over customizing your prompts and there will be upload limits. Using this open source repo, you can upload PDF docs without outages, limits, privacy concerns etc.

[@manuazoulay323]

Its available ? you have a link ?

[@kingarthur0407]

I've written a prompt for GPT-4 that I use with chatGPT in Macromancy formatting to transform it into a legal assistant. and the results have been stellar. Is it possible to encode this prompt into the system you describe so that the bot operates with it in mind?

[@msat7755]

What if you want to use contious data. For intance I want to create a model that uses embeddings from a csv. This csv file needs updated daily so that it has the most relevant information. How would I update the embeddings with new information from a csv without having create a new embedding everytime I had to do so?

[@chatwithdata]

I'll do a video on this soon, but you'd "upsert" the vectorbase only with the new embeddings from the csv file without re-running the ingesting from scratch. Where it gets tricky is when you want to "update" a specific part of your csv file.

[@SageLewis]

I got this running and am SO excited to keep working on it. Thank you.

[@argniests5357]

great video, thanks. I am subbing now. I was wondering, do you know what size limits (number of pages) that your code could handle when injesting pdf files? How long did it take you to figure all this out???

[@xCheWy99]

Hi I am a beginner and just wanted to experiment and play with the program. I followed most steps and I have reached the npm run dev step and opened local3000 to the chat bot, when I test prompted "hello" in the chatbox , it returned this error: PineconeClient: Error calling query: PineconeError: The requested feature 'Namespaces' is not supported by the current index type 'Starter'.

[@Jordy-t8y]

Hey man, great video! This is awesome, great content please keep going!! I do have one question. From the code I see that we are passing history to the chain.call() function but to me it doesn't seem like history is working? Because when I ask it something, and afterwards I ask could you translate that to french, is says it doesn't have any context of what to translate to french (?) but the question I asked before is in the history I'm sending it. How can I make this chatbot remember what was said?

[@josephhart7771]

I must say, you are an amazing person for doing this. So helpful and so explanatory. Id love to build with you

[@aimeeaidanu]

This is cool! Can it work with chat-gpt model or gpt--3 mode? I don't have gpt4 api access :)

[@farhanhubble]

Thanks for sharing the code and demo. Nitpicking a bit but in the video, under the "Visual Overview" section the "Ingestion pipeline" part of the diagram does not show use of OpenAI Embeddings.

[@LeiferMendez]

Awesome Job 🎉🎉 I will be translates this material to Spanish and post on Channel. Obviously give you all credits

[@jonathanchow7687]

Was replicating this project and encounted this error after running npm run ingest: error TypeError: Cannot read properties of undefined (reading 'text'). does anyone know a fix for this?`

[@jusbob89]

Is there any way to weight or improve the responses given? i.e. selecting this was useful/correct and have that feedback into the model?

[@chatwithdata]

Unless it's verifiable data from an external source the LLM can utilize an API to reach, it will need feedback from the user. Then you can optimize your prompts based on this.

[@moreshk]

managed to get this running locally, but on vercel I am getting a 504 SyntaxError: Unexpected token 'A', "An error o"... is not valid JSON. Any ideas?

[@ivanrododendro]

Hi, very interesting and well done, thank you. Do you think it's possible to achieve similar results with Open Source components ?

[@Jjjabes]

Really good explanation and diagram to get your head around the process!

[@johnzhu5735]

How does full document summary work when you are working with pinecone nearest neighbor document snippets?

[@ScottPendergrast]

How do we think this would do with a PDF of a long fiction novel? TY!

[@SanjeevKumar-hj1fb]

Great video. I was curious if we can follow this implementation if the data is kind of semi structured like Project Management data(both text and numerical fields). Does this approach work well only in case of Text data ? Was curious if embeddings work well for mixed data(numerical/text). How would query work if it involves multiple filters on text and numerical fields?

[@TelstarB]

How do you set the dimensions in pinecone? is just a sufficient number for a large pdf or it came from a calculation? Great work btw!

[@chatwithdata]

The dimensions are 1536 due to openai's embeddings function, but isn't related the size of your pdf, it's related to the number representations of your docs. If you don't set this figure, pinecone won't let you insert into the database. You set this when you created an "index". docs.pinecone.io/docs/choosing-index-type-and-size

[@BecomeAWebDev]

@@chatwithdata ..... now I'll subscribe to your channel. Important detail. thanks

[@mycloudvip]

Great Content! Which TS version is it built on top of? Thanks!

[@chatwithdata]

I believe it's the latest one. You can check out the github code in descriptions.

[@santiagobarchetta5401]

Your work is so valuable. Thanks a lot for this info and the code!

[@AiIntegrator]

Thank you for the Awesome tutorial, Please I want to use the ingest script as a function in the index file in order to upload a file and use it instead of passing a static path. Thanks in advance I already tried, but it's a litle bit complex with next js and typescript. Please if you can help

[@hichamzaidi1402]

I am stuck in the same problem 🙂

[@chatwithdata]

You want to upload the file from the frontend and run the ingest script automatically?

[@AiIntegrator]

@@chatwithdata Yes that's right, please if you can help

[@AiIntegrator]

@@chatwithdata Yes that's right, please if you can help

[@kirillsavik6615]

hi, thanks a lot for this video. Can you please tell me how to implement the output of messages not in English? e.g. in French

[@Steve-js7bp]

For the textsplitting part, do you recommend to tokenise the text first?

[@chatwithdata]

The textsplitter function in the repo does that all for you under the hood.

[@cuentadeyoutube5903]

So, the answer needs to be quite localized on a chunk of the document… right?

[@benohanlon]

What's the tool to do this discussion?

[@GameDownloader112]

Will these work on 1000 page pdf?

[@depenz]

Whats the difference between the ”standalone question” and the original question, do they differ a lot?

[@SergioMiramontes-tp1ed]

can it return images if you ask to the chatbot?

[@waelabou946]

Great Work! well done i have a question : do we load the data every time to the model or it is saved? i mean each time we need to run we have to do the embdeing to chatGPT 3.5 or 4? cost wise for large data this will be costly..

[@nisubhatt8212]

I have the same question…

[@VIPHDD]

‚Embeddings Ada’ Usage is priced per input token, at a rate of $0.0004 per 1000 tokens, or about ~3,000 pages per US dollar (assuming ~800 tokens per page): also you need a workaround for massive pdf’s as it’s not able to handle so many chunks (you can find a explanation on the GitHub’s issue page) .. so in my opinion for most use cases its not that costly.. Also the output of GPT is likely not that long so you just pay like 0,002 / 1000 tokens which is also pretty good

[@chatwithdata]

No, once you've made the embeddings it's stored on the vectorbase in permanent state and you can start running queries against it without re-running 'ingest' again. You can also backup and download your embeddings locally.

[@gustavomonteiro2087]

Thanks so much for the code and the video. Any preference between pinecone and supabase pgvector?

[@chatwithdata]

For smaller projects where you're already using supabase for auth etc., then the difference isn't much. But for large scale production-ready projects, Pinecone is a more specialized option.

[@FarooqKaiser-ca]

Awsome tutorial 🎉, I wanted to create a similar project. This will give me the head start on this project. ❤

[@shifraznazar1636]

love the audio quality

[@GyroO7]

How can I make it use links like you did instead of pdf?

[@tangdavid3317]

I have an error: - error Error: Cannot find module 'next/dist/server/future/route-modules/route-module.js'

[@bugushi]

Could using embeddings potentially lead to information leakage?

[@KOSTANNA1]

Really good video! It would be great to have a deep dive session around it.

[@chatwithdata]

I'm covering this in the upcoming workshop: tinyurl.com/zcks7jsk

[@maplesyrup272]

Is one PDF considered one token? Or does the chat analyse each word in the document as a token? This could make a world of difference when it comes to Api modelling.

[@chatwithdata]

The PDF is converted to numbers (embeddings). It is these embeddings that are analyzed and used to retrieve the relevant docs.

[@MrPsycic007]

Can we do something similar but with Opensource models incase the document is having some privacy realted stuff ?

[@mc-qf4bp]

Are you a contributor to Langchain? Your videos are awesome, but the langchain typescript sdk can be a bit buggy as of right now. Which is fair because it's still not in a stable 1.0 release yet, and is in the early stages, but it makes it difficult to use in production.

[@RedCloudServices]

what happens with an excel file or if there is a table of numbers inside the PDF? can langchain w gpt be used to ask computational questions? can you make a video using Excel into vectors?

[@chatwithdata]

Yes it can. I have an upcoming video addressing this.

[@Todestuete]

Why is the LLM needed to create the standalone question from the chat history? Isn't it just a concatenated string?

[@MaxXFalcon]

This looks helpful. But can i use a different language model, for example any other open source?

[@MaxXFalcon]

For those who also wonder - yes, you can, at least the bing ai said that

[@chatwithdata]

Yes, just swap out in "langchain/llms"

[@buddy-red]

Awesome open source sharing! I have make it working, but what if we want to add more Docs? like thousands of PDF files，ingest command still working good or how to set the files path(const filePath = 'docs/filename.pdf';) in the code? Thank you!

[@zhangke1350]

I have the same question as Buddy. Maybe just add the docs that hasn't been indexed?

[@chatwithdata]

Hey thanks. Yes as I mentioned in the video, it will take quite a bit of time to explain how that would work + Q&A around, so based on viewers requests I'll cover that in the upcoming step-by-step chatbot program here: tinyurl.com/37v3k2fz

[@DemoBailey]

Your ingest-data.ts file is different than that of the git repository. Is this pay to play?. I'm getting this from my local response. I'm sorry, but the provided context does not include information about the name, date, or summary of the legal document.

[@RezaYassi]

Is there a step by step by step tutorial on how to install this on a mac?

[@AdmineTube]

Great work, waiting for more videos and tutorials.

[@ogamaniuk]

As a Ukrainian I have to say you picked a good example pdf! ❤

[@gr8tbigtreehugger]

Many thanks for this helpful tutorial! Since there is only one PDF, do we really need a pinecone database? Couldn't we just generate a JSON data file?

[@TechnoZany]

the pinecone database also helps with the retrieval (finding the most relevant docs for the query). Without that, we will have to check every document in the database for a query.

[@thisnext6126]

Are you able to make it actually have speech in the responses vs just text?

[@chatwithdata]

I can try that next

[@mpfmax0]

I wonder if this is the way the new AskYourPDF plug-in works? It clearly can take PDF documents longer than the token window, I guess because it creates these chunks and embeddings, the issue I have with it is that it messes up when searching for the right embeddings, it does not really retrieve the correct info I want. For example, I give it a pdf of a research paper and I want a summary of just the introduction, which is just 2 pages long, and it gives me a mixture of information from around the entire document, not just the introduction. Because I'm guessing it retrieves chunks from all around the document based on similitude to my prompt. It works better when I ask for specific concepts, but when I want it to be more sequential and thorough, like summarize each subsection of a section of the document, it skips some or grabs some stuff from another section, probably because those bits and pieces are on different chunks that still have close relation to my prompt.

Loved your video, certainly learnt few things out of there, Im not programmer, github, it's something completly new for me, it's clear the explanation of the code and how things basically work, amazing, sadly I didn't manage to make anything happen. Probably the only thing I made happen was to clone the repo send it to visual studio and that's it. The instructions pnpm install and so on, dont know where to start. If there's any video or explanation how to make this work so I can move forward I would appreciate it. Thanks anyway for the great tips.

[@suyingluo]

so do i

[@resistance_tn]

@Chat with data; I'm curious to understand how the flow (specifically in terms of grabbing the appropriate context/embeddings that are related to the question prompt) when the related context is sparse across the original document (meaning they're spread on a lot of vectors/embeddings) ?

[@chatwithdata]

You mean the related context belongs to more than one chunk/vector? There are several options, but one is to pass the top say 3 returned source documents (by vectorSimilarityScore) as the context, then tweak the prompt accordingly.

[@resistance_tn]

@@chatwithdata yep that's what I meant thank you for the idea ! I guess that would be something to play with in the parameters right ?

[@chatwithdata]

@@resistance_tn Yes. So by default in this codebase 'returnsourcedocuments' is set to true. So you can play around with the 'k' value for no of returned source docs and also the final prompt sent to the LLM that contains the context.

[@resistance_tn]

@@chatwithdata cool thank you very much ! Looking forward for the next content :)

[@SebEric-vh7zu]

hey, great video! does this use conversational memory or a conversational agent? so you can feel like youre actually talking with the ai rather than just Q&A?

[@chatwithdata]

Hey, this uses a custom chatVectorDBQAChain hwchase17.github.io/langchainjs/docs/modules/indexes/vector_stores/pinecone The conversational agent is more applicable when you need to access external tools as part of your response.

[@manuazoulay323]

Did you manage to put a conversational agent and a pdf chatbot?

[@santiagocalvo]

I don't understand how the server works, I'm trying to replicate this using react but can't understand how the chat component "knows" about the api? Is this a next feature that I don't understand? Also thanks for the video!! Amazing content!