GPT-4 Tutorial: How to Chat With Multiple PDF Files (~1000 pages of Tesla's 10-K Annual Reports)

[@chatwithdata]

Timestamps: 01:02 PDF demo (analysis of 1000-pages of annual reports) 06:01 Visual overview of the multiple pdf chatbot architecture 17:40 Code walkthrough pt.1 25:15 Pinecone dashboard 28:30 Code walkthrough pt.2

[@johnpaulpayopay1074]

Instead of using an OpenAI API, can use local LLMs such as llama, alpaca, vicuna, koala

[@MidoBroski]

The GitHub repository in the description does not include this multi pdf bot? Could we get the source code?

[@chineseoutlet]

@@johnpaulpayopay1074 Agreed. I tried to use Vicuna7B to replace the OpenAI(my machine don't run 13B). It works OK, except it can't do query on the entire PDF. Like when I ask it to summarise the whole book, it just say "I don't know" trying to find ways to fix it. Any one here know how to fix it, please let me know.

[@tangdavid3317]

Do you know how to fix this? - error Error: Cannot find module 'next/dist/server/future/route-modules/route-module.js'

[@TheRonellCross]

This tutorial is legit the best. Custom Chatbots are going to be a huge biz op.

[@byte_easel]

The excalidraw documents are invaluable to the presentation and really help visualize the flow of data between the pinecone database and langchain is used to make it all work. Thanks a lot for those diagrams, your efforts are not unrecognized. The possibilities of this are literally endless

[@ReflectionOcean]

- Convert large PDFs into text for processing (timestamp: 6:36). - Split text into chunks compatible with OpenAI's context window (timestamp: 6:59). - Create embeddings from chunks and store them in a vector database (timestamp: 7:14). - Query the database dynamically using namespaces to extract information from specific years (timestamp: 11:03). - Use GPT-4 to parse and understand complex financial questions across multiple documents (timestamp: 17:19). - Leverage Langchain and Pinecone services to structure and query data efficiently (timestamp: 18:24).

[@rhythmgaidhani2149]

best video i found for my personal project on the internet

[@Grahfx]

I don't think this is the right approach. It doesn't know enough context on the whole document. When you hit a prompt, he will match a chunk to your prompt, but what if the answer is contained in the context of let say 200 pages ? You could make something clever. You could aggregate chunk togethers asking the LLM to summarize and group them in "meta chunks", you could repeat the process until all years are contained into a single max limit tokens batch. Then, with the meta Data, you'll be able to perform a much more powerful search over the corpus, providing much more context to your LLM with different level of aggregation.

[@user-jc3ys1yc2n]

Something like that already exists and is called a “recursive summarizer”. The hard part is figuring out how to actually perform search over the recursively summarized documents without invoking the GPT api an unreasonable amount of times except during preprocessing. I am currently doing research in the area of passage retrieval and would love to know your thoughts on this and possibly experiment with some of your ideas.

[@Grahfx]

​@@user-jc3ys1yc2n Well, I had this idea while having my morning coffee, but I haven't conducted any deep research to support it. However, as you mentioned, there may be potential issues that arise. Have you considered using GPT4 to help solve this problem? Personally, I anticipate having to address this issue myself because I plan to summarize big HR data.

[@JanBadertscher]

you lose more and more information by recursive summarization. better to use embeddings, so the LLM knows the whole context of your documents, no matter how many...

[@Grahfx]

@@JanBadertscher No, of course, it doesn't have complete knowledge, that's the main issue. It only has partial knowledge based on a vector match. This is why it provides inadequate responses when the prompt is not very specific. It is incapable of extracting critical information that comes from reading the entire context. A human expert is currently better to do this.

[@chatwithdata]

You can control the `k` returned source documents up to 1000 if you want to via Pinecone + gpt-4 context is 8k, so you can cover a lot of context across the doc.

[@labsanta]

1. What is the tutorial about? - The tutorial is about how to chat with multiple massive PDF documents across multiple files, specifically Tesla's 10-K Annual Reports. 2. How many pages of PDFs are involved in this tutorial? - The tutorial involves around a thousand pages of PDFs from the 2020, 2021, and 2022 annual reports of Tesla. 3. How does the chatbot work when searching for specific information in the PDFs? - The chatbot is able to cross-check and provide a reference page when answering a specific question about the risk factors or financial performance of Tesla over the past three years. 4. Can the chatbot analyze multiple PDF files simultaneously? - Yes, the chatbot is able to analyze and answer questions about multiple PDF files from different years of Tesla's annual reports. 5. Is the tutorial code available for public release? - The code for the tutorial is not currently available for public release as it is still being tested and may be buggy. 1. What is the process of converting PDF documents into number representations? - The PDF docs are converted into text, as PDF is binary and needs to be in text format. - The text is split into chunks to fit into Open AI's context window. - Open AI creates embeddings, which are number representations of the text. 2. What is a vector store? - A vector store is a database that contains number representations of documents in different categories or spaces. - The vector store can also store the text of the documents and relevant metadata. 3. How does GPT-4 retrieve information from the vector store? - The question is converted into numbers and a specific namespace or box is specified to retrieve relevant documents. - The relevant documents are combined with the question and GPT-4 looks at the context to provide a prompt response. 4. What is the challenge of analyzing information across multiple namespaces or years? - Extracting the namespace from the question is required to search for information in the relevant namespaces. - The model needs to dynamically determine which year or namespace the user is referring to. 5. How can GPT-4 assist in extracting the namespace from a question? - GPT-4 can be used to extract the namespace from the question and dynamically determine which year or namespace the user is referring to. - This allows for analyzing information across multiple years and namespaces. 1. What is the purpose of the script called "ingest data"? - The script called "ingest data" is used to load each PDF report in the reports folder into text. 2. How does the dynamic context work in the system being described? - The dynamic context specifies the namespaces to look at for relevant documents. - When there is a question, the system reverts the question to embeddings and checks the specified namespaces for relevant documents. - It retrieves the relevant documents for each namespace and then proceeds with the usual procedure. 3. What is the website called "secant Alpha" used for? - The website called "secant Alpha" is used by many investors. - It provides information on revenue growth. 4. What happened when the speaker asked the system about Tesla's estimated revenue growth for 2022? - The system searched for and found the relevant documents from the 2022 namespace. - The system calculated the estimated revenue growth for Tesla based on the consolidated statement of operations. - The result was surprising and unexpected. 5. What is the output of the "ingest data" script? - The output of the "ingest data" script is a JSON file containing information on each PDF report in the reports folder. - The file includes information on the year of the report and the name of the company. 1. What is the purpose of page numbers and references in the UI of the project? - They allow users to easily locate and reference specific pages and original sources. 2. How are the PDFs translated into text in the project? - The PDFs are ingested and split into different categories and chunks of 1000-1200 characters, with each chunk assigned a namespace. 3. What is the purpose of Pinecone in the project? - Pinecone is used as a database to store the chunks of text and metadata, converted into embeddings (vectors) for similarity calculations. 4. What are some limitations of Pinecone, and how are they overcome in the project? - Pinecone has limits on the number of vectors that can be inserted at once, so chunks are split into smaller groups (e.g. 50). The API keys, environment variable, index name, and number of dimensions must also be specified correctly. 5. What is the purpose of the dimensions in the embeddings? - The dimensions represent different spots in an array of vectors, with each dimension containing a number representing a specific aspect of the text or metadata being analyzed. In this project, OpenAI creates 1536 dimensions for each embedding. 1. What has been trending on GitHub for the past couple of days? - Answer: The transcript does not provide a clear answer to this question. 2. What are the indexes and namespaces in the code? - Answer: The indexes and namespaces are components used in the code to retrieve information from different sources. 3. How can one explore what the vectors look like in the code? - Answer: One can click "fetch" to see what the vectors look like, which includes the namespace, ID, and decimal number representations of the text. 4. What should one do to ensure successful ingestion in the code? - Answer: One should ensure that their config has matching namespaces as in the dashboard, set environment variables correctly, and avoid tampering with versions to avoid breaking changes. 5. What is the purpose of the second phase in the code? - Answer: The purpose of the second phase is to chat and retrieve information dynamically by specifying the namespace to retrieve information from. 1. What is the custom QA chain and what does it do? - The custom QA chain is a tool created by the speaker. - It takes the model, index, and namespace to effectively set the stage for the Standalone question and retrieve specific name spaces and relevant documents to provide a response. - The custom QA chain is just one of three chains available in Lang chain, which includes chat Vector DBQ a chain and Vector DBQ a chain. 2. What is the purpose of the chat history in the code? - The chat history is set to nothing and is not directly relevant to the custom QA chain. - It is used in the API implementation of the same logic seen in main.ts. 3. How many files were tested with the custom QA chain, and what were the results? - The speaker was able to read three different files with over close to a thousand pages of in-depth financial analysis. - Gpt4 was able to analyze all three years and provide decent analysis. 4. What is the purpose of the front end in the repo? - The front end in the repo is already available for use. - It is an adaptation of the custom QA chain tool. - The speaker is experimenting with talking across different PDFs. 5. What is the compound annual growth rate for Revenue over the past three years? - The transcript does not provide an answer to this question. 2. What kind of questions were asked about Tesla's revenue? - The questions asked were about growth potential, profitability, and risk factors management. 3. What is the K1 ? - K1 is the number of reference documents that are returned per PDF when using GPT to analyze data. 4. What kind of future changes will the repository make? - The repository will add features for analyzing multiple PDF files in the future. 5. Where can you find more information about GPT and its applications? - The speaker suggests checking out their workshops and signing up for the waitlist in the description section for more in-depth step-by-step details.

[@13bonewacker]

❤

[@tslg1988]

Thank you for your great videos, I really like them. One UX improvement comment: Try serving the PDFs as static content with your web application, then when you append #page=42 to the PDF's link, a user can go directly to that page in their browser. Very easy to implement and your users will save a copy and paste and 3 clicks.

[@chatwithdata]

Nice idea, thanks.

[@clockworkOMNI]

Thank you for this contribution to the common good. You are the man.

[@harischsood5479]

so interesting. as a non -techie trying to implement this is wild but fascinating

[@cigir2023]

Thank you bro... Your are a F*** genius!!!.. This is one of those things that will radically chage the world in a very short time

[@satyaschannel4391]

Hey, the tutorial is one of teh best of its king in this segment. Can please share the repo link to this multi pdf one. Its very useful for others to work on.

[@LimabeanStudios]

I work in an academic research setting researching an extremely niche subject in material science and these kind of tools are what I have been waiting for. I can't wait for us to be able to upload the papers and books that contain all current available knowledge on the topic and be able to interact with that knowledge pool.

[@PizzaLord]

I already built this tool. If you want to be a tester, let me know.

[@bendrybrough4362]

@@PizzaLord I would like to.

[@LimabeanStudios]

@@PizzaLord I would certainly like to hear more

[@ahmeda8042]

@@PizzaLord I would like to.

[@tylerd6962]

​@@PizzaLord how do I become a tester

[@JesusSalazar-kv7mn]

🎯 Key Takeaways for quick navigation: 00:00 📑 *This video discusses how to analyze multiple massive PDF documents, like Tesla's annual reports for different years, using AI.* 02:36 🧠 *The video demonstrates asking analytical questions about multiple PDFs simultaneously and extracting insights from them.* 03:21 💼 *It showcases asking technical questions and comparing data from different years in PDF reports.* 06:36 📊 *The video explains the architecture, including converting PDFs to text, splitting into chunks, and using PineCon for storage.* 11:03 📈 *It discusses how to dynamically extract namespaces from questions to query specific PDFs and retrieve relevant information across multiple years.* 35:52 🤖 *The video demonstrates the process of extracting namespaces and mapping them to specific years in PDF documents, enabling targeted queries across multiple files.* 37:58 📊 *The presenter showcases the ability to analyze compound annual growth rates across three years of financial reports using GPT-4.* 39:49 💰 *GPT-4 accurately responds to questions about growth potential and profitability based on the past three years' annual reports.* 41:03 📈 *The video outlines the architecture for using GPT-4 in applications involving multiple PDF files and suggests adjusting parameters for improved accuracy.* 42:36 🧰 *The presenter encourages viewers to re-watch the video for a better understanding and mentions upcoming workshops for more detailed guidance.* Made with HARPA AI

[@MidoBroski]

The GitHub repository in the description does not include this multi pdf bot? Could we get the source code?

[@JohnnyLonghorn]

Thanks for this, your vids are some of the best on YT now for this type of thing as right now in this domain there is only two words for all the content out there - information overload!!! I am still going through your prev video on PDF ingestion and processing, thank you for sharing your invaluable information in a style that makes it understandable.

[@calebperkins8776]

Why not use Embeddings as the documents title so that instead using Gpt to do a search of the exact name space, it could do a semantic search of name spaces similar to the initial prompt, thus finding things relevant more effectively. Thoughts?

[@davidaktary]

When will you release the code from this walk through (not just the "base template", but the actual code here that uses namespaces)? Also, do you know of any way to prioritize what source it uses? For example, if we have 3 PDFs for 2021, 2022, and 2023, and each declares a new color of the year, and I simply ask "what is the color of the year?" how can I ensure it gives me the most recent answer?

[@suniyamokipallo2571]

would be great if u could build this full tutorial in just python!

[@JasonMelanconEsq]

Can you please show how the front end connects and how to build it? This is definitely the missing piece of the puzzle for me. Thanks for the videos. They're great!

[@vetonrushiti19]

Hi, thank you for this great architecture, but I wanted to ask if there is a tokenization phase that happens here?

[@DaveShap]

Great work! Now whenever people ask me how to do this I will just point at this video :D Cheers, thanks for shareing

[@rugerdie4054]

Brother...! This is what I am talking about! My sellers are looking for this kind of capability for research as well as for searching our internal content management system.

[@matte3333]

Thanks for this. It would be helpful to see your CustomQAChain Class. How do I tell Pinecone to only search through specific namespaces?

[@markoguru]

Really nice work, thanks! How about loading files which are located remotely, like S3 bucket?

[@MehdiAllahyari]

Great video. But the problem with your solution is you have hit the OpenAI api at least 5 times and it would make it costly and not scalable. Other than that, a good project.

[@chatwithdata]

Hmm where are you getting 5 from? The retrieval is from Pinecone, which is very cheap.

[@duhai1836]

You are great at teaching the entire process! Please continue this series :) Thank you!

[@emmanuelkolawole6720]

Please use chromadb not pinecone. So we can reduce cost significantly. Also can you use Vicuna AI model not GPT 4?

[@ADHDOCD]

Woah! 😱These diagrams make it so much easier to understand concepts better than other KZbin videos! Thanks for spending so much time and effort!

[@Jingizz]

yes its so helpful to understand these concepts

[@astroid-ws4py]

Cool stuff, It is just unbelievable how much depth and breadth all the computing field has become, Too stretched out and too spread, Too much subjects for to explore, Chating with books is a cool idea that could really help us access and organize scientific human knowledge in all fields.

[@leonh-kd3cm]

So when you do Embeddings - maybe I missed it - I assume that you do not generate embedding of the entire PDF page (that you converted to text before that). So how do you chunk the text in the page? By paragraphs? By sentences? (The former would be a bit of a problem if you removed all " " while extracting PDF text.).

[@yajatgulati]

This is the best tutorial I've seen on embeddings search, the applications of this are endless, really excited to start building. Thank you so much for the work that you're doing :)

[@xoxtoree]

how do you get the page numbers in the source?

[@neerajkulkarni6506]

This is an amazing tutorial! More videos like this please.

[@georgesanchez8051]

Your Excali diagrams are invaluable too. I’ve been trying to get into the habit of creating similar diagrams. Do you usually come up with them before you code, make them manually as you go, or put it together once you’re done?

[@vinosamari]

I picked up the habit of doing it before and it helped tremendously. You might get sucked in for a while but it’ll help streamline the process

[@chatwithdata]

Yeah usually before because it helps me think through how to solve the problem

[@picklenickil]

Anything fancier than party tricks requires System design and engineering. Logic is almost always beautiful on paper

[@gambaweb]

This is like having the architecture of your system m. Imo it is a must to have. Good job meanwhile

[@PizzaLord]

@@chatwithdata what tool do you use for the diags?

[@yudhaesap]

Thank you so much. This is exactly what I need. To talk with a document, is kinda insane.

[@dingbono]

Hi Chatwithdata, can you share the canvas link of of your process design starting at 6:33, the full process diagram?

[@sathithyayogi99]

you are Legend brother

[@BlackStudies]

Sorry, I must have missed something. I've watched both videos, and I still don't know how to take a pdf and insert it into Pinecone. Do I upload it somewhere? Do I share a pdf link with Pinecone? How did you get Pinecone to interact with the Tesla pdf? How does Pinecone know that this pdf exists?

[@damnojs]

How many tokens does it use when you ask a question? Can you give a general calculation?

[@ColinTimmins]

Fantastic stuff my friend.

[@justinc2114]

Can this be wrapped into a web app package for ease of use?

[@user-wr4yl7tx3w]

can you consider python videos as well.

[@CowCoder]

Just found this channel, 16k subs now, not for long. You are making great tutorials that I will say I should have made months ago. Keep making videos you can hit 2m subs in less than a year if you make content more relevant and useful to a wider audience and more entertaining

[@Joshua-sp1kd]

14:25 when it retrieves the relevant docs, does it retrieve the relevant chunks, or even subchunks of the docs? Because what if there are too many relevant docs?

[@Joshua-sp1kd]

Consider a group of documents of teaching material that contain lots of information that encompass a diverse range of areas about life. I imagine the "Create Embeddings" step becomes very important. I wonder if there are tags for different overlapping ideas that get created, so that an idea that may not match the exact name of the namespace can still be integrated in responses...

[@TheSshahrukh]

@chatwithdata thanks for sharing this demo. In my line of work we sometimes deal with sensitive and non-public data. What risk factors are associated with analyzing sensitive data using GPT apis?

[@IStMl]

Would building an app that allows doing all this through a simple UI (and with other features) infringe the ToS?

[@timokaya296]

I want to use a tool where I can upload 2/3 pdf files so I can ask it questions during an exam for example so it can answer my questions according to the pdf documents I have uploaded. How can I arrange that ?

[@IvoSchmidt-w9z]

Are there any ready-to-use (free or commercial) applications where I can drop a bunch of PDFs (e.g. 100) and chat about them?

[@hidroman1993]

If you use GPT-4 to send all those tokens, the cost must be insane. Can you show the summary of costs in your OpenAI account?

[@EradicateLoL]

You don't, the embeddings handle getting that crucial context information and then you pipe just the relevant info to ChatGPT to help inform the answer. It's still not cheap by any means, but much cheaper than sending all of that data. We're doing something similar at work, and it's still super cheap.

[@trendkillsp]

The prompt sent is a normal one, the massive data is stored on the database

[@GeekFromPH]

Looking forward to LangChain Agents.

[@rathishmenon1612]

Very nice video, can we use metadata filter instead of namespace , will it work? Also could you please share me your main.ts , fewshotprompt template..

[@todaydailyeveryday]

Hi, great tutorial! I was wondering if it were able to read the financial reports such as the net income, debt etc.. within the balance sheet or income statement?

[@despo13]

bro which keyboard do you use, and thanks for this detailed video

[@angelocortez345]

So my understanding is that it's 1 namedspace per pdf? If I had 10 real estate books, should I create 1 namedspace or 1 per book?

[@greendsnow]

Can you please use other Vector DB's too for the same example?

[@shashankaadimulam4504]

Could you please let me know when the workshop is going to be?

[@chatwithdata]

I'm currently putting together the outline, probably next week.

[@Kryptikoo]

Thank you for sharing, please do more, your explanation is so easy to understand !

[@Phrog64]

I'm extremely interested in AI and Language Models, but I have a limited understanding of where to start. I was wondering if you'd be able to recommend any programs, courses, or tutorials for those in a similar position to me. Thank you, and what an interesting video this was!

[@henry7434]

Maybe read the transformer, Bert and gpt papers first?

[@1986xuan]

He has the course opened now. You will find the link the description below this video

[@rajns8643]

If you have a basic understanding of multivariable calculus, probstats and linear algebra; see few videos on ANN, then RNN, then transformers and BERT, and then finally LLMs. Its a long way but this way you would be able to understand it more deeply and you would be able to understand the recent developments in LLMs more clearly.

[@vladbph]

what if the query is not related to a year - but anything else, how are you going to approach this? Your namespaces are hardcoded essentially... In general case you want them to be dynamic.

[@chatwithdata]

In this demo, the namespace is hardcoded for ingestion purposes, but dynamic afterwards at the point of query

[@arsalanriaz3382]

Is there a python implementation ?

[@mathematicalninja2756]

Can you tell the software you've used to create the thumbnail? I want the visualization tool. Thanks.

[@klammer75]

Where do you get the reference/source in the code? I didn’t seem to see that part?

[@chatwithdata]

github.com/mayooear/gpt4-pdf-chatbot-langchain/blob/main/utils/customPDFLoader.ts

[@khenghuatlim7264]

Just curious how to read the pdf table and format it into JSON? 😊

[@BlaziNTrades]

This is amazing man. Thank you! It will take a while to fully understand how to implement this for myself but I appreciate the knowledge.

[@TylanPillay-l6p]

Will you be able to share the architectural diagram you provided in this video?

[@michaelebuka233]

Where can we get the multiple pdf architecture overview?

[@alivohra254]

Hey! Cool video!! Have you added functionality for multiple pdfs to the GitHub repo? 🎉

[@moz658]

Awesome. Question: While it is searching the answer through the document, is it possible to add an extra condition on the base code that it will be also using its own reasoning and to create some ideas based on both the pdf and its default model upon the question?

[@hayx210]

did u find an answer?

[@moz658]

@@hayx210 Nope. So for now I decided to use FAISS as local vector db. If Chroma repo owner fixes the problem, I will check it again.

[@senju2024]

About privacy concerns, Is there a way for GTP-4 to NOT look a certain data? How about if a PDF has sensitive data that should not scanned by GPT? or maybe it can scan it but due to a "INSTRUCT-POLICY" it is NOT allows to output that related info? I feel security policies need to added to the architecture. My 2 cents.

[@Jingizz]

Openai themselves should add some opt-out so the data doesn't get saved by them. EU might force them to add something.

[@sereeYengsakulpaisal]

what is the upcoming workshop ??

[@abdullaansari7163]

Very nice information, can you please create a video for python based for the same? It will be helpful thanks in advance😍😍😍

[@ArjanDuijs]

How is the amound of tokes used? embedding it and querying it?

[@abossy]

Why is it necessary to divide the documents into namespaces?

[@jasonhjohnson]

I noticed the Github source doesn't include the separation of PDFs into different namespaces as in your video. Is there a reason to do the separation? Does it give better answers?

[@chatwithdata]

The repo is a baseline that can be extended for your use case.

[@anujcb]

@@chatwithdata do you mind sharing the meta data version of the code? where you chunk and tag the documents by year and page number. I am really interested in what you showed in the demo. Impressive work!!!

[@KleiAliaj]

Hi, great video and app. I am trying to deploy it just for testing live, but it doesnt work An error occurred while fetching the data. Please try again. But in local development it is working. how can we fix this ? do you have any version deployed online ? Thanks

[@allthenewsthatsfittomock6578]

First off, very well explained, great detail in the video - thank you for taking the time to put this togeather. I was wondering, for use-cases where the documents you are working with should remain private... is there any way to create the embeddings with some locally run model, rather than sending the raw document text in chunks to OpenAI?

[@이주석-g7g]

Thanks for the great video, I'm using it very well. When I use the code you posted on GitHub based on pinecone, is there any limit to the number of PDF files/total capacity/total number of sides, etc.? It worked fine when the total size of multiple PDF files was around 84MB, but when it went up to 128MB, I got a SocketError:other side closed error in npm run ingest, so adding a PDF file to an already existing vector store doesn't seem to work well. Do you think I need to adjust the timeout for undici to be a few times longer?

[@Antonio-cn3ji]

Amazing job! When database as source? Thanks

[@Lutherbaer]

Thanks for the detailed presentation and explanation of your concept! This is really exciting to learn - subscription is set, I'm looking forward to more videos. Thanks mate!

[@shaythuramelangkovan5800]

how would you address the concern if multple pdf are required to asnwer a qn , do serve all the chunks into the LLM ?

[@mspraja1887]

great tutorial, can we increase the response token limit?

[@silberlinie]

are the years hard wired in your code?

[@svgtdnn6149]

Hi, this is amazing! thanks for the contents Wondering if you have any idea on the scalability and and cost of having such a system?

[@bourbonbristles]

How are you handing merging the results from multiple queries for each namespace? Also thoughts on using metadata filtering instead?

[@jazzyj2899]

Let me know if youre interested in putting together a group to try and work that part out. I've tried a few things but no luck yet. can share results.

[@picklenickil]

Just as a suggestion, Put a disclaimer. I tried running something similar (tensor flow Universal encoder decoder 4 running locally + some custom RNN + SinDy)with Kongsberg groups annual statement which consists many smaller companies such as Rolls Maritime and Security. Although publicly available. I swear to everything holy, All my files vanished. No updates or crash nothing. I went to prepare some food, came back and it was like I never started the project. Hell.. even my Bills for openAI were cleared. Like I never wrote a word. Im still doubting my sanity. But the paperwork is there so I couldn't have "tripped".

[@carltondaniel8966]

can you interact with images in pdf or tables by asking q and a , is it possible to extract tables and images from pdf , in the structured format ?

[@mpark3]

Thank you very much for the tutorial. I asked a question in Korean but it answered in English (understood the question). Where should I change to make it answer in another language?

[@chatwithdata]

In `makechain.ts` in the repo change the prompt.

[@ChrisAllenMusic]

Thanks! Terrific walk-through, really appreciate this tutorial!

[@eurasiantreesparrow7547]

I wonder if something like this could be used to read/analyze/edit entire codebases with gpt-4.

[@alexsov]

thank you! not clear how you select namespaces (pdfs). just by year in request? code in video not same as in github?

[@LukeTownsend-s2t]

This video and channel are awesome! I'm working on a version of this demo in python, would you mind adding an Apache or MIT license to the gpt4-pdf-chatbot-langchain repo so I can share without worrying about any weird copyright issues with github? (Accidentally posted this on the wrong video earlier...)

[@moakus]

Can you have gpt4 analyze the Mueller Report?

[@thedude9270]

This tutorial is mindblowing. How much does it cost to feed chatgpt all of the tokens from the context document though?

[@prednosttrake]

Question: instead of PDF, could you point it to the database to do the same? How would one accomplish that? Thinking of a BI tool or an ERP. I think digitizing PDFs will be great for organizations that have technical machinery - example field crew sales - if they need to answer the technical question, it can be input (via voice) into GPT and reference company information. One would need to make sure it is secure (ie ChatGPT cannot store what it reads).

[@JWC-uq1bs]

Hi Mayo, is there a way to include paramaters in the conversationchain for maximum length, frequency penalty & presence penalty? Apologies if it's simple but new to this. With your instructions + video, i was able ingest my own PDF's. It's a great start & now i'm looking to optimise the outputs.

[@marcofikkers]

Am i the only one that thinks this needs an animated clippy from old word versions as an avatar? This is how we wanted it to work!

[@anujcb]

@Chat with data , do you mind sharing the meta data version of the code? where you chunk and tag the documents by year and page number. I am really interested in what you showed in the demo. Impressive work!!!

[@infocentrousmajac]

Awesome stuff... Excellent video!

[@Iconic-scenes]

Thanks, but I am wondering is GPT-4 analysing a specific Chuck of a document or the whole document? Because in the diagramme is written specific document is sent to GPT-4 and we know that we have a token limitation