Chatbot Answering from Your Own Knowledge Base: Langchain, ChatGPT, Pinecone, and Streamlit: | Code

[@tzengyueng]

🎯 Key Takeaways for quick navigation: 00:00 🤖 The video discusses building a chatbot using Langchain, OpenAI's ChatGPT, Pinecone, and Streamlit for answering questions from your own documents or knowledge base. 01:09 📄 The chatbot answers questions from documents stored in Pinecone, demonstrating the ability to retrieve information from custom knowledge bases. 02:08 🔍 Refining queries is essential for accurate semantic search, helping the chatbot find relevant information based on user questions. 04:14 📊 The video covers indexing documents using Pinecone and the importance of choosing the right dimension for the index. 05:49 🧩 Langchain facilitates document processing and chunking for indexing in Pinecone, supporting various document formats. 11:57 📈 Demonstrates how to push document embeddings into a Pinecone index, enabling semantic search capabilities. 13:08 💬 The video utilizes Streamlit Chat for the user interface, displaying responses and user queries. 16:36 🧠 Explains the different types of memory (buffer, summary, etc.) used in Langchain to maintain chatbot conversation history and context. 18:14 📝 Discusses the use of prompt templates for instructing the chatbot's behavior and structuring conversation prompts. 20:01 🧩 Langchain is used to create chat prompt templates, message placeholders, and maintain conversation history. 20:30 🔗 A "chain" in Langchain combines memory, template, and language model components for chat interactions, such as question answering. 21:38 📝 Users input queries, which are refined for semantic search using Pinecone indexing. 22:49 🔄 Langchain's line chain handles queries and responses, utilizing ChatGPT for generating responses. 25:21 💬 The "conversation string" combines past conversations and the current query, allowing for refining queries to improve semantic search. 28:21 🪶 Refined queries are sent to Pinecone for context searching, enabling more accurate answers based on the conversation context. Made with HARPA AI

[@Slimshady68356]

Thanks

[@ngalenal1004]

You are my hero. I have a academic project in NLP and your tutorial is clearly explained. Thank you for your work. 🙏🙏

[@FutureSmartAI]

Wow, thank you!

[@youssefaserrar2001]

Great video. First time I hear about the Refined Query 👍

[@SaiMadhaviYalla]

Can you please mention versions to install as well, as i am getting error " installed packages version mismatch " while running the code

[@delphiDHS]

Great video. But can you please help me resolve this error: ValueError: variable history should be a list of base messages, got I have search everywhere nothing useful. My code same as yours btw.

[@robertovillegas2220]

have a use case with a haven't seen anywhere: I create a private GPT that has documents as contexts. This documents has criteria for a specific subject I give it system instructions so its function is to evaluate a user document that it's attached as a part of the prompt, to see if the document complies with the criteria in the context documents, and give detail response on the result of evaluation and the justification that includes, the content of the user document and the criteria in the context documents. I want to do that in LangChain but I don't know hot to add a user document as a part of the prompt for the RAG.. It would be great if you can you explain how to approach this implementation. Thanks you for the content!! Keep the good work.

[@TheDigitalSight]

If I'm creating custom facing chatbot and I've to upload pdf of multiple customers how should I store it in pinecone? Can you please explain more on pinecone usage and how its used in wild? Thanks for your easy to understand explanation 😇

[@sumangupta871]

Hi most of the langchain and pinecode codes are not working as of now. Can you please one more video on same topic based on present langchain and pinecode framework. It will really helpful. Thanks.

[@FutureSmartAI]

Noted

[@SamraShaikh-wu8im]

How can we make a chatbot retrieving the data through custom website

[@JainilShah-z8o]

Are all platforms used free ? Also the api keys , are they free ? please answer i have a upcoming hackathon soon need your assistance

[@FutureSmartAI]

Open AI and Pinecone are paid services. You can have pinecone free altervative Chroma DB

[@pranitingole5562]

Hi Pradip, sir, I am working on a user-base chat bot. It's a medical base chat bot. What I am doing is extracting user data from the database and then storing that data in a text file. that user text data is providing to the Langchain and GPT models for conservation with that data. I am able to chat with user data, but I am facing one issue, which is that if I try to search, some think it is different and not present in the document, but it provides an answer from the outside. Like, who is the PM of India like this? I want to restrict GPT from providing unwanted responses. Can you help me with this? I also tried prompt, which worked sometimes.

[@FutureSmartAI]

Hi you need to update prompt and add "If you don't know the answer, just say that you don't know. Don't try to make up an answer." from langchain.prompts.prompt import PromptTemplate from langchain.chains.question_answering import QaChain # Create a new prompt template template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum and keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer. {context} Question: {question} Helpful Answer:"" # Create a new Q&A Chain with the modified prompt PROMPT = PromptTemplate(template=template) qa_chain = QaChain(prompt=PROMPT, retriever=retriever, llm=llm) # Use the modified prompt qa_chain.invoke(question="What is the capital of France?")

[@vashistnarayansingh5995]

So behind the scene what we are doing, is doing the similarity may be using cosine similarity of the input query and all the 30 doc embedding present into the ventor db and returning the one with highest similarity

[@FutureSmartAI]

Yes thats what semantic search does

[@manikantasurapathi92]

Hey, how to erase the user query after we generate a response to the user query. That’s one flaw I’m not able to sort.

[@arora.ishant]

@Pradip- Thanks for video. However, I am getting SSL authentication error when running streamlit code on local machine. Did you do anything specifically to take care of that? How to resolve that?

[@FutureSmartAI]

No I didnt face this issue.

[@keshavchander5570]

Can i use this to ask questions to the code base ,for example : Find the repeated lines or clone lines in the repo .Will it answer

[@mohammednaseefp4681]

UnidentifiedImageError i encountered with this error while uploading documents even there is no images or image file. Please explain🙏

[@AndyHOTlife]

For people to believe your chatbot solution works, you need to provide the data!

[@IshitaJoshi-v9i]

This is giving a lot of errors. Pinecode's documentation has changed and all the functions have been replaced. what to do as a beginner?

[@FutureSmartAI]

let me see if I can update code

[@AbcXyz-lk6sh]

@@FutureSmartAI yeah please do! init function seems to be not working and i cant import Pinecone either

[@Tiger-Tippu]

How to fine tune the model when we use langchain for chat application

[@FutureSmartAI]

Hi finetuning model is independent of langchain and you should be bale to use fientune model id in langchain

[@Tiger-Tippu]

@@FutureSmartAI thank you

[@shashankalva9844]

What if my database is in the form of knowledge graph. What changes do I need to make?

[@FutureSmartAI]

Hi, I havent tried with knowledge graph. Check This: python.langchain.com/docs/use_cases/graph/graph_cypher_qa

[@bloodstream426]

i have a pdf but it is not recognizing it. why is that ??

[@kenzabaffoun5563]

hey, I really appreciate your work, and I really find myself learning with you, so I sincerely thank you for that. I also wanted to ask you, I'm in the process of creating a recruitment bot for my company, I wanted to know if it was possible to reverse the bot's process, i.e. have it initiate the conversation and ask questions, not the other way around, but still keep its intelligent aspect? While indexing the company's own database in case the candidate had any questions to ask?

[@FutureSmartAI]

Hi Yes bot can intialise conversations or show user possible topic or questions they would like to know more

[@cryptokingdom2038]

Brother thanks for the tutorial and i have a doubt What is the maximum capacity of Sentence Transformer Embedding - all-MiniLLM-L6-v2 ? Is possible to convert 50k words into an embedding format with sentence transformet embedding model or any other model available to achieve this?

[@sonikaarora-e1q]

I am getting error in "conversation.predict" method. Did anyone faced this issue?

[@Vijayaadithyan]

I wanted to access data from PostgreSQL, and i referred your lanchain and sqlagent. What should I do next after connecting the db?

[@FutureSmartAI]

Just find postgreSql connection string and see connection is successful. You should check my other videos of natural languagee to sql

[@jayrn4596]

Hi sir. Hope you are doing great. I just want to know instead of pdf, text can we give a database (CSV) as an input?

[@FutureSmartAI]

Yes we can. use langchain sql agent

[@outlanderz8]

​@@FutureSmartAI it is not taking the context in case of csv, can u create chat bot video for csv separately, it looks like completely different use case

[@Lyricverse116]

Hii pradip you are using text-davincii 003 model but that has been deprecated by openai are there any alternatives for this now?

[@FutureSmartAI]

you can use text-embedding-ada-002

[@Lyricverse116]

@@FutureSmartAI okay thank you

[@SaiMadhaviYalla]

Hi Pradip, this is awesome. I am trying to get the data source as well ( i mean the name of the document/ pdf) while getting the answers from the pdf. Can you help me woith this?

[@FutureSmartAI]

Yes when you chunk docs and index them you can store doc name , url as metatdata that you can retrive during semantic search

[@PorterIke]

i am getting AttributeError: type object 'Pinecone' has no attribute 'from_documents' any remedies?

[@FutureSmartAI]

I think I need to update code, those libaries keep changing

[@gabrielmartinphilot980]

Yep alot of parts arent working anymore. And please your video code its alot diferent from your repo, that confuse the hell off persons that are learning!@@FutureSmartAI

[@chinnuabcd]

Hi, instead of a single PDF, can we use multiple pdfs to build the chatbot?

[@FutureSmartAI]

Hi Yes, we just need to add more pdfs to vector database

[@chinnuabcd]

thank you@@FutureSmartAI ... can we make an android app with this framework to get the same functionality?

[@chinnuabcd]

thank you so much for making these videos

[@DonjiKong]

Great video, could you go over the process of publishing the site with streamlit, I keep getting errors. As for uploading the side, it seems the requirements you posted are lacking some things, but I am not sure what to put in it to make it work. Everything I tried throws errors.

[@FutureSmartAI]

Where are you publishing? Check this kzbin.info/www/bejne/b2GXlIpvoa9qgrM

[@ShankarMutkekar]

It refines even the very first query, which is not useful for me. What am I supposed to do so that it takes first query as it is, and then it may refine the query based on memory ?

[@FutureSmartAI]

We are storing all user queries in session state request if request has no query it means this is going to be first query , so you can add condition that if session sttate request is empty then refine query is same as user query.

[@ShankarMutkekar]

@@FutureSmartAI Yeah, that was easy. If- Else statements worked for me. Thank You

[@hernandocastroarana6206]

Thanks for the video. Do you know what components I can make the Chatbot throw citations at the end? something similar to pihind or perplexity. I want it to extract the information from websites or documents, but I want it to tell me where it extracted the information (citations) from. I would really appreciate the help

[@fakerandom2836]

hi Pradip, i'd also appreciate if you can add any color to this question. thank you in advance.

[@FutureSmartAI]

when we chunk to documents for embeding we can store document url or any other information in that chunk metatdata. When you get those chunks back from semantic search you will also get metadata that you can show at the end of answer as source.

[@outlanderz8]

@@Pradip I want to take directory as user input in streamlit app, not able to.. can u pl help.. i dont want to hardcode the directory, but streamlit has provision only to give filepath not dir path.. pl help Pradip

[@gabyfacio9701]

Is there any recommendation to solve the ""Vector dimension 384 does not match the dimension of the index 1536" problem? the 384 comes from the SentenceTransformer model and the 1536 from Pinecone. I tried to change the Pinecone dimension but is not working. Maybe anothe recommended SentenceTransformer model?

[@FutureSmartAI]

When you create index you can sepcify dimention of index

[@AyushBagwari]

do your conversational buffer memory support managing multiple user history simultaneously?

[@FutureSmartAI]

yes on streamlit it will be different session for different user. If you are uisng other front ends then we maintain conversation (or session id) in db and keep track of all message in that sessions. If we use latest assistant api , it maintains conversation as thread

[@justlutfi2890]

I have used it for my WA bots, referring chatID as Id, and you'll have contexr related to chatID

[@rudrachand6643]

How to insert data in pinecone? Only you mentioned .ipython notebook using google colab,but actual app it is not mentioned. Can you please also mention formal way of updating data to pinecone?

[@FutureSmartAI]

its the same code we are using in app. I shown it notebook so that we can see each step

[@thomasferreira306]

Hi last month everything was working but pinecone has migrate to a new version now "init" is no longer a top-level attribute of the pinecone package. I've made lot of reasearch but i don't find any solution to make it working again.

[@FutureSmartAI]

I think I need to update code

[@thomasferreira306]

@@FutureSmartAI Thank you for your response and your investment! In the meantime, which of your videos do you recommend to recreate a chatbot from txt files?

[@livewithsoumyajit173]

hi Pradip, Can you please create a similar video but replacing pinecone with elastic search ( basically storing the vector embeddings using python in Elasticsearch ) and rest as same.

[@FutureSmartAI]

Hi I dont have any experience with Elasticsearch

[@Priyanshukumar-zu5ei]

Hi sir , I'm getting n authentication eroor that says you didn't provide an api key whereas I have given api key the error is ,AuthenticationError: You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password.

[@jayrn4596]

Hi sir. Can you please share the documents in data content section?

[@FutureSmartAI]

use any text or pdf file. save some text in txt and use.

[@AndyHOTlife]

@@FutureSmartAI That is a lazy answer. Having the data would have really helped here. I've done another experiment with a PDF I found in another tutorial, and the LLM keeps on hallucinating on it. It seems you had a good PoC with your current PDF data, and I wanted to benchmark my approach against this. Saying "save some text in txt and use" is rather stupid and pointless imo, making me move to following other tutorials that do make the data available too. Dislike, hope you make better content in the future.

[@rohith646]

hi pradip as gpt 3.5 and api_keys have limitations so why cant we train model on open source like huggingface platforms and link it to our UI

[@FutureSmartAI]

You can use open source LLM. People are ready to pay Open AI for good results

[@deepak_kori]

Please mention the version in requiremnts.txt file

[@farhanafridi8694]

Can we integrate it to a specific website and also deploy it?

[@FutureSmartAI]

Yes. you should expose it as api and then use it any website chatbot widget

[@SMCGPRA]

Great video! Can we have detail video with open source LLMs and embeddedings with fune tuning using our own data with prerequisite system config needed, pls 🙏🙏🙏🙏

[@FutureSmartAI]

Sure. I havent much explored open source llms

[@boopalanm5206]

How can we create the Chatbot for the data using the LocalDatabses like SQL ,Postgres

[@FutureSmartAI]

you can check my Langchain SQL agent video

[@MushtaqPatel-up1fb]

How to get the page number of the file from which the answer generated?

[@FutureSmartAI]

you can get sources info, but syntax has changed now python.langchain.com/docs/use_cases/question_answering/sources

[@ankan54]

How can I show the sources from the metadata in the UI??

[@FutureSmartAI]

Yes you should be able to show easily. are you aable to access doc source ?

[@manalchavhan441]

What changes we should have to do if only want data from.csv file i.e data source only be.csv file?

[@FutureSmartAI]

You can use natural language to SQL. kzbin.info/www/bejne/jHicfIx5iJVgbqcsi=zKcXOmbhxYecvv6q

[@kunal4557]

Love your videos Pradip. Is it possible to define the query refiner function using an open source model like the latest MPT-30B-Chat, if yes how? I am finding it difficult to define the query refiner function

[@FutureSmartAI]

Yes you should be able to define, check prompt for that particular model. Personally I havent used open source models for generative NLP

[@justinechternach7817]

any recommendations for preparing your own code (in my case c#) to put into pinecone and provide as question context to chatgpt? Specifically the chunking part.

[@FutureSmartAI]

Both Pinecone and Open AI provide rest API, so should be able to do it in c#. But I dont think langchain has support for c#. You can build semantic serach without langchain

[@justinechternach7817]

@@FutureSmartAI Hi. I meant for vectorizing my c# code and chunking it so I can ask ChatGPT questions about it. I’m using Python for this.

[@ggg9gg]

Hi, how do I deploy this application. Right now it is in local machine

[@FutureSmartAI]

Hi I have few videos on deployment Deploy FastAPI & Open AI ChatGPT on AWS EC2 kzbin.info/www/bejne/bXe5gaFoatKge7Msi=iGwaCdZv21ZZcohB Deploy GPT Streamlit App on AWS EC2 kzbin.info/www/bejne/b2GXlIpvoa9qgrMsi=noGaNcPdzEMC4GI_

[@AaronAsherRandall]

What framework are you using to build out the actual UI in the browser that the user interacts with?

[@FutureSmartAI]

Streamlit

[@hispritez2x0]

Your clip is very helpful. May I ask more? Can we use documents in Thai language?

[@FutureSmartAI]

You should use embeding which is multilingual

[@apurvavhatkar8520]

Can we build chatbot for Indian laws using constitution book

[@FutureSmartAI]

yes

[@livewithsoumyajit173]

getting this error , please help , NameError: name 'context' is not defined

[@FutureSmartAI]

Can you check once again ? Code should work fine

[@ysf00yldz]

i need a separate memory for each user if so how do i do this i use phpmyadmin but i can also use mongodb please help me

[@FutureSmartAI]

hey sorry for delayed response. To maintain conversation as per user we need to create backend for chabot. when any conversation start we generate unque id for that session and then insert session id , user id , and user query , gpt respose in db.

[@ABHISHEKSHARMA-fo4tf]

Can you explain how we can make in jupyter without using streamlit

[@FutureSmartAI]

I have another tutorial in jupyter notebook

[@Cawow123456]

If the language of PDF not english still working? for example japanese, chinese.

[@FutureSmartAI]

Semantic similarity still works for non english better choose embeding model wich is multilingual

[@narendrakd]

You are superb bro,i have some technical documets locally stored into my laptop,i want free chatbot locally for my laptop so that i can query my documents....if u can explain me bro...waiting for tour reply

[@FutureSmartAI]

Hi you can try small LLM Eg medium.com/@kagglepro.llc/how-to-setup-mistral-7b-language-model-locally-without-code-lm-studio-30a8e71c9a61

[@RohitK-z1y7e]

how to deploy this code on cloud?

[@FutureSmartAI]

This will help kzbin.info/www/bejne/b2GXlIpvoa9qgrM

[@ShankarMutkekar]

Currently, it takes approx 8-10 secs on an average to get a response, which cant be reduced, I guess(Correct me if I am wrong). But if streaming is implemented, perceived latency gets reduced. Could you please help me with how to implement streaming in langchain ?

[@FutureSmartAI]

But if streaming is implemented, perceived latency gets reduced. Yes True

[@shivamgupta-fb4em]

I find your content very useful, I am pretty new to this area. I have requirement of designing a Q&A system for my company where we have data available in SQL that’s in oracle DB, but problem in tables are not straightforward, it’s like we heavily utilize foreign keys for associations. So, my guess is like we can represent this data to utilize a graph database and then serve it to LLM using cypher query. Do you think it’s the right way or we can go with SQL only

[@MsHalpinFrayser]

will this work with a .csv ?

[@FutureSmartAI]

Its betteer to use csv langchain agent

[@ahmedazeem3804]

Hello, re-create this project with chainlit or any new chat ui, and also it'll be great if you replaced OpenAI with Open source alternative

[@FutureSmartAI]

Sure

[@AKHILRAJEEVPBCE]

@@FutureSmartAI Sir can you help me with an error?

[@AKHILRAJEEVPBCE]

2023-07-30 12:17:58.956 Uncaught exception Traceback (most recent call last): File "C:\Users\Asus\anaconda\lib\site-packages\tornado\http1connection.py", line 276, in _read_message delegate.finish() File "C:\Users\Asus\anaconda\lib\site-packages\tornado outing.py", line 268, in finish self.delegate.finish() File "C:\Users\Asus\anaconda\lib\site-packages\tornado\web.py", line 2395, in finish self.execute() File "C:\Users\Asus\anaconda\lib\site-packages\tornado\web.py", line 2417, in execute self.handler = self.handler_class( File "C:\Users\Asus\anaconda\lib\site-packages\tornado\websocket.py", line 219, in _init super()._init_(application, request, **kwargs) File "C:\Users\Asus\anaconda\lib\site-packages\tornado\web.py", line 214, in init super()._init_() File "C:\Users\Asus\anaconda\lib\typing.py", line 1083, in _no_init raise TypeError('Protocols cannot be instantiated') TypeError: Protocols cannot be instantiated Stopping.. i am getting this error when i run main.py

[@AKHILRAJEEVPBCE]

@@FutureSmartAI 2023-08-01 08:56:12.942 INFO sentence_transformers.SentenceTransformer: Load pretrained SentenceTransformer: all-MiniLM-L6-v2 2023-08-01 08:56:13.591 INFO sentence_transformers.SentenceTransformer: Use pytorch device: cpu 2023-08-01 08:56:14.606 Warning: to view this Streamlit app on a browser, run it with the following command: streamlit run d:\java files\main.py [ARGUMENTS] 2023-08-01 08:56:14.606 Session state does not function when running a script without `streamlit run` and this is the output

[@FutureSmartAI]

@@AKHILRAJEEVPBCE what error?

[@Mostafa_Sharaf_4_9]

great but I have a question about refined query function... GPT has limitation of the input tokens what if the conversation is too long with the user then the input for the refined query function will be bigger than the limit ..?

[@FutureSmartAI]

Yes right in Lnagchain you have different kinds of memory . check this blog.futuresmart.ai/langchain-memory-with-llms-for-advanced-conversational-ai-and-chatbots

[@sinchana.nmakam3633]

i a getting an Attributeerror wen i try to add vectors to pincone by usin pincone.init...can anyone help me solve this problem?

[@FutureSmartAI]

Hi can you share detailed error?

[@sravantipris3544]

can u share git hub link please??

[@zunairakhalid7358]

can we add an image and ask qs

[@AbhinavKumar-jj6kr]

can anyone help me to complete this project

[@sanjanaepari9173]

Sir do the same thing with csv or excel file

[@Lyricverse116]

Is it llm

[@FutureSmartAI]

yes

[@018jumanamuneeram2]

Is the API key paid or free

[@harishankarmandavi1131]

Can we get answer from hindi knowledge base?

[@FutureSmartAI]

Yes. But make sure to use multiligual embeding model

[@vigneshm-zh5po]

what if its a csv file with like 50k records

[@FutureSmartAI]

you should use natural language to SQL or CSV agents

[@vigneshm-zh5po]

@@FutureSmartAI can you do a video using a large csv file or something like that. because most of our dataset is of csv type. sir

[@Slimshady68356]

20:21

[@shashankbarai1809]

Confused

[@FutureSmartAI]

Hi can you specify what was confusion? may be I can clarify or suggest some prerequisit material

[@shankarmutkekar6104]

Hello Sir, I am getting AttributeError: st.session_state has no attribute buffer_memory. Could you please help me in this regard.

[@FutureSmartAI]

It means you are not storing buffer_memory in session and directly accesing it

[@shankarmutkekar6104]

Yes, I got it solved. This was the exact thing I was searching for. Was very delighted to see this. Thank you❤️

[@nishkarve]

@@shankarmutkekar6104 How did you get it solved? I am unable to proceed due to the same error

[@nishkarve]

@@FutureSmartAI I get this error AttributeError: st.session_state has no attribute "buffer_memory". Did you forget to initialize it?

[@AbhinavKumar-jj6kr]

brother can you help me, to complete this project.

[@DadCooks4Us]

very poorly made content.. you seem to be in a rush barely explaining the steps in the notebook.

[@FutureSmartAI]

Hi , Thanks for feedback, Most of the code in that notebook was already covered in erlier video and dont wanted to spend more time on ti. will imporve next time

[@tiger21823]

Bro so we always need open ai api?

[@FutureSmartAI]

To generate answer yes. you could always use other open source model and see if they works for you.

[@tasneem2586]

Is it possible to change the icon image of the bot and user? How do I do that?