💬 OpenAI Chat with Excel CSV using LangChain

[@kumargaurav2170]

Maybe we can use prompt _template & agent module of Langchain to make it more robust at giving answers.

[@Ryan-yj4sd]

please make one on chatting with SQL database

[@anujsahu07]

@@mzerone-g6m any reference you have for the same?

[@efficiencydna897]

greatidea

[@mzerone-g6m]

@@anujsahu07 just read docs no need to reference

[@Manali_99]

It shows error during create an index using the loaded documents step, it asks to mention embeddings in first line

[@AkaThePistachio]

Hey for some reason when trying to create the vector store i get a conflict issue with pydantic saying "`BaseSettings` has been moved to the `pydantic-settings`" How can i fix this?

[@bourbe]

Hello, I am wonderng About something, when WE se a csv agent, WE don't need to use embeding, Vector data base or a memory ? I am currenly confuse

[@AmirBehjat-c9f]

First it worked fine but then when I increased the size of the file it became useless, avoid the question. Once I returned to the original file it remained useless. It seems that some limitations activated and made the code wrong.

[@hariram-pz5uz]

when i implement this using azure openai it throwing me too many input error. How to resolve this?

[@nishkarve]

Why do I run into ChromaDB authentication errors? Is there some setup outside this notebook for ChromaDB?

[@rheapoddar6912]

I am getting a rateLimit error, do you know what we can do about that?

[@matthewpan8518]

thank you for your helpful video, i am a new learner in ai, i gotta a question:what is the difference between gpt-index and gpt-embedding? i am a little confused...

[@xevenau]

do you have one that uses llama3 for chatting to avoid cost?

[@AbirKhan-j4r]

Did you get any solution with LLAMA

[@RetropunkAI]

Which OpenAI model does it use? I may have missed it.

[@chetan5581]

But can we somehow control the no of tokens being used ? I have csv files in GBs

[@Tadi859]

I am working on a project with similar data volume. Were you able to solve the problem?

[@smart-sg5cs]

dear little coder if we want to set a system like this and want to use SQL data base instead of a vector DB .which one will have better and more accurate results .also do we necessarily use vector DB in case we want to use LLM like open ai

[@judickaelsam3103]

Hi everyone, I get this error "RateLimitError: You exceeded your current quota, please check your plan and billing details." when executing the same code. I just want to do the same tests. Why would my quota be exceeded even if it's the first time I use an open ai key? Anyone having the same issue? Do I really need to upgrade as I just want to do exactly the same tests? Please help

[@1littlecoder]

Maybe your 3 months since OpenAI account creation is finished. the free credits expire in 3 months

[@judickaelsam3103]

@@1littlecoder Thanks. I just created it yesterday

[@ayushmansxn]

I am getting error "Failed building wheel for hnswlib" . How to resolve this ???

[@1littlecoder]

Did you try to install that library yourself?

[@ayushmansxn]

yes i did

[@top10collection88]

Bro instead of chroma db use FAISS db it won't show the error :)

[@AshishKumarRajak-xg7il]

I think i have to use paid api for this

[@intrepidani6422]

Can we do the same thing with doc or pdf form

[@estrangeiroemtodaparte]

This channel is awesome!

[@v.svishnu2380]

Do we have any security for the data that we upload as csv ?

[@dipanshumansingka]

How about confedentiality of data given as input

[@chenling2870]

Hey 1littlecoder Thanks for post the code, and that's super helpful. I love it. But I have one question when run the script. It seems like the code only can read 4 lines in the CSV, any chance you might know the reason?

[@VK-zg2bw]

Because, the default similarity search k value is 4, that why.

[@punithsai689]

@@VK-zg2bw can we change that value? i mean i have a csv file of 107 records . if so, what is the maximum value for k?

[@bhushiwakade5699]

I guess it would be k= len(records). Correct me if I am wrong.

[@dhanushdm5761]

same doubt

[@feizaltoefy185]

How would one interact with more than 1 csv file with different field names

[@RedCloudServices]

Langchain would need to help avoid hallucinations perhaps using agents and printing source for responses? i.e. csv rows/cols summary

[@danixkarim]

Can you giude how to add a validation layer?

[@aination7302]

I tried this and it keeps saying "I don't know" :/

[@thekaizer666]

hey can you run a tutorial on how to code your own personal chatGPT text-davinci-003 + text-ada-002 embeddings, and integrate with a vector-database such as Chroma, Pinecone, or Milvus? plus, that its able to do live web-searches and scrape webs for information it doesnt currently have? "David Shapiro ~ AI" did a full tutorial on the very-basic level of it using Pinecone, but the bot is too basic and dull and doesnt really work that well. the video: "Chatbot with INFINITE MEMORY using OpenAI & Pinecone - GPT-3, Embeddings, ADA, Vector DB, Semantic"

[@MohitSingh-ub9gc]

any progress?

[@MohitSingh-ub9gc]

this is awesome!

[@1littlecoder]

Thanks Mohit!

[@rohanchandanshive4258]

Please share, how can we integrate this with gradio UI???

[@WondrousHello]

What model does it use when you assign llm = OpenAI()? It’s important to know so you can monitor token usage I think

[@1littlecoder]

You can specify the model type inside that. By default it uses text davinci 003 if I'm not wrong. But please the documentation once.

[@WondrousHello]

@@1littlecoder thanks for the reply. Ok that makes sense. I assume gpt-3.5-turbo would perform better. Is there any transparency with how many tokens these processes use with LangChain? All this is very nice but I don’t want to burn through all my tokens 😅

[@DeepakSingh-ji3zo]

Does Langchain upload this csv data to openai?

[@RodrigoFigueras]

This works brilliantly but I realized the model will only return a max of 4 pokemon regardless of the prompt. Why is this?

[@dhanushdm5761]

got any answer?

[@extensivamente]

I wanted to ask you if you know the diff-svc project (exchange voices of musical artists). It would be helpful if you can make a video tutorial about it. Thank you!

[@abodh7046]

How to handle 2 csv file?

[@joeboulton7638]

Would this be able to do count calculations? E.g. How many pokemon's name start with the letter F or how many Pokemon have a health above 200

[@1littlecoder]

Ideally it should, did you try?

[@SomuNayakVlogs]

Can you create a video csv with opensource llm model, not with openai

[@ValerioDucci]

Any plugin that does this?

[@Lichen-er6zy]

I am getting langchain package error

[@beinginlife1591]

I have 1 lakh rows , and giving limited token error …

[@mzerone-g6m]

Could you use another open source model like rwkv or any thing.

[@pastoryoda2789]

Little Coder what do you use to edit your videos?

[@1littlecoder]

Screenflow mostly and Da Vinci resolve sometimes

[@isaiahsgametube2321]

thank you so much!

[@harsh7704]

could you implement the rag pipeline here

[@arpitkakkar2780]

What if CSV contains more than 5k rows and more than 20 columns?

[@1littlecoder]

I've not tried such a large one, did you try?

[@arpitkakkar2780]

@@1littlecoder Thanks for the reply. I'm trying to implement it using a pinecone vector database.

[@VK-zg2bw]

This code will not provide proper results, it will only give 4 results (rows) in total, and i can not do much like total count, difference and so on, And for the 4 rows, the similarity search in vectorstore gets only 4 rows in total, it's the default k value. And if not wrong, little coder also know this, that's why he matched his results like I know only Pikachu so I will ask question with it in this video😅 and there is no proper response in any of the questions below.

[@akhilvmathew392]

Yeah. Even after increasing the k size, only 4 values are retrieved. Is there a way to get all the records ?

[@unemployedtechie]

can you do one with gemini-pro please?

[@TerryKinder]

You have accused the Ai of hallucinating. It will remember.

[@1littlecoder]

Haha

[@real23lions]

9:50 soon we will all apologise to AI 😂😂

[@klam77]

Does this not imply that SQL is dead?!!!!!!!

[@smart-sg5cs]

hello little Coder i have been trying to get in touch with u i have messages on ur twitter account also i am a non tech guy are u open for collbration on project basis i am trying to create my app using Llm pp let me know how to contact thanks