Chat with Your SQL Data Using ChatGPT

[@funkedelic_bob]

Really cool to see MG. However, I was hoping this was going to be more of an expansion on Part 2 and showing how to hook up a SQL server to Azure Cognitive Search. Would love to see a video on that :)

[@blahblahblahblah8425]

This is gonna change the whole job of a data analyst to another LEVEL... Will try this and get back with doubts...👍 cool video

[@SamanthaAz-g9l]

Thanks MG for making such high quality content!

[@yusufkemaldemir9393]

@MG, how can you connect to existing server? Having SSMS, server name and lots of database in it. Without using/creating sqlite, can i directly connect langchain to existing SSMS? Thanks

[@80211Denver]

I think I'm echoing what others are asking for: I have SQL 2016,2019 servers with multiple DBs. I would like an interface application similar to what you're using, I think. That interface can process the schemas of the databases and then help me form new queries and retune existing queries as required. As I see it, your python interface application is producing a local .db file as source material for your questions. I'm hoping the interface can leave the data in place -due to its size. I imagine with the schemas, ChatGPT or some other AI engine can loosely understand the goals and produce, test, refine queries for me to QA and work with. Perhaps this is not possible yet? I suppose another issue is privacy and how I can use the engine without exposing my private data.

[@maciejgowacki5020]

Hi "MG", it would be good to see a film about basic Azure ML configurations. The problem I see is that Azure ML provides so many possibilities for teams working on a business solution that it can be confusing for beginners. Basically, Microsoft does not introduce a "standard"; you can use everything that is available in any way, for example, Databricks cluster, MLflow, bamboolib - a GUI for pandas - here you go, GitLab plus Databricks - no problem, ONNX, Feast, OpenAI - why not. In my opinion, this is a big drawback because at the "start," I have to choose tools myself (know the pros and cons of these solutions), and I have to have a solution architecture established. For example, should I use the Run class and methods (log, log_accuracy_table, log_list, log_predictions) or maybe MLflow?

[@CaribouDataScience]

Speaking of talking with you DB. What are you chances of you making a video on using the AskYourDatabase plugin for ChatpgtPlus.

[@Elve11e]

That’s exactly what I was looking for thanks so much

[@dvenkatnath]

Hey bro !, Amazing video and very clear explanation. Very clear and comprehensive. I tried and understood the works. Thank you.

[@somilmehta5929]

Hi MG, Can you please make a tutorial on how to connect it with SQL server as well.

[@vinitmodi1]

Hi, Can we connect this to a MySQL database directly? Also can OpenAI help us in modifying the data?

[@felipe.arce96]

Best intro ever!

[@olanrewajuadagba955]

How do i setup set-up MySQL in Visual Studio Code. Thank you for the video.

[@vijayakannanr2543]

Great Video, Is there a way to implement CRUD operations in the DB using Langchain and GPT? If the user types "I want to add a new artist", then the app should ask for the necessary details. Once the user enters the necessary details, it should create a new record in the Artist table.

[@creneemugo94]

You can run a stored procedure that brings back only the data you want and just place the data in a collections object.

[@sunny13sohom]

Excellent video MG. Love the content. Only one query. If the data inside the table changes while we are conversing with ChatGPT, does the change reflect in the model's output ?

[@nachopascual9085]

Excellent video!!! Is it possible to add memory to the agent so that it remembers the questions and reduce the number of tokens used in them?

[@CaribouDataScience]

Say, where did you get these spiffy jumpers?

[@sagnikc3]

Hi @MG, thanks a million for the wonderful video. I am trying this on my SQL DB and I get the token limit error. Can you please share any solution on how to handle the issue when applying it on a large DB?

[@dhemant859]

Hi MG, Is there any way to do this in dotnet (c#) with SQL Server. May be using Azure Cognitive Services with database. I tried a lot but did not find anything. Thanks

[@bitcoinjc]

Yes please I would like to know too.

[@bhaelen]

Really great video..! Question: I have an Azure SQL database. I had to use SQL Alchemy to connect to it from OpenAI, and it is working fine, albeit a bit slow. Is SQL Alchemy still the only way to connect from OpenAI or is there a way to connect directly? Thank you!

[@rameshh3821]

Is it possible to get pie charts/bar charts using the chatbot? Basically, it should query data from SQL and plot graphs

[@farexBaby-ur8ns]

missing something..is there a sql agent loaded locally that you interact with which in turn accesses db? Do conv go outside local, to openai servers or are all this local (if not I dont like chatgpt learning my data and database structure)

[@esamelkandro1542]

Thanks a lot MG!... Do you know how to connect Mongo DB with ChatGPT?

[@TheOntheskies]

Thankyou MG, for the amazing video. A couple of questions 1. It appears that you need to send a list of all the table names to chatgpt for each query which will cost money. Is there a way to avoid sending it each time let ChatGPT remember at least for that session if not permanent 2. If I migrate my RDBMS data into a vector database, would it be an overkill? any chatGPT cost minimization pro/cons?

[@pramodhb2881]

Hi MG, Can we try something in the reverse direction. Example, given a query with lots of joins and aliases, can OpenAI figure out all the columns (both alias and real names) used in the query? Want to know which tables and columns are used in the query.

[@scottwolfe6029]

Great video thanks. Question: What if your SQL schema has table names that are not intuitive. My ERP uses table names such as ABXXXXX for the payables...etc.

[@hansalas]

I would think if they have unintuitive names. Then the person has to type those unintuitive names in his input query as well.

[@SiyaMedia]

I wonder whether the model can use views since with a view you can rename the weird named columns

[@TheOntheskies]

Would we get better outputs if i move my RDBMS data to a db like Elastic Search?

[@AnuragKhareOracle]

Hi MG, wondering if you have tested sequence of prompts (sql) one feed to others as part of analysis.

[@Kabirsaxena-d7e]

hi, thanks for this video. Can you suggest me how should i for it if i am using mongodb?

[@Prakash-oq5ke]

Amazing. The code worked perfectly!

[@MuhammadNaseer-v6s]

This is amazing content MG. Thanks alot for sharing this!

[@mombadj4486]

This is safe for our database? have risk to drop o changes database? How connect my db of Databricks to OpenAI?

[@coolrock3733]

I believe that chatGPT doesn directly interact with your database. It just checks the meta information of your database, makes a query, and returns the query to you. From there you run the query on your device/server. In the example case, I believe the LangChain package is doing it for us. Correct me if I'm wrong though.

[@djblstampdjblstamp5159]

Wow that’s an intro ☝️

[@shriyasathish3422]

I know chatgpt doesn't have direct access to the database, it just generates sql query based on the input given, but does it save this information. Can I integrate this with organizational data. Does it raise privacy concerns ?

[@igorlaktic8891]

Hi, thanks for cool stuff. The shown example has tables with names that match the purpose, the same applies to the columns in the tables. In reality, tables can have names that are meaningful to the programmer, but that do not tell what data is in the table. The same applies to columns. Do you have a suggestion on how to install and in which place a translator that would instruct the AI what the purpose of the table is (what the data in it describes) and the same for each column in the corresponding table.

[@ashishlogin]

I am also looking for an answer. Great question! The solution can unlock many possibilities. I am a newbie here but I read there could be several langchain agents that can work together. In that case, we may feed all the logic in a vector db. Fire the LLM model, ask it to read the definition from a vector db and LLM will generate the sql query , and finally hit the sql db and return result to the LLM model. Not sure, but just an educated guess. waiting for an expert to reply. :)

[@Danyalktk-d6n]

Hey, I have a question the gpt model has token problems you know we can use a limited number of tokens I mean we can not train a model with a large amount of data. Just we will need to divide it into smaller chunks. So does sql solve this issues I mean if I have a table of thousands of data and I ask question gpt to tell me about specific entry in database does I will get sane error or I can do that . Please answer . Thanks in advance

[@micaelakulesz9387]

Hi! How can I retrain the model as to reduce the error in the translation natural language to query? Retraining is I guess almost absurd, but would you have any suggestion on how to correct these queries? For instance, I sometimes ask for an average and the query doesn't add a GROUP BY cluase.

[@thelist8850]

Brilliant! I love it.

[@swetareddy8786]

Hi MG! The video was really helpful. Can I know in details about the same scenario if we have the column's content of the data table in the jsonb that is "key-value pair" format?

[@MuhammadUsman-ix6jo]

Hi there, Is that Possible using PHP and MySQL?

[@maciejgowacki5020]

Microsoft does not support MySQL, and honestly, I don't know why. For example, Dataset only supports SQL Server. I have to load data using CSV or JSON files.

[@coolrock3733]

hahaha same question, leaving this reply here in case someone answers.

[@kylecheung9302]

How would you go about building nuance so that GPT knows which tables to pull to answer the question? Say if my database is a fake bike database with 100+ tables and I ask show me all customers who bought mountain bikes, how would you tell GPT that it should actually use type = Mountain Bike and to use the right table?

[@inspiredteaching4468]

When I use Langchain to connect my sql database to Openai api do my private data becomes accessible to OpenAI? I am asking this in context to data security. Suppose I am building a chatbot for my client and use this architecture then is there a possibility that the data is exposed to OpenAI no matter they use it or not to train their models?

[@inquizityofficial3272]

what is the privacy that we get from this. lets say we use this for company data then is this query / database accessible to others?

[@TimYeyuqing]

does this sql tables have to be relational tables? Can I use nosql for langchain?

[@LoneRanger.801]

Love your videos.

[@akarkabkarim]

The video I amazing but I have a question about chat GDP can replace the developers.

[@ChristianGUIDIBI]

This I great thanks for sharing. Will try it and see if I can have it generate sql query to build complex report joining those tables

[@gunjanprajapati6435]

Do we have an example with Angular?

[@gnosisdg8497]

can you do it for gpt4all local installation to communicate with an sql like the example? if yes how ?

[@sarcasticland-id4yi]

How to make this work for large scale DB's?

[@AmandeepSingh-lc9wh]

Hi Thank you for the video. Can you please explain something on Database testing using OpenAI normal Language

[@Sachin-kk3np]

Amazing!! I have a Doubt... Let's say I have created an environment where a user can log in, enter their query into the chat box, and ChatGPT will fetch the results from the database and provide them to the user, as shown in your video. Let's assume that my database contains income data, and since ChatGPT has access to data from all over the world, I use the income data from my database to fetch and do some analysis or reporting based on the tax policies of a particular country. The concern here is whether the data in the database can be leaked or put at risk in the open world While using ChatGPT.??

[@abhinavSin]

Same doubt!! @MG please explainnnnnnnnnnnnnn

[@papariding5865]

Amazing, Thanks MG.

[@neel-channel]

How about connect to postgres database?

[@Quenosoyyoo]

is there a way to connect it to a SQL Server Management Studio database?

[@121ashwani]

how can I achieve this is MSSQL database

[@ali57555]

very nice, thank you

[@placement8059]

Can you please make a video on a multiple PDF Chatbot where the answers contain texts,images,tables from the pdfs and also tell the PDFs from where the answer is taken(citation)

[@AhmedRauf-k8v]

how can you connect to own sql server?

[@faroukkassebi8414]

is there a way to connect it to a PosgreSQl DB?

[@BREx1000]

Great video. Can I teach chatgpt to generate code in some custom script which it didn't see before?

[@pylanookesh8227]

Can we connect this to KUSTO databases ?

[@adilmajeed8439]

Getting the following after cloning the repository locally " 1 validation error for SQLDatabaseToolkit llm field required (type=value_error.missing)", what i am missing here? This issue was raising by the interpreter at "----> 2 toolkit = SQLDatabaseToolkit(db=db)" line

[@adilmajeed8439]

It was resolved by enhancing the following statement; " dburi = SQLDatabase.from_uri("sqlite:///Chinook.db") llm = OpenAI(temperature=0) toolkit = SQLDatabaseToolkit(db=dburi, llm=llm) agent_executor = create_sql_agent( llm=llm, toolkit=toolkit, verbose=True ) "

[@amanshaikh7413]

@@adilmajeed8439 KeyError: 'tools' I'm getting this error after applying the above solution

[@ProgrammerAaron]

@@adilmajeed8439 Thanks, solved by your reply and saved me some time.

[@jdoejdoe6161]

Please how can we connect with data on AWS dynamodb?

[@bitcoinjc]

Can you maybe do a video where you show one of these models one can download, that is very good in SQL query creations and stuff like the video, that can also talk to a MS SQL database that is big. In other words, do something similar but not using ChatGPT where you have to pay for every large promnpt/result? This would be awesome.

[@_megazz]

I tried using this in a large database schema and I always go over the limit of 4096 tokens for any request. I guess this works in a small schema like this and where tables and columns have very descriptive names, but this is not the case for an enterprise database.

[@Monsieurbungle]

A data dictionary / catalogue which explains the tables and columns can be used to speed it up and reduce errors, I think.

[@creneemugo94]

Use the embeddings api to narrow down the most relevant data. Search for youtube videos that explain embedding.

[@SanjayRoy-vz5ih]

​@@creneemugo94freecodecamp recently released a whole tutorial on vector embedding...fyi

[@etabeta_nb]

Are you saying the whole schema was so massive it surpassed over the limit?

[@etabeta_nb]

Can we use Claude etc then?

[@shivrajpurohit8632]

Hi MG! Nice video! Could you please help me with multiple tables? the problem is I am not able to read the data from multiple tables so please help me.

[@PauloG-d6w]

Hi MG! Nice video, that;s what I was looking for. Just one question, I am working in a company that make intranet system, so we have our own database and I guess servers. Can you please tell me what's the cost to setup this sort of system? Do you need to pay just openAI ? Would be great if you include pricing you need to set up the system! Btw +1 Subscribed ! :)

[@davidmichaelcomfort]

This looks great. I would assume you do this thru Azure OpenAI too? Also, it would be great to see ChatGPT interact with tabular data in form of a CSV, pandas or pyspark dataframe

[@therav2007]

Hi MG I want to do natural language prompts against IoT data that's stored in timescaledb (postgres)

[@wakeeltalk8380]

did you figure it out?

[@avicool08]

wow, this is really cool which will really help many of our scenario. Please share more of these kind of videos. Here you've linked with local SQLite, is there a way we can connect to external source in SQL Serverless in Synapse? (which has like GBs of data)

[@younuskhan2841]

Can you please help me from out of this below error: ValidationError: 1 validation error for SQLDatabaseToolkit llm field required (type=value_error.missing)

[@vasubabujinagam5977]

Change to this: llm = OpenAI(temperature=0) db = SQLDatabase.from_uri("sqlite:///./Chinook.db") toolkit = SQLDatabaseToolkit(db=db, llm=llm) agent_executor = create_sql_agent( llm=llm, toolkit=toolkit, verbose=True ) instead of db = SQLDatabase.from_uri("sqlite:///./Chinook.db") toolkit = SQLDatabaseToolkit(db=db) agent_executor = create_sql_agent( llm=OpenAI(temperature=0), toolkit=toolkit, verbose=True )

[@Jonathan-ih9sm]

@@vasubabujinagam5977 god bless u bro I was so annoyed and this worked

[@dhayford92]

So I wanted to know how one can make AI response into json format . So let say I have an AI assistant which the user can ask it to help him to figure out some item for a party . I want the result to be like { “Ai_message”: “something here”, “Items”: [ #list of items here ] } And the Ai will get the results from SQLite database using Django framework or just python

[@TheBigStride]

This is interesting, but when I actually implemented this on an active MySQL database that has any complexity to it, I was very disappointed in the output. This is still in its infancy unless you have a very simple database and need very little information out of it. From a cost perspective, it was definitely not worth it.

[@101RealTalker]

Is anyone familiar with Chris Harrison and the graph he achieved representing the dense cross references of the Bible? (I would include a link here but some people are weary of such clickables, but if you Google it it comes right up)... I am looking to achieve a similar output, from an input of 2 million words directed to a single project, anyone know how this could be achieved?

[@azeezullasheriff7746]

How to connect the sql server to chatgpt,

[@MehebubMukut]

3:32 , But you shouldn't miss the starting, trust me. 🤣🤣

[@ibadkhan-h7m]

How I can create multiple chats like the ChatGPT where every Chat have own chat history? Please anyone who can help me.

[@Charango123quena]

I can't see this working .. There's no way any company will ever open its data to Chatgpt

[@failure__is__my__hobby]

Hello "MG" , great thanks !!!!......please make a video for connecting database (any database ) to huggingface model using langchain or anything .....THANK YOU 😊 without OPENAI_KEY

[@didieranastaseleon8146]

i worry about him

[@younuskhan2841]

Why I'm getting this error after run this code: db = SQLDatabase.from_uri("sqlite:///./Chinook.db") toolkit = SQLDatabaseToolkit(db=db) agent_executor = create_sql_agent( llm=OpenAI(temperature=0), toolkit=toolkit, verbose=True ) ValidationError: 1 validation error for SQLDatabaseToolkit llm field required (type=value_error.missing)

[@jjordalope]

me too 😭

[@stephenpeters4590]

This sure looks pretty neat, but I'm running into an error. Ubuntu 22.04, Python 3.10.6, langchain 0.0.198 In cell 8 of the notebook: agent_executor.run("Describe the playlisttrack table") Error in on_chain_start callback: 'name' --------------------------------------------------------------------------- AuthenticationError Traceback (most recent call last) Cell In[8], line 1 ----> 1 agent_executor.run("Describe the playlisttrack table") ...and then many, many more lines in the error message. Anyone know what's going on here?

[@Aidev7876]

is it possible to add memory to the agent?

[@LifeTravelerAmmu]

Hello "MG" , great thanks !!!!......please make a video for connecting database (any database ) to huggingface model using langchain or anything .....THANK YOU 😊 without OPENAI_KEY