Use AI to chat with a relational SQL database with natural language!

[@alexthecodewolf]

Part 2 follow up video with an enhanced app! kzbin.info/www/bejne/iHbaZKyVf9tmf8k The demo app for this video is also now available here: github.com/alex-wolf-ps/chat-sql-data-demo Sorry for the delays!

[@MarvinMarkhamBurgos]

Is it possible to run this on my mac m3? How can I get it up on my localhost once I have the Azure DB and the Azure OpenAI service up?

[@dipteshbose]

Awesome. Thanks

[@SocratesPonnusamy-pi9iu]

Hello Alex, This video was very helpful. Thank you! I have a question. Do you have a video showing how we can do the same thing from a chatbot?

[@kalyanpolasi]

Thank you for making this video - can you please allow this code to be accessible, for some reason, I am not able to access when I hit the URL.

[@alexthecodewolf]

@@kalyanpolasi The code is a public URL, I'm not sure why you're unable to access it unfortunately. I tried it on a few different devices incognito and it seems to work.

[@Jake-td8nm]

AI has so much coverage, and 99% of it is either "here's a photorealistic image that has 13.5 fingers" or even worse the generic "You should be terrified because AI is coming!" both of which hold no value, provide no context, and don't show any use case that's valuable to 95% of individuals, and 100% of enterprise. Conversely, this video shows an actual coherent use of AI that could blow up and be super valuable - actually improving the productivity of people using it. Bravo!

[@alexthecodewolf]

Thanks, I try to showcase real world demos instead of just flashy but useless concepts

[@511Burak]

Hi thank you for sharing. Is there a way to get the sample code?

[@ShaharNardiaAllOverTheNet]

thank you very much wolf🙏 exactly what I am coding right now for my customer

[@alexthecodewolf]

Thank you so much, generous!

[@sindhupillai1354]

Good starting point for the people who are exploring more with Gen AI. Thanks for sharing valuable real time use-case. I am in 'Subscribed'. Hurray! waiting for more such use cases.

[@alexthecodewolf]

Thank you!

[@lshagh6045]

That was very useful and professional. Thank you indeed and thanks for sharing 👍

[@alexthecodewolf]

Thanks I appreciate it!

[@alexthecodewolf]

Due to the requests here I'm working on getting the source code available for this, it was tied to a project I couldn't release but am working that out.

[@mukshobhit]

Thanks, did you get chance to get repo?

[@BruceLanskiy]

Thank you Alex, looking forward to learn how to do it

[@alexthecodewolf]

@@mukshobhit It's available now, thanks.

[@alexthecodewolf]

It's available now, thanks.

[@BruceLanskiy]

@@alexthecodewolf Thank you Alex, grateful for making it available

[@andrewherrick879]

hi this is pretty cool. is your code available open source? i checked your GH but didn't see it. thanks!

[@benwillman5106]

Thanks for this video, super helpful!

[@rubberuk]

Excellent video! Have you got a link to source code? thank you.

[@TellaTrix]

Wowo ! No more developer required to binding data and complex query.

[@alexthecodewolf]

Yeah one of my favorite things about this is it can help devs as much as analysts because it can write your sql queries for you....to some extent :)

[@DavidGuo-m6j]

the key for it to work is your systemMessage prompt, which is a huge string. it takes a lot of tokens and costs dearly. can't you create a customized model in openai and let it remember the database schema? in that way, when you run query, all you need is only userPrompt, isn't it?

[@alexthecodewolf]

This is a good point, I'm sure you could fine tune a model to have the schema preloaded. However this can be a much more involved process that you'd have to do for every database, this solution is designed as more of a quick hookup to any database. Part 2 shows how you can just rapidly load in multiple db connections

[@applepeel1662]

Hey! Please share the source code! This is honestly really cool

[@alexclark6777]

Remember Microsoft English Query? Pepperidge Farm Remembers...

[@sinamohammadpourinaeim1058]

Hi Code Wolf Thanks for your useful video, Is there any offline AI Service to use the same thing in our local machine or our own sever?

[@alexthecodewolf]

Hey, thanks for watching. There are definitely local AI models, but I haven't worked with them enough to recommend any sort of solution for this. Would probably be a good topic to research for another video :)

[@brunocarvalho3229]

Hi! Thank you so much for this! I'm trying to work on a demo for my company where I want to read a table of incidents and summarize the resolution note of each one and send back to Service Now. The incidents table is already a view on our Azure enviroment, so I think I need first to get the query from AI, get the data from the table and then ask again the AI to iterate over the retrivied table and summarize the notes, structure the data again with the summarized notes and send back to Service Now via API maybe. Do you think this architecture makes sense? Really appreciate this knowledge shraring!

[@AbhinavKumar-tx5er]

Well , I have used to similar kind of approach too. Did you get any luck on this? Did your AI responded with the correct answer?

[@brunocarvalho3229]

@@AbhinavKumar-tx5er Hi, yes! I've getting some pretty good results with the GPT-4o. I followed this tutorial to adapt to my case: kzbin.info/www/bejne/b5TGnWSVjNplarM

[@SocratesPonnusamy-pi9iu]

Hello Alex, This video was very helpful. I have a question. Do you have a video showing how we can do the same thing from a chatbot?

[@alexthecodewolf]

I don't have this video yet, but I'm planning to make one!

[@SocratesPonnusamy-pi9iu]

@@alexthecodewolf Thanks. Can you please give me some pointers on how to do it? All the chatbots deals with indexer and cognitive services.

[@AbhinavKumar-tx5er]

Thanks for making this video. So what is the probability that it won't hallucinate . Well I am using similar kind of prompting approach when using llama model but curious to know about hallucination.

[@alexthecodewolf]

It's difficult to estimate hallucinations due to so many varying factors. However I'll just say anecdotally that the results for me have either generally been correct or just not worked at all (the SQL query doesn't work) - I'd have to setup a more rigorous test plan to estimate hallucinations better.

[@AbhinavKumar-tx5er]

@@alexthecodewolf I took one approach by first storing the table in the form of JSON and then asking the question from the JSON. but again the things didn't go well . Is it I have to do something with the prompt?

[@ParagPandit]

It should be possible to just bind the json data or sql table to the tableview.

[@pvsherman]

Excellent video. But unfortunately I'm getting very inconsistent results. As soon as a little bit more complicated SQL needs to be produced - every run returns a different result. Subqueries, unions are too much for it. Or I'm doing something wrong. Probably will have to create views so AI will be able to do simple selects.

[@srikanthvarmakota]

here i have one doubt now we are also using the same methods but for same prompt it is preparing different querys when we submit the prompt multiple times

[@Beliver5894-l8h]

if we want to create reports and dashboards using nlq how can we do that? please let us know. will it also create reports and dashboards.

[@jigar61]

Any suggestions as to how you can make this work with a microservices setup with individual domain databases?

[@alexthecodewolf]

Do you mean querying each one or querying all of them at once? In part 2 of this video I show how to load in multiple databases and choose which one you query, that would probably be the simplest option.

[@pavanimmadisetty5099]

The solution is pretty straight and easy. Is there any option exists as of day that can take the whole DB or set of tables as input and provide the solution ?

[@alexthecodewolf]

I'm not sure if this is what you're asking, but there's a follow up video with a more advanced app that automates all the db setup for you: kzbin.info/www/bejne/iHbaZKyVf9tmf8k&lc=Ugxo2dpb2ER4Wqmd1zZ4AaABAg

[@MazimpakaYohaniPetero]

Can I have your E-mail ID please?

[@Trymore-r4i]

Good day bro .I need your help to setup my azure databse.I have been facing challenges for some time

[@gianni4302]

I tried this a couple months ago both with gpt4 and a local instance of llama3. I configured it in a similar way where the chain wasnt connected to the database but instead used only the schema to understand it. That is where the major problem is, context windows for LLM's are still relatively small... How have you managed to tokenize an entire schema without breaking the context window within 2-3 messages?

[@alexthecodewolf]

Hey, I talk about this a bit in part 2 of this video. Basically the best option is to use gpt-128k or some model with a large token limit. If that's still too small I believe some services let you request more. Otherwise you just have to try either break it into pieces, exclude tables you may not need, or find a different solution in general - this is definitely a limitation

[@gianni4302]

@@alexthecodewolf Thanks for the response. I struggled with higher context models as they hallucinated too much to make reliable SQL executions. Tbf, i was using langchain so there could have been other variables within one of the models that caused this. I have been told on github to look into a RAG system to retrieve only parts of the schema that are relevant? Well above my pay grade at this point! Thank you again for the videos man, you are very helpful

[@UmbertoK-dq6pt]

Any chance that you will do a video for dummies showing how to deploy from github ? ;)

[@alexthecodewolf]

Deploy this app or just any app?

[@feelzzgaming6306]

Hi @alexthecodewolf ! Thanks for the video. I have two questions: Is there a way to use OpenAI with and SQL database, without using the Azure OpenAI Service? It is currently only for people and companies who have requested special access. So Im wondering if there is a workout where I can use the OpenAI's API directly towards my own Azure SQL Database, and not have to rely on the OpenAI service on Azure? Number 2: Do you have the source code anywhere? Hope you will reply ASAP as im currently working on a project that has come to a halt.. Thank you very much! :D

[@alexthecodewolf]

Hey, thanks for watching. You should be able to use the regular OpenAI service (non Azure) with this exact same approach, there might just be some slight differences in the code and authentication. I'm not sure how pricing differs between them. Currently this code is part of a separate project I'm working on that I adjusted for this demo, so at this point I can't distribute the source code to for a few reasons, sorry!

[@feelzzgaming6306]

@@alexthecodewolf any idea what the code for that approach would Look like? Spent all day today, but to no avail.. also, thanks for the response!!

[@aeromeme]

Hi, could be possible to have database definition in azure storage file so that way we aboid large prompt, or is there any other solution.

[@alexthecodewolf]

Hey, good question, this is something I'm also looking into myself - how to only have to send the schema once and be able to update and maintain that schema in an active context. So far I haven't found a solution I'm happy with yet but I'll respond here again if I do. The models are getting better at deconstructing large inputs all the time, but I understand from a cost perspective its not desirable.

[@ShaharNardiaAllOverTheNet]

@@alexthecodewolf I tried using the assistant for that - sent him all the scheme in JSON files but it sucks 😔

[@twenty4marc306]

Hi, do you have a public demo?

[@polly28-9]

Thanks a lot! How about huge database? We send the whole schema with the initial prompt which costs a lot? How to solve this problem with huge database?

[@alexthecodewolf]

This can be tricky - I'd recommend this solution for small to medium size databases, I have some other content coming for larger database solutions. However, theoretically this solution can also work with large databases if the table structure is consistent/standardized. You'd probably need to use one of the large GPT models though, such as the 32k GPT-4, which accept higher volumes of tokens. I've been trying to figure out if there's a way to have the AI Model "store" or remember the initial schema prompt, but I haven't been able to find an ideal solution for that.

[@polly28-9]

@@alexthecodewolf Thanks a lot! Please, if you find some ways to have the AI model "store" or remember the initial schema, make such a video or write to me. I'm looking forward to it. Thank you!😀

[@goodtimeforever2035]

Hey @alexthecodewolf - Can you able do the same video using AWS ?

[@alexthecodewolf]

Hey, this should work using any powerful large language model and SQL based database, did you have a specific scenario in mind?

[@sajjadmussani]

very informative. can you please share the code of your Razor app?

[@jyothirmaimachavaram4307]

Hello, Could you please let me know where can I get the source code for it?

[@muratkoc4693]

hello, thanks for video. This video is very useful. How do i use Id to DtoName. For example. I get Supplier_Id from Customer. But i want to use SupplierName with Dto. Can I try to develope AppendSystemMessage for own database?

[@alexthecodewolf]

You could probably added information into the prompt to the AI to tell it to treat one column name as another.

[@muratkoc4693]

@@alexthecodewolf Okay, Thanks

[@andresfelipehiguera785]

Excellent video.

[@pwakabi]

Excellent piece

[@mohan007ish]

Can we use GPT-35-Turbo instead of GPT 4?

[@alexthecodewolf]

You can certainly try using GPT-35-turbo, but GPT-4 included major improvements to understanding and responding with structural content. However it'd be interesting to see how well 35-turbo does in comparison, I didn't actually try that myself.

[@Joly0]

Is it possible to have a version, that works without azure? So using the normal openai api with a key and maybe other llm providers such as ollama?

[@alexthecodewolf]

I actually just made a video about this using Ollama, hah. It shows this same setup at the end using codellama kzbin.info/www/bejne/Z2iaootsotWsg8k The concepts in this video should work with any model that is skilled at text to code (such as code llama) and a SQL database of some sort. You'll have to experiment with different models and prompts to see which works best.

[@gedelababi8]

hai brother..good job...i want added ajax method in index.cshtml page but ajax call not hitting method in index.cshtml.cs page..could u pls write the ajax method ..if u don't mind

[@GeorgeZoto]

Thank you for sharing another great tutorial :)

[@hirenshekhda3567]

Hello sir can you please source code for education purposes? Its working universe database?

[@dinamohamed13600]

Thank you , very helpful

[@ABHISHEKSHARMA-fo4tf]

Hello sir this is very help full video , i have doubt can i used the 500 tables and then we can used for this retrieving the SQL Query. because i am using the SQL Database so i give the error of Rate Limit Error 429 , where token limit is 60000 and i request the token for the 157656 so its give the error because the lots of table it taken can you give me solution for this, it is very helpfull for me

[@alexthecodewolf]

I believe current gpt models are limited to 128k tokens - I would recommend either breaking apart your schema to only the tables you need if possible, or searching for an AI model with higher token limits. There might also be local or self hosted models for code generation that support higher token inputs. I have some other videos about local ai models that show this database connection scenario

[@ABHISHEKSHARMA-fo4tf]

​@@alexthecodewolf​ Thank you sir

[@bengroseclose1795]

Your a helpful cheerful youtuber :D

[@OvRaf]

thank you!

[@THEDTSMAN]

What happens if someone says “delete all the data” ?

[@alexthecodewolf]

That's a really valid concern, you'd want to make sure the app is running as a user that only has read permissions.

[@jkaryskycoo]

This feature is not available to the general public. Would have been nice to know that.

[@alexthecodewolf]

Hey, I probably should have mentioned you have to apply for openai access, but it's fairly easy to get approved. If you can't get access you can also substitute the AI service with gpt-4 similar alternatives like standard openai instead of azure openai, or even local ai models that are specifically trained for code generation

[@iamalmostanonymous]

Sqlite would be a simpler starting DB. No need for MS subscription.

[@alexthecodewolf]

Thanks for the feedback. I chose azure because it can auto generate a decent sample db for you but I realize the azure subscription might be a barrier. The sample app works with any SQL DB connection string though

[@arjunmaniyani2975]

any opensource models that i can use instead of openai?

[@alexthecodewolf]

You could use other models such as Phi3 or the Llama models, these are easy to setup locally and should work fairly well. I have another video that shows how to set this up using Phi3 and Ollama locally, towards the end of the video kzbin.info/www/bejne/Z2iaootsotWsg8k

[@arjunmaniyani2975]

@@alexthecodewolf Appreciate the quick reply!

[@Baraka-yl2dp]

Code Wolf, can you please share your source code?

[@liammcmullen4497]

Thanks

[@alexthecodewolf]

Thanks so much for your support! It means a lot.

[@mouradaissani8957]

very good, thank you 🐺

[@alexthecodewolf]

Glad you enjoyed it

[@mouradaissani8957]

BTW ; 11:00 warning , ur openaikey is exposed ;) @@alexthecodewolf

[@chandrashekarsrinivasbabut2454]

Your Open AI Key is visible in the video.

[@alexthecodewolf]

The key/service were deleted after I made the video - thanks though!

[@alexanderkamenev5529]

GIT REPO IF YOU DONT MIND PLEASE

[@codeadmin2]

Hey wolf can this demo app work on local DB or on premise and if yes How? @alexthecodewolf

[@alexthecodewolf]

Hey, yes it can work with local DB and on prem. You would just replace the connection string placeholders in the demo app with localdb connection strings. You could then use the same Azure AI service or you could use a local AI model - I just posted a couple videos explaining how to setup and connect to local AI if that's what you're looking for.