The missing pieces to your AI app (pgvector + RAG in prod)

A step-by-step guide to going from pgvector to prod using Supabase. We'll discuss best practices across the board so that you can be confident deploying your application in the real world. Learn more about pgvector: supabase.com/docs/guides/data...
Workshop GitHub repo:
github.com/supabase-community...
It's easy to build an AI proof-of-concept (POC), but how do you turn that into a real production-ready application?
What are the best practices when implementing:
- Retrieval augmented generation (RAG)
- Authorization (row level security)
- Embedding generation (open source models)
- pgvector indexes
- Similarity calculations
- REST APIs
- File storage
00:00 Intro
01:06 Demo & setup
05:28 Step 1 (File storage)
31:40 Step 2 (Documents & splitting)
1:19:02 Step 3 (Embeddings)
1:36:32 Step 4 (Chat & RAG)
2:10:11 Demo & next steps

[@shinchima]

Give this man a raise. Good speed and complete. 9.99/10 - nobody gets a 10

[@Supabase]

[@HashimWarren]

so the scale is from 0 - 9.99 🙂

[@shinchima]

@@HashimWarren absolutely!😉

[@rembautimes8808]

Excellent tutorial, so well explained. Thanks so much

[@dheerajs2838]

this is one of the best video came out of Supabase. Please do more such detailed videos. thanks

[@gregnr]

Good to know this style is helpful - thanks for watching!

[@sgruhier]

I had this in my todo, it's mind blowing. Well detailed, great speed, tis is amazing. My only regret is not have watched it before. Thank you so much

[@brucewayne2480]

One of the best tutorials I've seen in my entire life. Everything is clear, no shortcuts, no analogies, no abtractions. we learn a lot of things from different fields along with production best practices. This is how tutorials should be. Thank you so much ! Looking forward to learn more from you

[@Mburdo511]

This is truly an unbelievable tutorial. College professors could learn a lot from you. The fact that this quality is free is mind blowing

[@JohnBucy-rz8qo]

Joining the crowd - this is one of the best tutorials I have ever seen (and I have seen many). Great Job!

[@JohnBucy-rz8qo]

And the first one I have ever commented on

[@sumodd]

Please do more videos like this, truly amazing work. This helps me prevent a whole lot of headaches. I love Supabase 💚

[@Supabase]

We love you too 💚

[@TheStabacco]

Incredible clarity! More like these please.

[@mtin79]

Fantastic in depth walkthrough with code examples and reasoning behind implementation decision. Helped me understand supabase, its services and architecture and how things fit together much more. Thank you!

[@MrArbitraryNumber]

Man you are consistently saving me when I hit a wall on my projects. Thank you!!!!!!

[@benjaminw9624]

This an amazing guide. Like absolutely amazing, bravo.

[@gregnr]

Thanks! Don't hesitate to give me a shout if you have any questions/issues

[@hawarhekmat1174]

Next time a recommendation system.

[@gregnr]

Great idea!

[@BaileySimrell]

Yes please

[@zzej]

This is the best video tutorial I have ever seen.

[@ferrychandra8585]

Thank you for the great video!! Would like to see more videos on implementing Supabase using Python (not sure about the demand actually) if possible. :)

[@2an_sound]

Thank you so much for this tutorial! You are an amazing teacher

[@MachineLearningZuu]

Superbase. I freaking love you. Long live the king

[@shaharnechmad2586]

This is one amazing video. Thanks so much! One suggestion, will be super cool to have aversion if this video using langchain as well. There are a lot of great benefits using it instead of going directly to Open AI (like the ability to easily switch or use multiple model providers)

[@swissturtel]

Thank you very much - you are such a great teacher 🧑‍🏫

[@ipranay]

Vaults looks cool! More more more!

[@gregnr]

Comes in very handy in some situations 👍 thanks for watching!

[@rembautimes8808]

There were so many parts to like in this video, my favourite was how to extract the authorisation headers in making the call to a REST endpoint. Will probably implement the endpoint in python with Fast API rather than Deno. 😂

[@funkdefied1]

This video is PERFECT

[@NixBiks]

What an excellent video! Amazing work - I love all the "rabbitholes" which are all very important. I have two questions though; instead of using Supabase functions, one could use NextJS Route Handlers, right? Also; are there some open source alternatives to OpenAI LLM that could easily be integrated instead? Thanks for this video!

[@samanderson4881]

Good tip. I think it could be done, seems like the edge functions on the free tier time out when doing the calculations.

[@theitaliandev]

Amazing video! Thank you. I have a question: best way to set up multiple supabase projects locally using docker?

[@Supabase]

That would be through using Supabase CLI. supabase.com/docs/guides/cli/local-development

[@nikitastriuk]

Great stuff, thanks a lot! One question. What's the point of deploying Deno edge functions (and calling them with pg_net inside postgres) instead of simply using Next.js actions for processing files after upload? It adds a lot of complexity imo. Any real benefits?

[@nishanthg5532]

Brilliant 🥂

[@jeffshatton]

Great tutorial! Do you start running into problems with chat conversations as time goes on ... given you are including all previous messages and the limited window that OpenAI provides? How do you handle that? Just truncate it?

[@PavelLang]

It's brilliant. Just keep me breath outhht between important pieces of code! …next time! Dynamic of video is really good - but pieces where I need to learn something new, wish to look for references, sources… I cannot ever hit space… tracking back 🙂 BTW Thank you for so great tutorial! 🙂

[@DouglasHewitt]

I followed the entire tutorial, very good, thank you for this. I am a beginner with nextjs and supabase, two things that I can't get working at the end: my supabase does not create the sections and therefore embeddings when a file is uploaded, I guess I missed something with either the migration or edge function? also, the chat doesn't work because CORS blocks it when its coming from the supabase cloud, how do I configure CORS on the cloud dashboard?

[@jonathanma4206]

This is great. What changes would need to be made to use this with an open source model like Mistral or Llama 2? Is it just whatever model library is used and the embedding model that goes with it?

[@maryamashraf6370]

Hey did you get any further with this? I'm building a similar model using Mistral 7B - would really like to hear how you went about with using a local llm

[@davidho2901]

Without this video, how could we possibly learn to do this? 😢

[@Lamevire]

interesting video!. so the whole reason for using RAG here is to minimize the token inputs when eventually passing it to GPT? (also maybe getting more accurate results because of using a specific embedding model that's better than GPT)

[@OviDB]

First, thank you! One question, how does one go about debugging the functions defined as database functions?

[@yorgohoebeke]

Amazing tutorial! Could you make a similar tutorial but for using supabase with AI agenst (+ RAG) that use function calling. For example, how to create a chatbot that can add tasks to our to do list or complete tasks on our todo list.

[@tamsssss6765]

Would love a video on how to easily migrate supabase ssr w/ this! =D

[@tamsssss6765]

This is exactly what Im looking for! Thank you! Now only if I could get it to work locally =(

[@gregnr]

Glad it resonates! What issues are you having locally?

[@tamsssss6765]

@@gregnr I think it's the Deno? I keep getting errors like these "Type error: Cannot find module 'common-tags' or its corresponding type declarations." even if I've installed them. =/

[@tamsssss6765]

nvm I got it!@@gregnr

[@gregnr]

@@tamsssss6765 got it - just to confirm, are you getting those errors at runtime, or just in your editor (ie. VS Code)? If it's in VS Code, can you double check you have the Deno extension installed? Without that extension, VS Code doesn't handle Deno dependency management correctly.

[@saminamanat]

ctrl shift i just blew my mind

[@samanderson4881]

Very good tutorial. Only problem I have is that I don't get embeddings generated for every item in the documents_sections. I followed the code to the letter and it only generates the first 5 embeddings.

[@eleah2665]

Hi not-Jon, this looks good. Thanks.

[@gregnr]

Thanks for watching! Let me know if you hit any road blocks.

[@JonMeyers]

I agree! Non-Jon is killing it! 💯

[@learn-with-dr-sajjad]

Thanks for making this video for my favorite platform. I have followed it along and ported this method to use Google gemini api but I am having a weird problem. In chat function in the part where we add injectedDocuments to the system prompt. It runs fine for the first time, i checked by console.logging the completionMessages and all injected Documents and system prompt is added to the first prompt by the user but for all later messages neither the system prompt nor the injected documents are added and are not added to the user messages. However strangely enough the output from the gemini pro clearly indicates that it is getting the context and gives spot on replies. (I compared with the output from gemini pro without any context and answers were way different). Can somebody guide me if this is default behavior of vercel's ai sdk or there is problem with my code?

[@stonedizzleful]

How would you handle this if you actually wanted to reference the document/location where the RAG has pulled the info from (ie. like a references list on the front end)?

[@gregnr]

Yep this is a great question. We are actually in the process of bringing this type of functionality to the Supabase docs via Supabase AI assistant. The strategy more or less comes down to: 1. During the RAG prompt injection step, prefix each section with a heading (or id, link, storage path, etc) that references the document it came from 2. As part of the initial prompt, ask the LLM to insert references to these respective section headings throughout its response 3. On the frontend, parse the response coming back to extract these references, replace with a [1], [2], [3], etc, and add them as footnotes

[@stonedizzleful]

Sweet that makes a lot of sense. I pulled something similar together using pinecone but found I was double handling a lot of the prompt injection and then parsing the references. The way you have described it within the Supabase framework makes a lot of sense.@@gregnr

[@GilbertZhuo-uv1ux]

can i check for the generate embeddings part why we need to remove the javascript elements from markdown thanks!

[@adamtesla284]

I want to integrate supabase with my flutterflow app but the problem I'm facing is that I dose not allow to present user dispaly name . Any solution ?

[@cjpettigrew]

I think something has broken with the repo. The Chat function for example no longer deploys (i have pinpointed it to the AI library import from Vercel) Can you or anyone else reproduce this?

[@user-xd6ut8vm4d]

Thank you so much one of the Best Tutorial. Query - when we are using cloud based option means files uploaded to supabase server and embeddings as well. Just want to confirm how secured our documents will be and can we use for financial and health care files as same application. Really interested to signup with Supabase if this query is solved.

[@Supabase]

Supabase can be HIIPA compliant with certain plans, so it is safe to store those types of information. supabase.com/blog/supabase-soc2-hipaa

[@user-xd6ut8vm4d]

Thank you so much for your response. Request to please share direct link or email id where I can ask more queries for any further doubts@@Supabase

[@kirso]

One thing when reseting the DB because of Todos, there is a directive how to do it locally but not via the cloud. `pnpx supabase db reset` doesn't work unfortunately and I can't find it in the docs.

[@sumodd]

Getting this as well... did you figure it out? Edit: actually, here is what I did: 1. npx supabase db reset --linked 2. Deleted 'files' from storage in cloud. 3. npx supabase db push. Both of my migrations (the files and documents) were applied.

[@kirso]

@@sumodd sorry I accidently replied to a wrong video on another issue 🤦🏻Actually the wrong was on my side, since db reset is for the docker, I think you just need to do db push

[@laggy9764]

if my data is confidentiel will i have t use gpt even so

[@JJ-sz8wf]

In the embbeding column 7 - 10 rows are empty, but the rest is filled. Why is this? Has nothing to do with the code, since the nummer of blank space is different each time.

[@nikhilsingh4329]

Hey, i am beginner i received api key and base url generated by my organization , this tutorial just include API key only directly by open ai , I need tutorial which can help me to create the chat bot with my API key and base URL can anyone suggest me tutorial or code base ?

[@rydonahue1]

Couldn't a lot of these edge functions just be handled by API routes since you're using Next?

[@hatimmoaiyadi8388]

Please guide me to create a logic to upload excel and pdf files

[@marksughnenter526]

can you do this in python?

[@JT-oh6th]

Great tutorial but if you actually deploy this to supabase the CPU time is SO restrictive the embedding pipeline doesn't work. 🤦‍♂

[@hussienhassin7334]

anyone facing could not Auth user when trying to sign up?

[@Mavesound]

Looking for a tool to get text from my pdf. Is this possible aswell?

[@gregnr]

PDFs have notoriously been difficult to pull text from in a sane way (because there's lots of variance between PDFs, and some PDFs embed text while others are just images). One solution we're working on is using GPT's new vision model to extract the text - still WIP right now, but stay tuned!

[@Mavesound]

@@gregnr thought about that aswell. All libraries I tried worked so bad

[@moritz8618]

hiawd

[@moritz8618]

hallo

[@namesare4fools]

its good & detailed, but why can't you make this more straightforward to setup

[@gregnr]

hey, which parts did you find were slow to setup?

[@namesare4fools]

@gregnr why cant i just run git clone, then run npm install and insert my env vars and run.. also do you have to use deno ?

[@gregnr]

@@namesare4fools if you don’t care about the details, you can 100% just clone the repo and run it. Yes, Supabase’s edge runtime is built on Deno - you wouldn’t run this in vanilla Deno yourself though, instead use the supabase CLI to serve the edge function as shown in the video/readme.

[@user-lu5wm7vu1x]

worst ever tutorial. so unclear, it cannot be more confusing

[@sidhuk3128]

Thanks for a detailed video. but why did you use @supabase/auth-helpers instead of @supabase/ssr as the docs recommends.