Build gen AI features powered by your data with Firebase and PostgreSQL

Learn how Firebase Data Connect uses PostgreSQL to increase velocity at every stage of app development including powerful new gen AI capabilities with Genkit. Learn how tool calling and structured output can make programming with LLMs feel more like programming with any other API. Learn how to safely use your application's data to drive AI-powered experiences.
Speaker: Michael Bleigh
Watch more:
Check out all the Web videos at Google I/O 2024 → goo.gle/io24-w...
Check out out all the Firebase sessions from Google I/O → goo.gle/io24-f...
Subscribe to Firebase → goo.gle/Firebase
#GoogleIO
Products Mentioned: Firebase Data Connect, Firebase Genkit, Firebase App Hosting, Remote Config
Event: Google I/O 2024

[@mralek5720]

Amazing demo! Really impressed by the seamless integration between Data Connect & GenKit and the associated dev tools. This is hands-down the easiest way to integrate RAG into an app I've seen, well done! 👏

[@michaelbleigh8737]

Thanks, that's kind of you to say!

[@rahulgupta7720]

Very Informative tutorial! Genkit is truly the future of building AI driven apps. Would love to get much more tutorials in the future

[@missedme0063]

VERY cool demo, thank you!

[@robinryf5]

Very awesome introduction! This is doable for everybody who is interested and eager to learn. Thank you. I wonder what other use cases there could be besides search 🤔

[@kevingrondin974]

@Michael Bleigh you can share your code on your github ?

[@michaelbleigh8737]

A version of the Compass app will be published to GitHub in the coming weeks (I think)...stay tuned! You can also go through this codelab which covers much of the same content but uses Firestore (since it's generally available): firebase.google.com/codelabs/ai-genkit-rag#0

[@samario_torres]

is this easy to integrate with cloud firestore? is there a simple way to connect my firestore database to data connect and have both databases collecting data?

[@michaelbleigh8737]

Firestore integration is something we're looking at for the future. You can vote on it here: firebase.uservoice.com/forums/948424-general/suggestions/48434618-cloud-firestore-datasource

[@samario_torres]

@@michaelbleigh8737 awesome thank you for replying, just voted!

[@nickjunes]

Is the error you're getting because you already did a vector comparison search with your searchDestinations function? If not why do you need a retriever if you already did a vector comparison search in searchDestinations with plain text? It seems like the vector comparison search can already understand plain text. What am I missing?

[@michaelbleigh8737]

The error was actually because I forgot to alias deatinations_embedding_similarity to "destinations". The database doesn't store embeddings as text or have the ability to natively query text against vectors -- instead you have to use an embedding model to convert text into a vector then compare vector to vector. That's what _embed does.

[@nickjunes]

@@michaelbleigh8737 So what does destinations_embed_simlarity do then? At 26:44 you pass it a query with plain text and are able to get some results from the database. How can you do that without first converting the query into a vector?

[@michaelbleigh8737]

@@nickjunes destinations_embed_similarity performs a vector similarity search, compare_embed generates an embedding from the provided query.

[@vijayjangid8967]

Finally!!!!!!!!!!!!!!!

[@mohamedkarim-p7j]

👍

[@darthvader4899]

what's that .prompt file? is it special to idx?

[@michaelbleigh8737]

It's part of the Firebase Genkit framework, see firebase.google.com/docs/genkit/dotprompt

[@darthvader4899]

@@michaelbleigh8737 really cool. Is this going to be standardized?

[@ruandupreez6144]

His shirt?

[@__user__name__]

Despite the bit of dirt on his shirt, he carried on without a care. That's what kept me watching the entire video and earned my respect for him.

[@k-c]

Your priorities?

[@JohnMcclaned]

I'd trust a dev with stains on their shirt over a corporate goon any day of the week.

[@jerryf196]

how do you generate the embeddings? You ran a script but does it use the vertex firebase SDK?

[@mbleigh]

The embeddings were generated using the Vertex AI API through Firebase Genkit.

[@jerryf196]

@@mbleigh Thanks. I got it running with my project. Is it possible to add words that refine searches. For example: good exploration board game that plays best at 2 players. In my firestore data, I have a field that contains number of player count

[@jerryf196]

@@mbleigh also for the dreamVacation.ts how do you run this flow? Like thiis? const response = await runFlow(dreamVacationFlow, 'Some input');

[@mbleigh]

@@jerryf196 runFlow(dreamVacationFlow, {imageUrls: [...], description: "..."})

[@mbleigh]

@@jerryf196 to do something like that you'd probably want function calling - you use defineTool to create a tool with parameters like "maxPlayers" then map that to a Firestore combined vector + filtering query. It looks like we don't have great docs for function calling yet for Genkit, will take note to improve that.