Google Vertex AI Agent Builder Tutorial

[@MichaelHoward-yv2py]

One of the best tutorials I have watched. Thorough yet easy to follow. You are clearly gifted in teaching.

[@Mr.Cool.Dragonpresent]

Simple and clear, no extra crap. Straight to business.

[@pimcore_warrior]

quality guide, greetings from 71

[@superfilipe26]

Hi, great video! I'm wondering if it's possible to automate the creation of this agent using infrastructure as code tools like Terraform. Are there specific modules or providers that you'd recommend for this task?

[@BiotechMat]

Thanks for sharing!

[@babusivaprakasam9846]

Fantastic!. Straight to the point.

[@ravendfj]

Hi, I tried to run the agent using authorized API at minute 13:317 but it asks "Access Request" because I did not provide token. How does it work for you please? Thanks.

[@ZerayacobMeshesha]

Hi everyone, im having an issue with creating the agent, after I select the display name and region, google continues to try to create it but never succesfully does so. Im wondering if anyone dealt with the same issue and has any fixes.

[@santosharakere]

Excellent ! Thanks for the video.

[@YunometaDeveloper]

but product quantity is not coming when i am testing it

[@pythonlover20]

Thanks for the great video. Noticed you selected "Allow unauthenticated invocations" for the cloud function. How do I use the cloud function in a tool when "Require authentication" is enabled ? I have tried adding cloud function invoke permissions to the dilaogflow cx service account but it's still getting 401 error.

[@ArchitectureBytes]

You will need 'Cloud Run Invoker' role. Please see the readme in github repo, have added a section on how to set this up.

[@pythonlover20]

@@ArchitectureBytes Thanks

[@sagar00000007]

Great video. I think in my workflow everything works. However, I need fallback to Gemini LLM after trying answer from my data store. No matter what I have tried like creating separate agent, calling it inside the workflow, the first call to tool just exists saying no answer found instead of trying to fallback. Do you have any recommendation?

[@ArchitectureBytes]

In your (main) agent instructions add something like (maybe at the end) - If the user asks any other question try answering it, instead of saying you don't know.

[@MrAlex0803]

Could I make it call a external not google service like jira to make an api post to create a ticket?

[@ArchitectureBytes]

Yes certainly, you can invoke a Google Cloud Function - and from there make call to an external service like JIRA etc.

[@yesweet]

thanks for this excellent video example, very helpful👍. Could you please show us how to call that agent via an API? I need to call it from our own message platform, but fail to handle the auth, that permission and service account configuration are confusing and no doc about this use call.

[@ArchitectureBytes]

Please see the Readme file in the Github repository (link in video description). Have added a section: Invoke Agent via API call. Hope that helps!

[@yesweet]

@@ArchitectureBytes Thanks! I post an issue in that repo following the error message I got.

[@shubhamGarg02]

can i upload data in csv or tablular format to my data store?

[@ArchitectureBytes]

A csv format with proper headings should work. HTML is also one of the accepted formats.

[@johnleryvillanueva766]

i got error when i put an schema same as you do Draft 2020-12 schemas are not yet fully supported.

[@ArchitectureBytes]

You mean when you add Open API Schema like at 6:10 ? Are you using the same schema as used in the demo or one of your own? If it is your own does it have a version at the top - and what is it?

[@johnleryvillanueva766]

@@ArchitectureBytes yah same schema that you used. i just copy from your source code then paste it.

[@ArchitectureBytes]

@@johnleryvillanueva766 Please use the yaml version of the OpenAPISchema. This file is available in repo now. Hope that helps.

[@MohammadMajed-hu4ms]

Great guide! Is it possible to send hidden params not inputted by the user and the agent will understand? Dialogflow used to support $session.param... when we set it from df-messenger.setQueryParameters Now when starting with VertexAI Agent and using the df-messenger integration, I am not sure how to do that

[@ArchitectureBytes]

You would add that information to Agent instructions. When invoking a Tool (eg. OpenAPI) - agent passes parameters based on available information whether provided explicitly by the end user or available in the instructions. Your Open API Schema definition (eg. param name, description) should be well defined. Your instruction could be something like: Always assume that City name is Chicago. So, if your Tool (API) expects City as parameter, it would get Chicago as parameter, and agent will not need to ask for this information from the end user.

[@MohammadMajed-hu4ms]

@@ArchitectureBytes Thank you for the reply! Will this work if the parameter is not a predefined one? Let's say my system where the chatbot is deployed on decides the City Name and I would like to send that as a parameter. How can my instructions be something like "Get the City name from the System" and how can i send it? My user interface is using integration

[@ArchitectureBytes]

You can instruct the Agent to fetch City name by invoking a Tool (API). Please watch the video completely - it shows how Agents are instructed to invoke Tools. I also suggest that you watch this video (AI Agent Architecture) to understand how Agents work: kzbin.info/www/bejne/fHa4Z2B_oZ2doNk

[@shashwatsri4547]

Hey do you know if the agent app type is able to be integrated with telephony service? If not direct integration through google is there any other way I could do it?

[@ArchitectureBytes]

Yes sure! Pls see this video: kzbin.info/www/bejne/h6LUiJ94ncqSiNE

[@nkarthik44]

Is is possible to get the inputs like phonenumber , casenumber etc from customer and pass it to function in Google Vertex AI?

[@ArchitectureBytes]

Yes, in the tutorial we are passing productId to the function - the OpenAPI schema defines this interface.

[@NMISDEEPAKPOOJARY]

can we train the model with 2 data stores , cause I tried and its not working

[@ArchitectureBytes]

Yes, it works. You could create separate Tools of type Data Store - for each Data Store, and have your Agent Instructions point to them. Make sure Data Store is connected to the Application. Remember that upon creating a new Data Store - it takes some time to import/index the data. So, agent can use it only after that.

[@Inside25]

Hi, thank you for this valuable video. I am having trouble finding help., Dilaogflow Messanger when Try 13.48 says "Somthing went wrong" not work

[@ArchitectureBytes]

Your question is not clear to me. Do you get an error when you start chatting with the bot Or at some other point?

[@ManikantaKomma]

i am getting hallucinations results in chatbot in agent app, how will fix it, please let me know

[@ArchitectureBytes]

Please make sure that your Agent Instructions are clear and accurate. In addition you can provide Examples 11:28. The above two are the most important measures to take. You can also play with Temperature settings under Settings > Gen AI > Temperature. A lower temperature settings reduces randomness of tokens selection (therefore response).

[@ManikantaKomma]

​@@ArchitectureBytes i get JSON schema, have i modify in this schema & how i get url for this schema, like at 6:07, and at 7:15 how will get python code (flask import) & i am working on multiple tools on the same agent, is it will be work?

[@ArchitectureBytes]

@@ManikantaKomma In the Open API Schema The 'url' is url of your Google Cloud Function (without function name at the end). Path has the Cloud function name. OperationId is the method name in function. If 'flask' dependency is not automatically handled, then in the requirements.txt file present alongside the CloudFunction python file - add a line 'flask'. Of course you can use multiple tools in an agent - this is shown in this tutorial itself.

[@ManikantaKomma]

@@ArchitectureBytes i given multiple examples but and getting hallucinations results and getting Failed to generate response, Temperature option not visible in Gen AI

[@ArchitectureBytes]

@@ManikantaKomma What's there in your Data Store? Is it 'Cloud Storage' with unstructured documents (pdf?) from a bucket? By hallucination - do you mean it gives you incorrect information? Or it says it doesn't have the information (which is not exactly hallucination)?

[@LigisStories]

can i upload data in pdf format to my data store?

[@ArchitectureBytes]

Yes! This tutorial demonstrates that.

[@LigisStories]

@@ArchitectureBytes Thank you so much.

[@StefanoGraziotin]

I followed all the steps you showed in the video step by step. it gives me an error when I ask for the quantity ( " 404 Page not found Error: Page not found The requested URL was not found on this server. "Incorrect type. Expected "object" ) and when I try to integrate dialogflow. could you help me?

[@ArchitectureBytes]

Please see the Readme in github repository linked in video description. Did you update the OpenAPI Schema with your own cloud function URL, name and operationId as suggested?

[@StefanoGraziotin]

@@ArchitectureBytes I fixed the problem with the getProductQuantity, but i can't integrate dialogflow. can you help me with this?

[@ArchitectureBytes]

You mean integrate with the Dialogflow Messenger? What problem are you facing?

[@עמנואלחסן-ח3ס]

hi. i have error Error 400: redirect_uri_mismatch after i follow your video. please help as soon you can. thank you

[@ArchitectureBytes]

At what point (video timestamp) do you get this error?

[@MariuszKen]

I love this accent.