Flowise AI Tutorial #11- Custom Tools, OpenAI Functions and Webhooks

[@toppu10animes]

This video is incredible, I didn't see a tutorial teaching how to do this anywhere on the internet

[@leonvanzyl]

Wow, thank you!

[@aidentenvelden7775]

Awesome to have a fellow South African as a teacher!!!!!!

[@JulianHarris]

Super helpful. It’s really fantastic, demonstrating the power of natural language at each of these steps. Even 12 months ago, so many of these steps was incredibly difficult if not impossible.

[@leonvanzyl]

Absolutely, this tech is incredible.

[@jbhearing6392]

Love the videos! It would be great to get a video on appointment bookings through AI chatbots, Google calendar API, calendly etc or lead data collection (name, phone number, email etc). I think this would be a really popular video. Keep up the amazing work!

[@leonvanzyl]

On it 👍

[@krasimirivanov6627]

Hi Leon, your videos are great! For me it will be very useful to see a tutotrial on using AI plugings in Flowise. If this is interesting for the rest, it might be a nice video to make whenever you get the chance. Thanks a lot for the super valuable content!

[@leonvanzyl]

Thank you for the feedback! Any specific plugin that you'd like me to cover?

[@krasimirivanov6627]

@leonvanzyl Thanks for the reply, Leon. I am more interested in the general way of setting it up. I noticed in Flowise that there is a standard template that uses the Klarna plugin with post and get APIs. I assume all plugin setups will be similar in Flowise. An idea for a plugin/usecase could be Clearbit, to extract data about a company (potential client) and evaluate the attractiveness of the lead. But really, any plugin tutorial will be useful. I couldn't find any similar tutorial, so perhaps another video of yours to land in the Flowise documentation.

[@tekhnocafe]

​@@leonvanzyladvanced data analysis would be rhe best

[@ThePandaGuitar]

This video is dangerously valuable. Thank you so much for sharing 🙏

[@leonvanzyl]

You're welcome

[@bourbe]

Hello, hereunder some interesting subjects : > Qdrant langflow & flowise intégration > Voiceflow langflow & flowise intégration > Botpress langflow & flowise intégration > Langflow vs flowise, which is the best ? Thanks again for all your dedication for us !

[@leonvanzyl]

Excellent suggestions. Thank you. I am working on Voiceflow and Botpress series with Flowise integration.

[@navturn]

Awesome. Thank you and continue the good work! Note that I didn't need to restart Flowise after Custom Tool modification (Flowise v1.3.4)

[@leonvanzyl]

Thank you!

[@rajeshjsl]

Great Video Leon. I have been seeing this problem with Flowise w.r.t memory as well. Even if the memory tools like zep,motorhead,db’s only return say like last 6 messages, flowise keeps all conversation in memory and sends it to chatgpt which defeats the purpose. I can’t restart flowise if some of my users are in a chat session. It’s definitely not ideal for multiple user scenarios. Restarting flowise fixes the problem, but this is a big deal for proper functioning.

[@leonvanzyl]

Thanks Rajesh! Does that issue only occur when using buffer memory?

[@rajeshjsl]

@@leonvanzyl Buffer memory doesn't have the "max window" feature, only zep, motorhead, etc have with conversation chain. So basically, I don't want to send all conversation to chatgpt. Only last couple of messages. On the backend I can see that zep,motorhead, etc is sending the 6 messages to flowise, however, flowise has previous messages in memory and sends all!

[@leonvanzyl]

@@rajeshjsl ok, I'm with you. I wouldn't use Buffer memory for a "production" situation. It's mostly for testing.

[@yorgohoebeke]

This is super helpful! Thank you so much! I was thinking of using buildship but this is much better as I can self host flowise. Could you make a tutorial similar to this one, but on how we would use it with flutterflow and pass the authentication token to the flowise flow ? Basically authenticated user on flutterflow asks ai to add/edit/delete todo -> flowise -> supabase with auth token (can only add/edit/delete todos for themselves)

[@muneerAbro]

very helpful video, learnt properties😍

[@toppu10animes]

YOU ARE THE BEST EVER!!!!!!!!!!!!!!!!!!!!!!!!

[@leonvanzyl]

No, YOU are the best! Thank you for the kind message 🙏

[@itayba76]

Thanks Leon! Your tutorials are very helpful. When building tools and flows in a different language, should I write the tool descriptions and other system instructions in that language, or in English, assuming the model will better understand the instructions this way?

[@leonvanzyl]

Wow, thank you very for the support! That's an excellent question. I've found that I get really good results by prompting in English. Not all languages produce good results. You can still instruct the model to only respond in a certain language though.

[@itayba76]

@@leonvanzyl Thank *You* for sharing your knowledge! The question is how do I make the agent/model use the correct names of business entities. For example organizational units or sub-divisions. Should I prompt in English but use localized entity names? if so, should I enter these in quotes? Is there a point in creating an organizational glossary with definitions and localized names?

[@spencermroczek]

Love this, I signed up for Flowise cloud, using your link. I have been waiting for that service forever. Also, video idea: with Flowise cloud, it looks like the ability to write a file to a disk is gone. Is there a way to still use FlowiseAI to create a file that could then be downloaded by clicking a link?

[@leonvanzyl]

Thank you very much for using the link 🙏. I'll look into your enquiry.

[@pedroespecial9999]

Hey Leon - thanks for these Flowise videos, they are really helpful. I'm wondering how you got to see the Console view of exactly what was going on behind these scenes? I've played with Dev Tools etc. and can't find anything as clean as you showed in this video. What you showed @6:00 - 6:40 is exactly what I need in order to help me understand what these Flows are doing. It all feels a bit 'black box' to me right now and I need to take it apart to understand it properly. Thanks Leon and keep up the great work 🙂

[@leonvanzyl]

You're welcome! You can set Debug to True in the environment variables file. Or run the start command with this attribute: npx flowise start --DEBUG=true

[@myandrobox3427]

Flowise and YOU both are Excellent💯💯

[@leonvanzyl]

Thank you!

[@ChatbotWhizAmir]

✨AWESOME LIKE ALWAYS✨ Thank you Leon for doing this

[@leonvanzyl]

You're welcome 🤗

[@sitedev]

Great video. I love the idea of using other chatflow endpoints as tools. I can imagine a ‘build your own adventure’ style logic tree where each node is a chatflow configured for a specific scenario. Combined with a vector store you could construct an ‘engine’ with memory of ‘events’ (node outputs) that could be a grounding reference for all other nodes/chatflows . *head spinning*

[@leonvanzyl]

I love the way you think 😁

[@rajeshjsl]

Good luck with that as currently flowise has problems with memory, it keeps everything in memory until you restart. And you can’t keep restarting if multiple people are using. They seriously need to look into this.

[@OfficiallyArtificiallyInformed]

​@rajeshjsl thats what vector storage is for....

[@jonfirman7766]

what about creating webhooks to move between different chat flows? in other words, how do we post a variable from one chat flow into another?

[@hlankhine]

Thanks, Leon. As always, your videos are easy to follow and very helpful. Quick question - is it possible to bring Google Sheet data to FlowiseAI? Currently we have CSV as an input source but I would like to get the data table from Google Sheet.

[@leonvanzyl]

You're welcome 🤗. Yes, since you can call API endpoints from the custom tools, the sky is the limit. There are API endpoints for fetching data from sheets. Make.com might even have a template for it.

[@homeapps]

Wondeful Leon, you make the best Flowise and AI video tutorials. When you said we can add multiple tools to the same OpenAI Agent, in your other videos, can we also add multiple Flowise Nodes to Agents or Chatbots? Thank you!

[@leonvanzyl]

Thank you very much! You'll have to experiment. Not all inputs accept multiple nodes.

[@CarlosChang11]

Great video! i didnt know i had to restart flowise after modifying the agent.

[@leonvanzyl]

Thank you! Maybe I'm just unlucky, but the changes wasn't taking affect after changing the tools 😊

[@romanioamd5319]

How to import finished json file for custom tools file into Custom Tools? Or what directory the finished ones are stored in window select tool?

[@reinhardtjohanneswiese2650]

Dankie Leon! Is dit moontlik vir flowwise om konteks te hê oor die spesifieke persoon wat die boodskap stuur na die chatbot toe? Met ander woorde as ek byvoorbeeld ’n whatsapp chatbot opstel met flow wise wil ek graag weet van watter nommer die boodskap gekom het sodat ek user spesifieke data kan gebruik binne flowwise? Soos in jou voorbeeld, hoe sal ek username kan oordra sodat elke user sy eie to-do lys sal hê? Ek hoop my vraag maak sin. Dankie!

[@leonvanzyl]

Groot plesier! User auth and management is not included in Flowise. You would typically do Auth outside of Flowise and then pass the context via the Flowise API.

[@OfficiallyArtificiallyInformed]

Best flowise teacher on youtube. Is it possible to make the chatbot voice enabled with eleven labs? If so would this be done through a webhook or tool?

[@leonvanzyl]

Wow, thanks! You won't be able to do that in Flowise itself. You'll have to build a custom app and then call the Flowise API from it. Here is a tutorial that I made to do just that. kzbin.info/www/bejne/q5nUe6l_iqqGbc0si=PZ_AppYgFMl9ymH1

[@360vragency9]

Great work, you've earned one new subscriber! I have a question, if I want to deploy the agent in my application and allow my users to fill in and connect their google sheet, how much work do you think it would take? Thanks

[@leonvanzyl]

Thanks for the feedback and sub. Your example seems very similar to the example in this video, but just with a few more fields?

[@360vragency9]

@@leonvanzyl I may have misspoken, how do I get the user to add the connection to his google account and excel workbook via the front end of my application? Would this require a lot of work? Thanks

[@adriaanb7371]

Is there a way to pass the vector representations between models, instead of having to process it into natural language. Could be a lot more precise, quicker, lossless etc

[@GoodMoneyMoneyy]

Fantastic video as always, is it possible to pass an image as an input ? Also I realize that with this tool you can create Agent cohorts with multiple high level agents specialized for certains tasks as you can connect them each through APIs.

[@leonvanzyl]

Thank you 🙏. There is no way to upload images from the chat window. Only text.

[@GilbertMizrahi]

What's your use case with an image? You cannot pass an image, but you can pass a URL of the image and then do something with it. I will give you an example, although it doesn't necessarily involve Flowise. You have a chatbot that allows you to take pictures. You create a flow that after taking the picture of a doc, it performs OCR. Then it sends the raw text (free form) to an OpenAI GPT3.5 API to structure that data and finally to save that structured data to a Google Sheet (or a database). I can make a video showing that in action if people are interested. Depending on the flow you may use Flowise for some of the workflow.

[@GoodMoneyMoneyy]

@@GilbertMizrahi that's interesting, I'm currently using Zapier to automatically upscale images from a google drive folder, I wanted to do something similar but using Flowise. I'll try to recreate my zapier flow in flowise.

[@breathandrelax4367]

Hi Leon , Thanks for your videos! I have a question, If you would create a structured chat, what node would you use to create a sequence of predefined questions ? My purpose is to force the llm to ask certains questions in a certain order in a similar way of voiceflow . thanks for your answer !

[@leonvanzyl]

I would suggest that use botpress or Voiceflow in that instance. What you're describing is a traditional kind of chatbot. Newer bots are more open ended.

[@breathandrelax4367]

@@leonvanzyl sorry for the late answer thank you for answer, I am iterating in different approach I have tried voiceflow, I'll give a shot to bot press thank you for your kind answer 🙏

[@jacobriedel5326]

Great stuff. I am struggling to take the output from the OpenAI Function Agent and use it for another execution within the same flow. Here is some background: I have 3 custom tools hooked up to my Function Agent. Since the application I am building is not utilizing Flowise AI for the chatbot experience just the LLM/Function agents, I have to make three separate calls to my Flowise endpoint, where each call specifies the function I want to run. Although this is fine, I was curious if you could chain together calls to the function Agent where the output of one custom tool will be the input into another custom tool call...... Let me know if you have an advice for this.

[@leonvanzyl]

Have you considered creating seperate flows and then calling the flows from the custom tools? I have done that myself. The flows all have their unique functions,.systems prompts, etc. So it's kinda useful to have one master flow which calls tools based on the prompt, and the tools themselves just call other flows. Maybe this is not ideal for you use case either.

[@jacobriedel5326]

Okay thanks for this. I am currently doing this. I have multiple flows just did not know if that was the best way to design this. This design is okay for my use case but I was not sure what is right or wrong. Thanks! @@leonvanzyl

[@maniecronje]

Hi Leon just revisiting some older videos you posted, I would like to execute a JS function whereby if the user types something like “open my KZbin channel” in the prompt then it will execute a function where a new tab opens in the browser and opens a specific url defined in the JS code, how may I achieve this behaviour would you use tools in FW or JS function , appreciate your opinion on this one , thx 🙏

[@leonvanzyl]

Hi Manie, the code in the custom tools actually run on the server, so it will not have access to your browser. This is something that other people requested as well, so maybe the FW team will implement it

[@maniecronje]

@@leonvanzyl thank you 🙏 would be great if you can execute ion the client side too 🤞 FW add this feature.

[@redbaron3555]

Amazing!!! Thank you very much!! Can you make a video of flowise flows calling each other for an ai assistant chatbot accessing phone books and to-do lists?

[@leonvanzyl]

Thank you 🙏. You can create custom tools that call the endpoints of your other flows. Hence linking them all together that way.

[@leonamnagel9999]

thank you, pls keep continue with these videos!

[@leonvanzyl]

Will do, and thank you for the feedback

[@dniliveact]

Bro I love your tutorials, you're awesome! Thank you for all the value you bring to us! Can we do something like 'user intent' custom tool, let's say a user wants to cancel an event, then flowise will ask his name, email and cancelation message to store on gsheet?

[@leonvanzyl]

Absolutely. I actually have a few videos on the topic. Check out the appointment booking agent video.

[@dniliveact]

@@leonvanzyl Ok another question, how do we integrate that with QA retrieval tool where we want to have the retrieve data from the vector database? I'm pretty lost like I don't know how to connect those tools with our retrieval QA chatbots

[@zakaria20062]

thanks for video , have any chance to FlLma2 function calling instead ?

[@leonvanzyl]

I haven't had much luck with Llama 2 function calling.

[@MeinDeutschkurs]

Hey Mr! That’s genius!!

[@sermonkokosov2137]

Please tell how to search the Internet and process documents using a local neural network. If you just replace it it won't work. Unfortunately, there is still quite a bit of information about this.

[@leonvanzyl]

I think the GPT models play best with agents. Will look into this and create a video once I find an opensource model that works for this.

[@sermonkokosov2137]

@@leonvanzyl Thank you very much for your answer! I'm looking forward to this video.

[@Remixdance01]

great video 1 quick question how can I add a prompt tempalte to this Open AI function agent?

[@leonvanzyl]

Thank you! You can't add a prompt template to an agent, BUT when you click on Advanced on the agent, you can set the system message 👍.

[@Remixdance01]

@@leonvanzyl did you have an email to send you a question ? I have a few problems trying to create a custom tool, when I declare my JS function the agent response with this error : Received tool input did not match expected schema and I dont know why

[@slddive9025]

all that just to push text to a google spreadsheet?! It's so early. When will I be able to just chat to my personal assistant bot and all these things are done automatically?

[@mehdibelkhayat5088]

Thanks a lot for this very good tutorial, As i use Ollama instead of OpenAI, it works well in general, but it won't call the custom tool, do you. have an idea ?

[@leonvanzyl]

Hey there. Function calling is a feature offered by OpenAI. It won't work well (or at all) with open source models. Things move fast, so I'm sure this will change in the near future.

[@mehdibelkhayat5088]

THanks@@leonvanzyl , but i just had a good response with groq api (llama 70b) and custom tools a minute ago, .. you must give it a try

[@mehdibelkhayat5088]

​@@leonvanzyl , thanks it works well with groq api, but i am limited to 3 variables, if more then a get memory buffer error

[@garymdmd]

I am doing this I a version of flowise on Render - any reason it should not work in that setting? I have postman working with Make and Sheets but cannot get the tool to work.

[@leonvanzyl]

I've realised that you actually have to restart the server each time you change the tools.

[@garymdmd]

@@leonvanzylI did restart the service on render but still not able to get and unfortunately render erased everything I had done but I learned that lesson before and had them backed up this time. So I decided to go again and try to install the dev server locally to seeif that would work rather than on the render platform. I actually got it to work there using hte same chatflow and the same tool and same Node code. I wonder why it will not work on the render platform?

[@113256]

how do i save the whole question as well as the chatbot answer on google sheet every time a user asks a question? this seems to only work to process certain keywords in the question asked

[@leonvanzyl]

Hi there. Yes, what you're asking for is quite different to how custom tools work. I'm not aware of a solution for recording the conversation history in that way, but I'll look into it 👍. You might need to use a memory store using Redis as an example.

[@TinyMagiq]

This is brilliant thank you so much

[@leonvanzyl]

Thank you 🙏

[@volodymyrliashenko1024]

Hi Leon, greate videos! I watched all videos about Flowise and LangFlow for two days. Could you please help how can I implement such feature: I would like to ask AI to save "this" information (which we just discussed) into database, so later he can use it. If this information already exists he should update it. I would like to achive like AI training in real-time.

[@leonvanzyl]

Thank you for the compliment! That's definitely possible, but involves a bit more coding than we covered in this video. Your JS code could perform an existence check and then perform the update or insert. I would probably move this logic into an API which the tool can call. Definitely out of scope for this series, but easy enough to implement 👍

[@simarpreetsingh6604]

Hi. Can you please create a video using retriever tool? This retrieval tool is connected with my pinecone db. I’m not sure how to tell my Chatbot to search my pinecone db for any questions. Please can you create a video on this. Thanks

[@leonvanzyl]

Hey there, check out the document chatbot video in this series.

[@sathyanarayananramamurthy5681]

Hello and thanks for this great tutorial, however i am getting the following error "404 Unrecognized request argument supplied: functions" All i added is just a return statement on the custom tool...Any help ?

[@leonvanzyl]

Try restarting the server. Seems I have to restart the server each time I change a tool.

[@sathyanarayananramamurthy5681]

@@leonvanzyl thanks for the response, i have done that multiple times, one thing i figured out is when i use OpenAI fucntion agent, i am getting that error, if i use any other agent other than OpenAI fucntion agent, its working, again it works only with limited functionality

[@asrulmunir]

As always... Awesome Flowise video 👍

[@leonvanzyl]

Thank you 🙏

[@leonamnagel9999]

Can you do an automation where it searches for information in a Google Sheets table and responds to the user? Could you make a tutorial on this?

[@leonvanzyl]

You can definitely do that. I wouldn't really be able to create videos for everything you can do with tools, but you can use the foundation of tihis video to read information from sheets as well 👍. I'm sure you can find an API endpoint for reading data from sheets, get it working in postman and then plug it into a custom tool 😁

[@sdcharly]

Hi Leon, I tried to use Custom tools with schema with a date field, it doesnt work. have you tried a date type?

[@leonvanzyl]

Hey there. What's the issue?

[@sdcharly]

@@leonvanzyl API post is not seen when a date datatype is selected. So you are it working? Also a tool edited a few times totally breaks. It's better to edit a couple of times with a restart and then build a new one

[@davidmaugrion5442]

Great video ! Could you make one on MS Team integration ?

[@leonvanzyl]

Thank you! There might be a webhook with Teams integration on Make.com.

[@aliszaa.w7914]

Why my console is not show any response?

[@leonvanzyl]

You need to run Flowise in debug mode.

[@traxtibidox5775]

You are unbelievable 🔥 I have no words 😃 People charge 10 000$ for 1 course like yours Please make buy me a coffee acc for your community, thank you very much you explain it 100% perfect !

[@leonvanzyl]

Thanks

[@orthodox_gentleman]

Dude you barely explained how to use tools

[@leonvanzyl]

Was there something specific you wanted to see?