How To Auto Update Your Knowledge Bases | FlowiseAI

[@leonvanzyl]

Hope you enjoy this video - I tried to add as much value as possible! You can down the Flowise and n8n workflows used in this video (link in description) as well as the sample data. Remember to like the video and subscribe 😊

[@nkofr]

Hi, thanks for the awesome content! Could you please create a tutorial showing how to replicate Claude 3.5's "artifacts"? Specifically, the ability to iteratively refine AI-generated results in an intuitive, user-friendly way. This would be super useful for refining AI automations and could help solve many real-world business challenges. I think it would resonate with a lot of the community! Thanks!

[@hosst]

If they ever make an Oscar category for amazing tutorials I'm sure Leon will take the first one. The amount of easily explained fountains of knowledge that help others to make the world a better place thanks to Leon's tutorials is truly amazing. Thank you Leon, really. I'm confident one day we will meet because I want to shake your hand and thank you in person.

[@leonvanzyl]

Holy smokes, what a cool comment! You've made my day 🙏

[@mohamed.hesham]

Me too

[@epokaixyz]

Buckle up for a rapid-fire rundown: 1. Sign up for FlowiseAI, Airtable, and Pinecone to access the necessary platforms. 2. Create an Airtable base with business listing information. 3. Build a FlowiseAI chatbot using the Tool Agent, ChatOpenAI, Buffer Memory, Retriever Tool, and Document Store nodes. 4. Connect your Airtable base to FlowiseAI using a Document Loader. 5. Set up an n8n workflow to automate data updates. 6. Schedule the n8n workflow to run at regular intervals using a Cron node. 7. Configure the n8n workflow to fetch data from FlowiseAI, process it, and upsert it to the vector store. 8. Consider filtering out static document loaders for increased efficiency.

[@MaliRasko]

Legend!

[@leonvanzyl]

Thank you so very much for the generous support ❤️

[@yagel1461]

❤

[@DresElMagnifico]

You are an absolute legend. Thanks

[@leonvanzyl]

Thank you for the support 🙏

[@WebAdaptive-t1n]

Fantastic tutorial! The quality and clarity are outstanding-one of the best I’ve seen!

[@leonvanzyl]

Wow, thanks!

[@IVAN-zg9vl]

@leonvanzyl Your work deserve to be commended, the clarity of your explanations and the the availability toward your community is something of unsual. Hope everyone keep supporting your work as educator and as pro.

[@leonvanzyl]

Amazing feedback!! Thank you 😊

[@gmcwhinney]

This was an extremely thorough and well executed tutorial, Leon. Thanks so much for taking all the time to put it together. You've become one of my favourite educators to follow on YT🙌 Keep up the good work

[@altonjones5271]

A bit late to this one , but what can I say - once again you hit it out of the park with another amazing tutorial! Baie dankie Leon !

[@leonvanzyl]

You're welcome 🤗

[@toi500]

Best video of the series so far.

[@leonvanzyl]

Thank you

[@practical-skills-school]

I was among those who asked. Thank you for reaction and such a detailed tutorial!

[@leonvanzyl]

Then this one was for you 😜

[@mohamed.hesham]

Thank you Thank you Thank you Thank you Thank you Thank you Thank you Thank you Thank you Thank you Thank you Thank you Thank you to the moon

[@leonvanzyl]

You're very welcome 🤗

[@TeamsWorkAI]

Very good. Thank you Leon 🙏

[@leonvanzyl]

You're welcome 🤗

[@GilbertMizrahi]

Great video as always. I really appreciate the effort you put into doing them.

[@leonvanzyl]

Appreciate it. This one was quite a bit of work 😅

[@HenrykAutomation]

This is amazing!!!! Haven't been so excited about anything like this since getting a colour gameboy LOL. Donated

[@leonvanzyl]

Hahaha!! Awesome comment.

[@leonvanzyl]

Thank you for the support 🙏. It helps me out more than you think

[@HenrykAutomation]

@@leonvanzyl nps :) do u know if flowise allows document uploads? and then so that document can be used in the chat?

[@jakkalsvibes]

Dankie Leon die 'knowledge shares' van jou help baie, is daar 'n manier om aan 'n 'gitlab' te konnekteer met gebruik van Flowise?

[@leonvanzyl]

Ek is seker dit moet moontlik wees. Mense scrape GitHub, so daar moet n soortgelyke proses met Gitlab wees. When there's a will, there's a way 😀

[@HyperUpscale]

AWESOME - a complete tutorial WOW

[@leonvanzyl]

I'd really like to do more project related videos.

[@HyperUpscale]

​@@leonvanzyl Is it possible to get an email with a content someone you follow posted or to update the databese about him? Example: You post a video... [automation]... I receive an email what it is about, based on the content of the video OR I got an update of the knowledgebase about you? How does that sound?

[@김정률-i9n]

I am your big fan. I always wait and watch your videos. Thank you. If I have the chance, I would like to invite you to Korea.

[@leonvanzyl]

I'm honoured! Would love to visit Korea one day.

[@김정률-i9n]

@@leonvanzyl Come anytime. You are very welcome!!

[@AbdulrahmanHariri]

Thanks Leon. Quick question: Would it be possible to set and use dynamic user specific fields to define which documents from the vector should be retrieved. For example, in Pinecone, you can set something like a namespace and then use it if retrieving documents related to a specific user. Basically, I am building a RAG flow that should only feed documents related to the user to the Ai to provide them with relevant responses. A user would never get answers from data related to another user.

[@petertorbenroden3659]

Thank you Leon for all your incredible videos. They are incredibly useful and very in-depth. I am experimenting with the document store and have uploaded a number of documents about a certain topic. I also uploaded a document with additional information that included sample pricing. When I asked the chatbot about it it said it couldn't find any information. Also if I ask how many document there are in the store or the names of documents it also cannot find any information. I would appreciate any tips you may have.

[@practical-skills-school]

Hi Leon, there's an error you couldn't see because of only one loader. With more loaders the loop still works 1 time, because the loaders array is not split. There should be a Split out node before the loop to extract the array items before looping them. Even can substitute the Set node, for it cleans stuff away.

[@leonvanzyl]

Great advice. Thanks 👍

[@Col-pd2zd]

Hey Leon! Thanks for the videos. Could you please do something on flowise production. So like, ollama + flowise and possibly render or ngrok to make it work as a whatsapp client? Something to take what you have taught us, and turn it into a finished product.

[@leonvanzyl]

Thanks! You should check out my Flowise & WhatsApp video 👍. kzbin.info/www/bejne/b2LEiGyPfNdpotE

[@MagieCannizzaro]

Really incredible good stuff thank you! How can I do some Data analysis on some long csv data? I mean if upload a 100 record csv (or json object), and I ask : "how many records we have" or "how may records has this tag XXX" it simply answer with the value used in the "Top K". I used the Rag chatbot, what and How should I use? thank you very much Leon! I'm spreading your channel on all my Team workers!

[@MagieCannizzaro]

What is the correct method to make query on CSV datas about total records, or percentage? like: "how many restaurants have 4 stars review" ?

[@k1r0vsiii]

now connect it to make aswell :D

[@leonvanzyl]

The process in Make is pretty much identical 😁.

[@joragca]

Amazing video! 🔥How should we modify this if the database were PDF documents in a Google Drive folder and subfolders?

[@ronwilliams4020]

Awesome video as always. If an airtable record is updated, will this process update the vector store with the new information?

[@leonvanzyl]

Correct 💯

[@marioromero6409]

Hi Leon. Thank you so much; your videos are very useful. The content in your channel is really easy to follow and to apply. I have a question. I am not a developer, and sometimes I struggle with some things. I had some issues following the process API section, and I noticed that there is another API called Re-process and upsert all documents in document store. Should I use this instead? If so, how? Thanks.

[@leonvanzyl]

Thanks 🙏. Yes, you can use the new API instead. It was literally releases yesterday 😁.

[@Jérôme-o8u]

Thx for your lekker vidéo Leon ! On RAG subreddit, I have read some comments about LLM having a tendency to provide poorly relevant answers when document store is too large (10K pages of documents for example) What is your opinion on the above ? Any experience on this ? If true, any tips you may advise?

[@leonvanzyl]

I would definitely spend some time looking at improving the quality of the data source in that instance. Rubbish in, rubbish out. Do you really need 10k pages of content? Does ever line in those documents add value, or could the content be summarised before loading it into the vector store? Also consider using an LLM that supports a large context window, like the Claude models. Also consider using agentic RAG, whereby a second agent reviews the relevance of the responses and assists in refining the vector query. I have a video on agentic RAG.

[@Jérôme-o8u]

@@leonvanzyl Thx so much for your precise reply, very much appreciate it. I will look into each tip you mentioned. I love your work :)

[@tangiblemark]

I have my database in Supabase. How can I upsert the data from Supabase into Flowiseai. (like the Document Stores do)

[@sukuna8547]

Hi Leon. Once again a great video. I have a question: Is it possible to change the first message of the chatbot in Flowise. The default message is “Hi there! How can I help?” I would like to change that

[@leonvanzyl]

Absolutely! Check out my website embed video where I show how to customise the chat window 👍

[@PhunkyBob]

At 29:12 we can see that 17 items have been inserted and 16 have been deleted. Is there a way to only update / insert modified / new items? If I have a huge knowledge base, I don't want all to be processed every 2 minutes, since it comes with a cost, isn't it?

[@leonvanzyl]

Oddly enough, that only seems to happen the first time you run the automation, after that is only inserts the delta.

[@PhunkyBob]

@@leonvanzyl Great! I'll give it a try with PDF documents.

[@robertopereiratorres9159]

Amazing tutorial! Would the process be too different if my knowledge base consisted of a bunch of .txt files in a local folder that is updated daily? My idea is to keep all the processes local.

[@leonvanzyl]

Absolutely! It will work exactly the same way. This process can be used with any loader that deals with dynamic data (like a folder containing files in your case).

[@tahreersquare]

Thanks a lot Leon for your valuable videos. I just want to say that this video is in a wrong playlist. Review plz. I'm a newbie subscriber from Egypt.

[@maverick9399]

Is there an option of creating multiple schemas in flowise database, as I have a project, based on the schema the database is arranged. I need for flowise as well, using single installation and use different schemas for different chatflows

[@ramonlansangan7307]

I have question, regarding unstructured data like HTML page, slack messages, Ms teams, how do we put that as RAG? I don't see built in Document Loader for Slack and Ms Teams.

[@logansanoh]

Wondering if it's possible to achieve this without n8n. Seems possible in theory, as flowise can be triggered directly from Airtable/Google sheets "change" event.

[@leonvanzyl]

Flowise is not a "workflow automation" tool, so I don't think it's in their interest to try and be a jack of all trades. If they do decide to add some auto refresh features within the tool I'll be the first to cover it 😊 .

[@practical-skills-school]

Is there any way to upload file-based loaders? I didn't find any file sending API-method. I'd like to send a file somehow and upsert it.

[@ravedead3909]

is it possible to have chatbot have access to your files on PC. then ask it something like, go to my file with movies and pic me out a pirate movie to watch, then it plays one of the movies?

[@saileshr1971]

Hey, its really a cool feature and thanks for sharing the knowledge with us. As an organization that is actively enhancing our capabilities in AI, we are interested in exploring further collaboration with you. We would greatly appreciate your support in providing training and consultation to help us deepen our knowledge and application of these technologies.

[@leonvanzyl]

You can find my email in the about section on my channel. Let's chat.

[@saileshr1971]

"Thanks, Leon! I’ll check the about section and send you an email shortly. Looking forward to our chat!"

[@jawwadhussain8457]

Please make a video like we have database MSSQL server, and we need output from that any test case demo that could how make a chatbot that get data from database and show the values

[@SoloJetMan]

is it possible to build this knowledgebase entirely on n8n?

[@leonvanzyl]

Unfortunately n8n does not provide a Document Store / Knowledge Base solution with these benefits. You can of course add document loaders and integrations to n8n. I have an n8n RAG Chatbot video that you might be interested in.

[@BirdManPhil]

what happens when you have many users chatting with the directory at the same time?

[@leonvanzyl]

Each user will have a seperate session, so there's no overlap.