Why Microcontrollers are a good AI companion

Artificial Intelligence offers a lot of chances for microcontrollers! As always, around this time of the year, I will look at this topic from a broader angle to see the implications of this technology on our hobby and professional lives. It extends the video on “rules” and shows how they are interwoven with Artificial Intelligence. As usual, I will stick to the implications on sensors and microcontrollers, and I hope you will get some useful insights for your decisions in 2025. And the project I will build will be “for the cat”, as we say in German.
My second channel: / hb9blawireless
Links:
Smokey's fountain: s.click.aliexp...
ESP32-CAM: s.click.aliexp...
Code: github.com/Sen...
Playlist with some wisdom: • Basics learned in a Lo...
Patreon supporter companies:
www.welectron....
The links above are usually affiliate links that support the channel (at no additional cost to you).
Supporting Material and Blog Page: www.sensorsiot.org
GitHub: www.github.com...
My Patreon Page: / andreasspiess
Discord: / discord
If you want to support the channel, please use the links below to start your shopping. No additional charges for you, but I get a commission on your purchases to buy new stuff for the channel
My Amazon.com shop: www.amazon.com...
For Banggood bit.ly/2jAQEf4
For AliExpress: For AliExpress: bit.ly/3MtXUY8 (just go on from here to your product)
For Amazon US: www.amazon.com...
For Amazon.de: amzn.to/2r0ZCYI
For Amazon UK: amzn.to/2mxBaJf
For ebay.com: ebay.to/2DuYXBp
I am no longer on X
As an Amazon Associate, I earn from qualifying purchases
#no#midroll#ads

[@peter-hy3rp]

One of my requirements would be to have everything running locally - no outside dependencies. This is for two reasons: first is so that the device continues to work if the network is down. Also if the third party changes the API or withdraws it.. or even if they decide to make it far more expensive.

[@GoodBaleadaMusic]

It's a path of resistance so it won't be widely adopted. It will be suitable for some cases but for most cases it won't be. That's what 5G was for.

[@MinZhang-b4m]

Any idea for his cat? I'm thinking PIR lens change, ultrasonic or laser distance sensor for height to activate PIR.

[@AndreasSpiess]

I am sure other and better solutions exist for the cat problem. However, the topic of the video was to spark your imagination. And the discussions show that this happened ;-)

[@squelchstuff]

I've been experimenting with the llava model on Ollama on some modest hardware (intel i5 minipc 64GB) The llava model can handle images as well as general language, so my system consists of a camera triggered by a mmwave sensor and the resulting image is then sent along with a prompt and interpreted by the model. The text response from the AI as to what is seen can then be parsed and used to operate other devices. This broadly follows the AI agent outlined by Andreas. It should be able to recognise a cat and maybe even, in theory, *which* cat is present, and is all running locally. PS I don't have a cat

[@thorntontarr2894]

@@AndreasSpiess And you succeeded in "sparking" my imagination. I didn't know exactly where you were heading but at the end it became totally clear. Well done.

[@ennobepunkt]

If I’m not mistaken, this marks your 500th video! A truly remarkable milestone! Congratulations from the bottom of my heart! Your videos are a constant source of inspiration for the community you so thoughtfully cater to. I always look forward to each new episode, filled with fresh ideas and, of course, that delightful Swiss accent, which adds an extra charm to even the most fascinating content. Greetings from Cologne!

[@AndreasSpiess]

You are right. Because I do not number the videos when they appear, I did not mention it. But you are a good observer! I particularly like that you mention my Swiss accent. Because if I get criticism about it, it is always from German-speaking people :-(

[@ei23de]

00:00 AI and Microcontrollers 00:52 Welcome and Introduction 01:07 Rules and AI 01:31 AI: Rule Finding vs. Rule Creation 05:26 AI vs. Conventional Programming 06:30 The Future of Programming 08:43 AI-Supported Process Optimization 11:20 AI Agents and Hybrid Approaches 13:41 The Cat Water Fountain Project 18:00 Summary and Outlook These chapters are created entirely locally (with AI) on my Linux Debian PC (transcript with Whisper-Turbo & chapter with Ollama) Quality and accuracy may vary. Use in the video description is permitted.

[@justinlinotte2981]

hello, i'd like to ask you about your hardware. What is your configuration, is it cheap or not, is it a dedicated computer in your home or only your desktop that you run your ai models on ? i'd like to make my own local "chatgpt-like" tool that everyone in my home network would be able to access via my local network. i don' t want to consume a lot of power 24/7 or to buy pricy GPU or RAM. Thanks you in advance for your possible advice. ;) I hope one day i'll be able to speak german too

[@ei23de]

@justinlinotte2981 I run a desktop Linux PC with an RTX 4090 (24GB VRAM) and an Intel i9-14900K and 64GB RAM. I also use the PC for other hardware-intensive tasks (Video and 3D Renderings). A slightly cheaper option would certainly suffice. For transcription with Whisper, 4GB of VRAM is already sufficient. Whether I can actually recommend running the models locally, I'm not entirely sure. DeepSeek's API is very good and very affordable, for example. Good hardware isn't necessarily required for this. I can't say whether learning German is worthwhile right now. In my opinion, English is a better option. If in doubt and not using it very much, use API services

[@justinlinotte2981]

@ thanks

[@jofland123]

As always: learned something, had much fun and "it was for the cat" 😂

[@AndreasSpiess]

:-))

[@devluz]

In my experience AI is to programmers what a calculator is to a mathematician. It takes over the most repetitive and boring part of the job but not the job itself. A programmers job isn't to write C++ really fast but to solve real world problems that often have no known solution yet.

[@enjibkk6850]

Don't fool yourself, your management sees it as we need only half of developers \o/. You can rationalize the nature of a developer's job, but maybe it won't be yours.

[@devluz]

​@@enjibkk6850 This is not quite the point I was trying to make. If your developers are now twice as fast it might mean you only need half as many. That is a fair point to make. You still need at the very least one developer though because AI just made them more efficient but did not replace the entire development process. This is why I chose the calculator as an example. It makes a mathematician more efficient but the calculator alone won't solve any maths problems. Edit: ChatGPT is a bit like a novice typing out formulas it found online but unable to actually solve any novel problems it hasn't encountered yet.

[@AndreasSpiess]

I tried to make this point by distinguishing the two jobs of creating something new and executing something using learned "rules of thumb." However, I think that programmers use quite some time to write code. Otherwise, I cannot understand the discussions on IDEs and frameworks. I like the comparison with a calculator. This is why I compared AI with a compiler.

[@joehopfield]

Lol - AI suggests using AI.

[@AndreasSpiess]

That is how addiction works ;-)

[@mdeeen]

A.S suggest A.I Andreas S.

[@EnergyTRE]

​@@AndreasSpiess😂😂👍

[@JorgenHenningsen]

“If the only tool you have is a hammer, you tend to see every problem as a nail.”😉

[@florinpandele5205]

Yup, you ... nailed it 😅

[@cl3132]

what do you mean? ... just guessing

[@JorgenHenningsen]

@@cl3132 The esp32 maker hammer.

[@AndreasSpiess]

The title of the video was about AI. And I thought the project was just an example. But you are right: I have no other MCU for connected Systems.

[@toddbellows5282]

Suspend the pir sensor above the fountain, looking down. Creates an umbrella activation zone. Simple fix.

[@SnakebitSTI]

I feel a bit like this was a very good video on how to drive a screw with a hammer called ChatGPT. Though I think the purpose of the video was demonstrating the problem solving process, not the specific use case. If I were to use AI for this, it'd be to train a locally running model to interpret sensor data.

[@DaveEtchells]

Nah, not enough technology 😂 (Maybe a dangling PIR sensor would be in the way though.)

[@florinpandele5205]

Nope, it will still "see" your feet if you're nearby.. but you could designate a cat only zone on the floor😅.

[@AndreasSpiess]

Remember that the title of the video was not "how to build the best cat fountain controller". It was to explain AI (and it used a silly project as an example) ;-)

[@jeffking3693]

@@AndreasSpiess Perhaps, but the car detector was brilliant, and more to my liking just how flexible this can be.

[@mattchinnock7064]

This is the best explanation of the current and near future state of AI in any context, let alone microcontrollers.

[@AndreasSpiess]

Glad you think so!

[@SkillsToLearn]

Thanks!

[@AndreasSpiess]

Thank you for your support!

[@SkillsToLearn]

@AndreasSpiess Amazing knowledge about rules. The line between invented and discovered is really hard for me.

[@AndreasSpiess]

@@SkillsToLearn I ask myself: Was the rule here before, and can I change it? When it was there, and I could not change it, I have to learn it. If not, I obviously created it. There is always a gray area, of course.

[@klassichd10]

Thank you! Preventing entering cats with mice is a really good improvement! Currently I am using AI to support programming in Python, JavaScript and XLS-tricks. Very handy.

[@AndreasSpiess]

So far, I did not test this one. I first want to get experiecne with the fountain. And indeed, because I sometimes need an Excel macro, I also ask ChatGPT. It is much faster than me...

[@klassichd10]

@@AndreasSpiess btw and offtopic: Just recognized what is going on at Grimselsee and the new dam in front of the old one. I am very impressed, Kudos!

[@AndreasSpiess]

@@klassichd10 We are still not sure it can be built. Some opposition even if the vote was for it... There are 19 approved projects to increase energy production in winter.

[@klassichd10]

@@AndreasSpiess Grimselsee dam seems to be nearly finished. The distance between old and new dam seems to be very tight. Trift seems to be under discussion. We will see.

[@WACkZerden]

respect for getting cat a fountain

[@AndreasSpiess]

I am not sure if this is a positive or negative comment ;-)

[@WACkZerden]

@AndreasSpiess flowing water is generally less prone to slime growth than standing water

[@ideacafe07]

​@@AndreasSpiess It is a positive comment.

[@SnakebitSTI]

There are trade-offs. I've yet to find a pet fountain that's free of difficult to clean parts in direct contact with the water. Water in a bowl sits stagnant, but bowls are easy to swap out and clean in a dishwasher. I think the most important consideration is how the cat likes to drink. Some love fountains, some hate the noise they make and refuse to use them. I had a cat who refused to use a fountain and instead preferred to ask me to refresh his water bowl or open a faucet whenever he was thirsty. He had a particular meow for water.

[@MinZhang-b4m]

@@SnakebitSTIi add iodine to my standing water and to prevent slime growth. It's a supplement and it's anti microbial. Try a small amount like an mg or so at once.

[@madeautonomous]

What happens if your cat sleeps in front of the water fountain...? Would it keep calling the API frequently?

[@InfiniteQuest86]

PIR needs motion to activate. So only if they are running in their sleep or something.

[@javierolavarria]

Refine the prompt: Is the cat sniffing the fountain or the faucet waiting for water to come out?

[@largerification]

I think the newer ESP S3 cam has a built in model for recognizing cats. Anyways. I am experimenting with Frigate in Home assistant and amazed how can the model recognize so many objects, which are not there in reality. Especially Zebras and Giraffes 😂😂😂

[@jmr]

It's called the "Model Zoo" and the capability has been available for around 4 years already. Great name @espressif 😆

[@MinZhang-b4m]

Is it yolo?

[@AndreasSpiess]

I started with Frigate. But my Server Consumer a lot of CPU that the Fan ran all the time. How was the CPU consumption of your Frigate?

[@largerification]

@@AndreasSpiess 3 cameras and it went through the roof. But I was expecting this. I purchased a Coral TPU, PCI version, B+M key and put it in a PCIe adapter on the motherboard. Worked and now my 1225 V3 is barely hitting more than 20%. Still my E32 workstation was not supporting PCIe passthrough, so to use the TPU, i installed HassOS on bare metal. I actually have 2 machines, same specs, the production one is on site in a VM, I tested at home, streaming the camera feeds across Zerotier (Tanks, Andreas for this too:) ), but cannot figure out why I am only getting the low res feed... . What I am trying to say is that my server now is loaded at 20% (as oppose to 99%), but not sure if this is because of low res feed or it is the TPU. The main problem now is storage. Apparently I cannot use my HDD to store media only. Have to move the entire data disk. A VM NAS scenario would have helped, but then the TPU will not work... :)

[@largerification]

​@@AndreasSpiesscpu consumption was bad, until I added a Coral TPU, however I still have to experiment, but definitely a pci version B+M key put the cpu to 20% . Lenovo E32 with Xeon 1225v3 and 32GB ram, with hasos on bare metal, due to no pci passthrough. I can easily go with just 8gb as at the moment it uses less than 2 :)

[@gregorymccoy6797]

Very interesting. I never considered off-loading requests to chatgpt for evaluation of complex conditions. This was well worth my time.

[@AndreasSpiess]

I'm glad you found it useful!

[@thorbenh]

Not really happy with the idea of sending picture taken inside my home to openAI.

[@AndreasSpiess]

I have no problem with that in this case. Keep in mind the low resolution of the ESP32-CAM. So, the area in the picture is very small because the camera has to be quite close. Otherwise, I agree.

[@ToTo-od9wz]

I like you, your projects and...your cats! Thank you for your effort for good presentations!

[@AndreasSpiess]

Thank you. You are welcome!

[@flirk2818]

On one hand - an interesting use of chatGPT. On the other - setting up 2 PIR sensors shaded in such a way as to detect heat sources below 1 m on one and above 1 m on the other and turning the fountain on only if the lower sensor is triggered would be cheaper than ESP32-Cam and would not require internet access nor would it send data on who comes and goes into your kitchen (with pictures and timestamps) to the chatGPT owners...

[@AndreasSpiess]

I am sure other solutions exist. I do not care about sending a picture from the fountain (with or without a cat) to chatGPT. Keep in mind that the ESP32-CAM has a low resolution, and I had to place it close to the fountain.

[@zhenghe1065]

What about a simple YOLO v3 classification? Is a ESP32 not fast enough? I wouldn't like my kitty depends on Internet connection and chatGPT. Bless Zheng

[@AndreasSpiess]

My experiments with small-scale ML were not encouraging. And I do not want to run the ESP 24/7. The easiest to solve your problem would be: If no answer from ChatGPT means cat detected, BTW ;-)

[@zhenghe1065]

​@@AndreasSpiessYes, simplest failsafe! ...but you don't need 7/24 if you wake up from deep sleep on pir. By the way: why not ESP 7/24?

[@peter-hy3rp]

This sounds like a great project to run on a Jetson board.

[@jjdawg9918]

Seems the same old saying applies. " the person that knows HOW will always work for the person that knows WHY" even if that "person" is an AI. Thanks as always Andreas

[@AndreasSpiess]

A good point!

[@brunosalezze]

I solved the fountain problem with a 24Ghz LD2410C radar sensor. As they measure detection and distance. So the distance sensor is the only thing I need to turn it on or off. This also works offline.

[@AndreasSpiess]

Good point! I did not try it, but fear that our kitchen is so small that the distances would not be a sufficient discriminator. But for sure much better than the current "general purpose" PIR.

[@j.f.christ8421]

I was going to do a cat doorbell like that, but RF used too much power (couldn't wire the sensors in.) I put cheap BlueTooth trackers on the cats, and at the door put an IR sensor, which when triggered woke up the microcontroller (AtTiny) that started listening for BLE. If it got a match it sent a 433MHz signal to the base station that beeped & lit up as to what door the cat was at. Was going to use ESP32 but didn't have any, so random modules I had on hand was it. Detector boxes last about 3 months on a charge. Base station shows battery alert. Lipo battery, USB charged. Works well! Primary user is an 80 year-old woman, she thinks it's great. The cats do as well.

[@AndreasSpiess]

@@j.f.christ8421 Good project! Or cat door reads the implanted RFID and opens only when our cat wants to pass.

[@j.f.christ8421]

@@AndreasSpiess Brickwork & glass doors meant a cat door wasn't an option. The cats are microchipped, but I don't think RFID has the range compared to PIR & BT. As a bonus I've added everyone's phone to it so it works as an automatic doorbell. But yeah, nice simple little useful project.

[@electronics.unmessed]

Thx, this kind of videos is helping a lot to estimate what AI can do and what not. I am using it for image creation. I made the experience that AI is not at all understanding content or context. But sometimes it is helpful to find an approach. In the end I always need a lot of my own modest natural intelligence.

[@AndreasSpiess]

I agree that prompting is and will be important in the future. Here, you can provide the context you feel is essential.

[@wolpumba4099]

*AI and Microcontrollers: Exploring the Future of Programming and Automation* * *0:00:02** Introduction:* The video explores the impact of artificial intelligence (AI) on microcontrollers, sensors, and our lives, extending the discussion on rules from a previous video. * *0:00:54** Defining Rules:* Humans use rules to organize society and understand the world. These include ethical, religious, legal, and practical rules, including physical laws. * *0:02:36** AI and Rules:* AI systems are not intelligent in the human sense but excel at pattern recognition, which is closely related to rules. They have two phases: training (learning rules from examples) and inference (applying learned rules). * *0:03:59** Rule Creation vs. Rule Discovery:* Humans can create new rules through free will and decision-making, while AI excels at discovering and executing existing rules. Creating rules involves defining strategies and considering consequences, which is currently beyond AI's capabilities. * *0:05:28** AI vs. Conventional Programming:* AI can potentially replace conventional programming by learning and executing rules automatically. Traditional programming involves humans defining and coding rules. * *0:07:03** AI's Role in Programming:* AI is becoming a new type of compiler, with prompting as the new programming language. This shift will change the role of programmers, requiring them to focus on higher-level tasks like defining specifications and strategies. * *0:08:45** AI for Business Processes:* AI can be trained on a company's past transactions to automate processes and improve decision-making without requiring extensive conventional programming. * *0:09:33** Prompting:* Writing effective prompts is crucial for guiding AI, as prompts must contain the human-created rules that AI should follow. * *0:10:19** Challenges of AI-Driven Systems:* AI systems can be unpredictable and may "hallucinate," making testing and validation challenging. This poses issues for implementing AI in regulated environments. * *0:11:28** AI Agents:* AI agents combine AI with conventional systems to solve complex problems. AI decides which systems to use and creates specifications, while conventional systems execute tasks predictably. * *0:13:38** Practical Application: The Cat Fountain:* The presenter uses an example of a cat fountain to demonstrate how AI can be integrated with microcontrollers. An ESP32-CAM, triggered by a PIR sensor, takes a picture when motion is detected. * *0:15:48** AI-Powered Cat Detection:* The captured image is sent to the ChatGPT API, which determines if a cat is present. If a cat is detected, the fountain pump is activated. * *0:16:41** Flexibility and Scalability:* This setup can be easily adapted to detect other objects or animals by simply modifying the prompt sent to the AI. * *0:17:32** Potential Applications:* This approach can be used for various applications, such as opening a garage door based on license plate recognition or preventing a cat from bringing prey indoors. * *0:18:02** Conclusion:* AI excels at pattern recognition and rule execution but is not yet capable of independent rule creation. The future likely involves a blend of AI and conventional systems, with microcontrollers playing a key role in bridging the gap between AI and the physical world. * *0:20:16** Call to Action:* Viewers are encouraged to support the channel and stay tuned for future videos. I used gemini-1.5-pro-exp-0827 on rocketrecap dot com to summarize the transcript. Cost (if I didn't use the free tier): $0.04 Input tokens: 26774 Output tokens: 773

[@MatSmithLondon]

Hello. The abstraction layer of this video was quite high. A lot of philosophy at the start, but also it continued into more high level concepts. Please can you make another video about the nuts and bolts of actually doing this project. Maybe it is not as interesting, but it can be more useful. How can I set about interfacing my esp32cam with ChatGPT? What type of subscription do I need? what code can I use to achieve these questions “is there a cat” and how can I get the response from ChatGPT in a binary format that can be used by the ESP? Are there cheap or even free alternatives to ChatGPT, and what are the cost comparisons? For example locally installed AI such as CodeProject. Thank you for your excellent videos as usual.

[@AndreasSpiess]

You need to be able to use the APIs of chatGPT (I charged 10 dollars on my account). As always, the link to the code is in the video description. You can experiment with other models if you want. I am happy with ChatGPT. For one month, with lots of experimenting, I was charged 1 dollar.

[@radry100]

That's a really nice idea and surprised me. However, how fast is this whole system? Is there a noticeable delay when the cat approaches the fountain? What if the camera is triggered by a human first but the cat comes a second later? I think there are a lot of potential problems with this.

[@AndreasSpiess]

1. It takes maybe 2 seconds to trigger. However, the PIR triggers before the cat is at the fountain. It only takes one picture during one session. If no cat there is no water fountain :-(

[@matneu27]

As in your home, the most important task for an AI system would be a cat flap that blocks cats with their catch, like birds or mice. BTW, I heard a couple of times before from some Swiss guys that they had already invented this system. Never know that if it was offered to buy. Anyway, as a fish tank owner we didn't need a drinking well because all of our cats prefer self-service water with fish-taste ;-)

[@AndreasSpiess]

Such an add-on is offered for the cat flap. But the salesman did not want to sell us one because of bad experience. It is expensive and seems not to work. Maybe with a powerful engine like ChatGPT we could change that...

[@ionciubotaru]

A capacitive mat placed under/around the water fountain would activate it only when the cat steps on it. I love your videos keep up the good work

[@AndreasSpiess]

Also a good possibility!

[@xanokothe]

I agree with you. Prompt is the new language, AI is the new "compiler". But, it is a very bad compiler. You know when you are struggling with your code, where you are doubting everything. You know what you do not doubt? The compiler. But for AI, whatever I tried to use it I gave me crap code. It is funny, because from the distance the code looks good, then if you look into the details you see that it makes decisions and mistakes of rookie programmers. For now I am not going to use it, I think maybe for the next AI cycle. Yes, AI have cycles, I think it had 2 "dark ages" already.

[@AndreasSpiess]

I hope they will use all the "crap" code including the error messages you entered for training... I have mixed results from ChatGPT: Some code worked without changes and other was halucinated. I think it depends on the number of available examples for training.

[@mcconkeyb]

Excellent summary of current "AI"! Also a good tutorial on the use of current "AI". But some day far in the future we might get real AI, then you will have to update this great video. 😉

[@AndreasSpiess]

Thank you! That is something for my successor ;-)

[@qzorn4440]

AI is a new tool for me. Learning the terms and buzz words to do AI Raspberry Pi, ESP32, and Arduino projects is my wish 🎉 Thanks for the wonderful FYI AI video. 19:53

[@AndreasSpiess]

You are welcome!

[@coryhisey8431]

I like the idea for sending api calls instead of a micro tensorflow or something. Definitely seems a better use of (current) resources. Like you said those micro ML libraries are unreliable while internet connection and api calls usually are. I can see this application being good in other edge AI use cases

[@AndreasSpiess]

I also think that having a full-blown AI infrastructure at home is not the best idea for every application.

[@doejohn8674]

Hi Andreas, off topic question: Have you looked into DAB+ and TPEG messaging? As you certainly know, we don't receive TMC messages anymore in Switzerland as they have shut down public broadcasting over FM band, and my on board navigation system now doesn't know about traffic jams :(. One interesting solution could be to translate TPEG messages to TMC and emit them on a very small FM transmitter (just theoretically), another could be to build a dongle receiving TPEG and displaying it at least on an Android tablet either on a map or as text messages.

[@AndreasSpiess]

No. I only use Google Maps for driving because it shows jams with high precision, calculates my delay, and even recommends another route if it is better. So for a long time, I did not use these messages. I also decided not to replace the radio of our old Volvo and added a Bluetooth receiver. So my wife can still listen to the radio, and if I am alone, I can listen to my podcasts...

[@portfedh]

This is very ingenious. Loved your solution

[@AndreasSpiess]

Thanks for your comment!

[@ristomatti]

Your imagination in making use of these new possibilities is better than mine. Well done Andreas!

[@AndreasSpiess]

Thank you! I tought this is a good idea to show the possibilities...

[@fedesalva5074]

Awesome video! Congratulations!!

[@AndreasSpiess]

Thank you!

[@mikenielsen8781]

9:03 "It _should_ be able to find the existing rules..." But how would you know that it actually _has_ found them and correctly reflects them? What if there are multiple paths through the rules? How do you prove completeness, uniqueness and correctness? The "AI" I have experience with is frequently wrong and its answers require careful examination, to the point where I spend as much or more time getting a result via "AI" as I would the old-fashioned way. I occasionally write some code, and my IDE has "AI". I would say it does an OK job at "peephole" coding -- inferring the next few lines -- but it seems like it might be some way off before it can write anything but relatively mundane parts of programs. It's hard for me to understand how it's going to "replace algorithms" if that's what I understood you to say. Sorting a long list, factoring a large prime number or multiplying huge matrices (an operation at the heart of "AI") will still rely on efficient algorithms. I don't think "AI" will be inventing any *new* algorithms, it can only regurgitate (albeit in remarkable fashion).

[@AndreasSpiess]

I agree with your first paragraph. That is why I mentioned ISO9000 and that it will take a lot of work to assure predictibility (testability). Conerning paragraph 2: We should keep in mind that the LLMs are quite new and the development speed, at least at the moment, seem to be quite fast. Paragraph 3: I separated "formulas (and algorithms)" from the other rules because I believe that they will be faster executed by non.AI systems. So we agree.

[@ed.puckett]

Thank you, this episode gives me a lot to think about.

[@AndreasSpiess]

Glad to read this!

[@MinZhang-b4m]

Any reason why you didn't use the nut or another cat tracker and used AI? We're they too slow? I'm addicted to offline and low power solutions so I would have used a different PIR lens, maybe a laser/ultrasonic distance sensor for human height to detect when the PIR should activate too.

[@AndreasSpiess]

Because Smokey is very active outside the house, we do not want him to carry a collar. And reading his RFID chip only works inside a spool (I once made a video). That is perfect for a cat door where he has to put his head through a small hole. But not for the fountain.

[@greg4367]

Thank you for this, it has me thinking in new ways

[@AndreasSpiess]

My pleasure!

[@christianaastorp6105]

Brilliant use of resources.

[@AndreasSpiess]

Thank you!

[@deslomeslager]

I would add a second PIR, one that faces up. Find a sweet spot so that the original PIR sees the cat, but the original PIR does not get to send his 'on' signal when the second PIR is triggered (that one faces up). A second option I am not seeing is a mat with weight detection (pressure). Or simply learn the cat to push a trigger in any way in stead of the PIR. To be honest, I did not think of making a picture just to let AI detect what is on it. That is what I learned, although I did not expect such a solution given the problem to solve.

[@AndreasSpiess]

As said in the video: You need a human to create such crazy ideas ;-)

[@kwazar6725]

Wow. Great sunday info. Merci Andreas!

[@AndreasSpiess]

Glad you liked it!

[@claudiomoles]

Happy New Year, Andreaas, excellent approach, I am not too into AI in my hobby developments, but I have an ESP32 CAM and a Cat, I think is about time to give it a try, I recently attended a course at the local university about Python and Natural Language Interpretation, which I believe is very important to learn "prompting". Thank You.

[@AndreasSpiess]

As you write, learning how to deal with AI is important. Even if we decide it is nothing for us.

[@andybarnard4575]

If the API call costs 10c and I spend an hour in the kitchen cooking Sunday lunch and trigger the PIR every time I move, does this not work out quite expensive for all the times the AI answers "no" cat?

[@AndreasSpiess]

With the resolution I use on the ESP32-CAM it costs much less than one cent. But I have no longtime experience. Still, AI hardware is quite expensive. I experimented with Frigate on my HA server and the fan ran all the time. CPU usage was quite high just for monitoring the camera.

[@andybarnard4575]

​@@AndreasSpiessIt was only a thought experiment, but thanks for the reply. Optimisation of PIR location, or alternative use of a RADAR sensor might also optimise the set up. I also wondered if AI could determine if the cat looked thirsty before turning on the fountain, but perhaps AI is not ready for that yet.....

[@yasirrakhurrafat1142]

​@@andybarnard4575 having a local model running will be better, as we would be able to modify and improve it by personalised configs and tuning as required. Plus, less cost

[@321ooo123]

@@andybarnard4575 neither are humans. I've yet to recognize a thirsty cat from a regular one...

[@Pismoboy]

Thanks for sharing this! OpenAI probably has a better model, but you might consider trying a multimodal model in ollama running locally to see if it can do an adequate job, cheaper (and maximize your privacy by not sending photos of your kitchen to the internet). You wouldn't even need to change your code much- just point it to the local ollama endpoint instead of the openAI URL. Maybe Nvidia will send you their new Jetson Nano (2?) to try it on if you contact them. As long as the cat doesn't get bored and run away, it probably doesn't matter much if it takes 5-10s or longer to wait for the "yes there's a cat" response from a slower local model.

[@OldCurmudgeon3DP]

I had a cat named Smokey when I was a kid. Most mild-tempered (but quirky) Siamese I'd ever seen.

[@AndreasSpiess]

That must have been an impressive cat that you still remember him!

[@OldCurmudgeon3DP]

@AndreasSpiess all of them.

[@Alex000113]

A lot of philosophy in this one !, but I'm not sure how accurate /useful your summary of AI history is. AI is not so new , as I remember expert systems in the 80s based on rules and programming languages such as Lisp that supported 'self updating'. Its also worth remembering that non-linear systems such as neural nets can produce extremely unexpected results. Training is a key phase of supervised pattern recognition, unsupervised AI does not require training and relies on static rules or 'anomaly detection', to trigger responses, so theres different kinds of AI. ChatGPT is amazing and I agree it presents something evolutionary for sw developers. It will write html, JS, php C++ and even arduino code for me but BEWARE as it often writes code that doesnt compile/work. Interesting you mention SAP as I think we're at a similar point as when the DBA role appeared (maybe SystemsEngineering needs to recognize a new AIprompter role ?). I think your cat fountain AIarchitecture is interesting but overengineered for the problem - requiring networking and remote processing when a small simple standalone microcontroller with PIR sensor will do a good enough job. KISS ! I'll close by saying I agree microprocessors are a good companion to AI systems, for example a ESPCAM might be useful to trigger relays when recognizing specific objects but maybe a background function could be to upload images when appropriate to a remote 'deep learning' archive for AI analysis. Opportunity/risks abound for new standards and business processes that help define systems and products that include AI components.

[@AndreasSpiess]

I thought I mentioned the rules-based expert systems in the video. In my design, I use an ESP32-CAM board, BTW. We all know that LLMs can be very creative. We even have a name for it: Hallucinating ;-) I also thought I explained why I wanted to enhance the PIR sensor. But I agree this solution is over-engineered (also mentioned in the video).

[@thesimplicitylifestyle]

Very useful, thanks! 😎🤖

[@AndreasSpiess]

I'm glad you think so!

[@Heisenberg2097]

A simple RFID-Tag reader would have solved the cat problem at a much better CBR. Anyway. Nice video. Btw. much of the randomness of AI is implemented on purpose to make the outputs look more human. 3Blue1Brown has great videos detailling LLMs and more in depth.

[@AndreasSpiess]

Thank you for the tip about 3Blue1Brown. Anyway, I am unsure if RFID would work because of the considerable distances. In my video, the cats had to go through a coil to read the RFID chip. The same is true for the cat door that reads the RFID chip.

[@simonsayshomeassistant]

Happy new year Andreas! 2025 is the year of AI

[@AndreasSpiess]

Happy New Year, too! I fear we are in a similar situation as during the dot-com bubble, which crashed in 2000. It could crash because of too high expectations, but in the long run, all our dreams will come true, just not as fast. But what does an old man know ;-)

[@mrrboo]

Cool projet 👍 have you considered doing the inference directly in the microcontroller? ST has a new kit with camera and STM32N6 that should be perfect (with their NPU to accelerate the AI model) This way you don't need connectivity or external API: the fountain itself should recognize your cat 😊

[@AndreasSpiess]

My experience with such small scale AI was not good in the past. Plus I would have to run the ESP32 24/7. I have no problems with external APIs. My live depends in many situations on such services (like google maps while driving or credit card services for paying)...

[@tonysfun]

Nice job again! Happy New Year to you and your family - include your pets!

[@AndreasSpiess]

Happy new year to you, too! And enjoy your project for your friend.

[@BALTAZAR6174]

You inspire me to learn more about the way of the electron. May the electromotive force be with you.

[@AndreasSpiess]

That is a good idea!

[@PhG1961]

Remarkable video. Awesome and inspiring... Happy New Year btw, just in case I forgot it until now.

[@AndreasSpiess]

Happy new year, too! And thank you.

[@Martin-DL]

Great video. Thanks a lot. Sorry for the question: what happend to Dishka?

[@AndreasSpiess]

Because she was already old, she passed away :-(

[@JulioSalim]

Brilliant as usual! Thank you!

[@AndreasSpiess]

You are welcome!

[@MrKelaher]

Very good ! Your title may scare some off, but this is very good summary of some interesting thoughts on LLM, in general. We do not agree about SLM though - they can be very good indeed

[@AndreasSpiess]

So far. I did not see very convincing results. So your experiences are good?

[@timog7358]

amazing video. so much useful information

[@AndreasSpiess]

Thank you!

[@doejohn8674]

Next big step in AI development: AI suggests asking Andreas Spiess the question to get an innovative answer :)

[@AndreasSpiess]

:-))

[@dermick]

Apologies if you've already done a video on this, but could the Raspberry Pi AI camera not do this locally? Just curious about your views on that device. Love your videos, Andreas!

[@AndreasSpiess]

I never did it because the results I saw with such small models were less than convincing.

[@MikeKranidis]

Wonderful video, thanks Mr. Spiess

[@AndreasSpiess]

Glad you enjoyed it!

[@hejasverigeee]

15:50 CatGPT 😂

[@AndreasSpiess]

:-))

[@hejasverigeee]

@AndreasSpiess Greetings from Stockholm Sweden Europe 🇨🇭🇸🇪

[@seabassvn]

Very good vid. Makes me think of new projects. Merci!

[@AndreasSpiess]

Glad to hear that!

[@adenwellsmith6908]

AI is great for helping write code. However you do pick up on one thing I've noticed. It's biased towards older code. You can help it by sending it the new API then following up with getting it to write your code based on that. It's a bit more fiddly.

[@AndreasSpiess]

Thank you for the hint!

[@TheBeefiestable]

poor kitty dead if your internet goes down. Did any ai suggestions default to motion only required if internet is down?

[@AndreasSpiess]

There is a simple solution to this problem: If no answer, switch the fountain on anyway ;-)

[@jameshughes3014]

I love things like that that have so many possible solutions. I think I would have used yolo on a pc , and let the microcontroller talk to that to keep everything local. I wish someone would make a small cheap asic with Yolo or some other fast vision model built into it. That would be such an amazing thing for tinkerers

[@AndreasSpiess]

I tried Frigate first. But my server's fan run all the time becasue it used a lot of CPU. And 24/7...

[@notofinterest]

I’m not entirely of your opinion. AI can do an excellent job when it comes to the basics, like workflows, structures, definitions, etc. But when it gets into the details and the individual pieces become very small (assembler, bit/byte juggling, UDP packet structure, …), the usual problems emerge. These include stubbornness regarding changed requirements by sticking to the existing approach, faulty/incomplete summaries for a protocol, and general lack of creativity. Essentially the same issue as the finger problem in image generation. It can also draw a piano very well, but the keyboard ends up with incorrect keys. And you can tell it a hundred times: don’t change A and B - it doesn’t care and does it wrong anyway. I’ve only tried this with GPT-4o so far, maybe there’s already something better out there.

[@pascalfleury972]

indeed, AI solutions are not well reproducible. makes evolving code very difficult.

[@SnakebitSTI]

These are the reasons I was hoping this video was about training a small model, not about throwing ChatGPT at a problem.

[@notofinterest]

@@pascalfleury972 good point!

[@AndreasSpiess]

I am not sure if we already see the end of the possibilities of AI. It is less than 3 years since I heard the first time about ChatGPT. I agree that AI has (and maybe willl have also in the future) problems with details. My simple explanation for that is that not enough training material exists for each detail. Whether this will change, I do not know. If I see how many lines of code are already entered into AI today (including all the error messages if it does not work), I am positive that they will use this knowledge for future training. Think of Github...

[@notofinterest]

@@AndreasSpiess there is enough trainingmaterial to build models; problem is the validation of it, let me give you an example: a kid will learn early that it will hurt when its head bumps against a doorframe, and come to the solution as possible avoid bumping. I see ai as similar, but with no sensor telling him this is to avoid. Back to realworld: how can ai distinct between good and bad code?

[@____________________________.x]

Just as an alternative idea, a PIR and a colour sensor. So if an orange heat emitting object is detected, turn on pump

[@AndreasSpiess]

Interesting idea. Unfortunately, Smokey is a grey cat. Not so easy for a color sensor...

[@karthikdani]

Insightful video. What should someone starting out Embedded Systems as Intern, possibly should focus on as the "Experience" that is valuable and stands still for the future?

[@AndreasSpiess]

As shown in the video, writing specifications (prompting) will be very important in the future. However, it is not easy without understanding one layer below. So I recommend to search for a problem you want to solve and start with a project. I learn nearly everything with a project (as this video shows). BTW: Many things I learned were not immediately useful. But sometime in the future...

[@rafaelnatera97]

A good video to prepare us to the future. For this use case a more efficient approach to performing all locally is to use Ollama alongside a high-quality open-source "Image-to-Text" model such as MiniCPM-V-2.6. It is preferable to use structured output responses, for example, in JSON format. If you have sufficiently powerful hardware (such as an Nvidia RTX 4060 Ti 16GB or better), this method ensures consistency, reduces costs, and enhances security.

[@AndreasSpiess]

I tried first Frigate with pattern recognition. But it consumed a lot of CPU power, and my server fan ran all the time :-( So, I would have replaced the pump motor with the fan motor. Not to speak about the additional power consumption of the CPU 24/7. ChatGPT's response is in JSON. However, I only need one field with the answer.

[@rafaelnatera97]

@AndreasSpiess Frigate is an excelent solution, again if you have the hardware. Probably using a Google Coral I think you can do it and it's not to expensive.

[@deathcore420]

Great video as always! Not sure if legit, but this is why I like to differ two words - coding and programming. AI migt soon be able to be good at coding, but not programming, where programming includes everything - starting with understenging the problem and reasons behind it.

[@AndreasSpiess]

Because coding and programming are not sharply defined I tried to distinguish between things that can be learned ("rules of thumb") and thing that cannot (create newe rules). I included the understanding the problem into "prompting". So we agree and only use different words, I think

[@watchedsvideo]

Fantastic! I wonder if I could use such an approach to determine when my cat is approaching the catflap with a mouse?

[@AndreasSpiess]

I do not know if it would work. Smokey is grey, and the mice he brings are small, so it is sometimes even difficult for me to see them.

[@nunamvseravno]

It's a price gift for you being a good owner!

[@watchedsvideo]

@@AndreasSpiess There are a few commercially available catflaps that claim to be able to do this, but they're new and quite expensive. On my ever-growing backlog of projects, I want to build such a contraption. I already have a Sureflap RFID entrance that only accepts my cat and dog (there are neighborhood cats that used to come in and make themselves at home!). The integration with Home Assistant is cloud-based and very unreliable. So, my objectives are: 1) Replace or add to the existing controller with my own ESP32, integrate the onboard RFID tech, thus allowing me a local HA integration for locking, unlocking, tracking and setting times. 2) Add an RPi5 + camera + Google Coral to monitor the entrance, and lock the flap when it detects "Cat with prey"... This would all be local. However, it hadn't occurred to me to outsource the photo analysis to something in the cloud. Do you think this would be fast enough?

[@watchedsvideo]

Also pondering the pro's and con's of motion-based photo analysis vs continuous real-time video analysis, complexity vs cost...

[@sergeyp2932]

Interesting video and very elegant solution! Thanks. IMHO, there is no difference between "created" and "discovered" rules. I think, "created" rules are just a way to formalize some goal, and creating those goal is the main human's job. For example, human laws are just a proposed solution to a goal "create a safe and comfort living condition for as many people as possible" (roughly, but I hope everyone got the idea). So, "rules" we set in AI prompts are the way to achieve our sub-goal: communicating our main goal to AI system in form it can process it. And, I think, such rules can be considered "discovered" as well.

[@SnakebitSTI]

Where is the elegance in a cat water fountain that requires a server farm to operate?

[@sergeyp2932]

@@SnakebitSTI In reusing existing components, fitting them together to solve the task. Some people like writing code, some people don't. And for those who don't, modern world provides many way to compose complex system from already existing components (open source programs and relatively cheap cloud services). But these possibilities require some effort to discover. I think, running server farm is not a problem as long as it fits cost limits.

[@AndreasSpiess]

This is why I said that created rules cannot be learned (I called them "prompts"). They result from strategies and goals that can be created (invented). Discovered rules, on the other hand, can be learned because they existed before. Think of a programming language that you cannot change, but learn how it works.

[@wojciechzielinski7825]

Router-Based Approach First, we need a cheap, small, and fast model. This model decides what to do, then creates the appropriate prompt and directs it to the specific agent. This can be nested-one router inside another. With this approach, you can do just about anything: from training small, local neural networks on the fly, to controlling complex systems like lighting, temperature, and ventilation, and even predicting future behaviors. Everything is cached. I use stemming, lemmatization, and a few other simple tricks for caching, which quickly reduces repeated queries and boosts speed. This method is flexible enough that the way a cat is detected can evolve over time without significantly disrupting the workflow. It can also handle auto-corrections and self-reflection smoothly. What’s more, it’s fast-results from self-reflection, for example, can be processed later without holding up the system. This is how I build systems that initially rely on LLMs (large language models) but are gradually shifted to fast, local solutions. On top of that, I use the same approach to generate datasets for smaller networks, making it easy to handle almost any type of radar for detecting positions, gestures, or other tasks-without needing tons of manual work or complex math. (As much fun as that can be, I do have a children and wife)

[@AndreasSpiess]

An interesting approach. It would be worthwhile that you document your projects!

[@wojciechzielinski7825]

@@AndreasSpiess yes, I need, some day, now I am on training NN's for ms72sf1 radar with fisheye cameras, also interesting project. Maybe this year I will start some blog or something.

[@notofinterest]

another comment as it saw i further… a generativ ai for image recognition/object detection is wrong by default. there specialized ai‘s out. CV, OCR namely YOLO, OpenCV, TensorFlow, OpenALPR and so on, they even can run locally. afaik gpt does recognize objects, text and so on, but uses a subroutine for that, maybe another ai in the back. tensorflow is my recommendation for cat detection, i get a notification if a cat is found outside by the cams when they should be home. drawback: not only my cats are detected 🐈🐈‍⬛. a coral tpu is highly recommend to use with.

[@AndreasSpiess]

Did you try ChatGPT? If not, I suggest you try it once. I was impressed what it returns when I upload a picture and ask what do you see. It recognizes details and understands the overal scene. I do not know how they make it and if they use models like the ones you mention. ChatGPT for me is only a "brand" and I am sure they have to use specialized tools for their multi-modal system. Anyway, after these comments I maybe will create a video to compare different sulutions...

[@notofinterest]

@ hello andi. Yes, i did, but manual not over api. Many months now im a paid subscriber of chatgpt and talk a lot with my new friend. Point is: chatgpt is a generative ai (thats where the g in gpt comes from), and this implies that it is trained for other purposes, even if it can do it. For your problem to solve a ai who is trained to detect and recognize fits better. In example: millions of housewifes use a butterknife as screwdriver. It works, but its still wrong.

[@pendarischneider]

AI prompt "Is there a God in this picture?" will help determine if AI has learnt the fundamental nature of cats.

[@AndreasSpiess]

Do not forget: AI can hallucinate ;-)

[@pv9448]

Thanks for your video. I prefer offline solitions. So the cat still gets to drink, when the internet is not available. 😄

[@AndreasSpiess]

You can solve this issue with a simple if statement: If no answer: Run the fountain anyway ;-)

[@jmr]

What about when the internet is down? Maybe with no internet a query triggers a default of yes there is a cat?

[@AndreasSpiess]

That would be good for the cat, I think. Depends on your strategy…

[@jimdean7335]

I used ChatGTP and Copilot on my last ESP32 project. It was my first attempt at using AI for programming. Since I’m not a seasoned programmer I used it to give me ideas on how to approach a section of the code. It often gave me solutions using functions new to me, which I then studied and implemented in my program. I tend to use it as a coach/teacher - kind of like “sitting in the front row”.

[@phillies4eva]

This is the way.

[@AndreasSpiess]

I agree with you and @phillies4eva because I do it the same way!

[@ewm5487]

You can try Microsoft Florence 2 for that. It's very small, very fast. You can run the model even without GPU. Very powerful and "at the edge".

[@AndreasSpiess]

Never heard of it. Where does it run? In the cloud, on a server, or on the microcontroller?

[@melmaki1118]

I notice recent Seees Studio AI gadget workshop very similar implementation orocess. Make it simple to train model, upload and local control outputs on for example Esp32Sense+cam. Your explanation and context great.

[@AndreasSpiess]

Yesterday, I watched their life stream and was not amused about the results of their system. But I will try it once and maybe compare it with ChatGPT...

[@audas]

Thanks. Hailo 8 would be good here - but of course the price and not microcontroller. But Espresiff have released an ESP OPencv for ESP - so can we do our detection locally and save a few cents? (You only mentioned TensorFlow). As things grow and we start analyzing if the wife is in the kitchen and wants a cup of tea to turn the kettle on, if its a bill (to be ignored) or a postcard (to be collected) in the mail box, the mother in law at the front door, or something unwanted in the garden and its sprinkler time - the AI requests to C-GPT will grow.

[@AndreasSpiess]

You showed all the useful stuff done by AI in the future. Hopefully, the list of bad things will not be longer ;-) I am sure we will see a lot of innovation in the AI field in the future and many new chips. Big business right now.

[@valegreg]

The idea is cool and innovative, but what about the environmental impact ? Prompting a huge multimodal AI running in huge data centers every so often, to try and alleviate the power consumption of a small pump... It would have been a lot more interesting to try and run a simple AI model trained at recognizing cats directly on the microcontroller (and would have been a lot more environmentally responsible in my opinion). Maybe a follow-up video can be made on edge computing ?

[@AndreasSpiess]

I agree that the whole project is not a good idea. I should train the cat to drink from still water ;-) But I will not try it with local AI because then, I have to run the ESP32-Cam all the time. And the experiments I did with such AI systems were less than encurraging :-( Not to mention the effort for training.

[@valegreg]

@@AndreasSpiess Can't you reuse the same technique of only powering the esp32 temporarily when the motion detector detects something ? The power consumption at the esp32 would still probably be much higher though...

[@AndreasSpiess]

@@valegreg A good idea. I do not know how long such a model would need to initialize.

[@paullacatus8975]

I think that training the AI system and detecting cats uses much more power than you spare by powering the pump only when the cat is there.

[@AndreasSpiess]

I can only give you the answer of a Business Administration guy: If the price they charge does not even cover their energy cost, they will be bancrupt in thge long term. So I assume, the energy consumption is not too big. But these days, you never know who pays the bill and you might be right.

[@danielwait8555]

EdgeImpulse seems to be pushed very hard lately. I think what they're doing is very cool; but I'd like to know if this is marketing at work or if they've genuinely made such a large impact to cooperate with STM and be a top recommendation by GPT.

[@AndreasSpiess]

I also am a bit skeptical about EdgeImpulse. So far, I did not see good results from AI on small MCUs.

[@DrRChandra]

ChatGPT is not the only "game in town." Google's Gemini could be of great help because AFAIK it's not time-limited, and might know of that recent API you referenced.

[@AndreasSpiess]

I agree. I tried Gemini and was "not amused" (and I like Google in general). Its anwers were not as useful as the ones by ChatGPT. But things move fast, and my judgement might be already outdated.

[@spacewolfjr]

Is Dishka (?) still there too?

[@AndreasSpiess]

No, she passed away last year. She was already old :-(

[@ralfwiessner5944]

😢

[@spacewolfjr]

@@AndreasSpiess oh, I'm sorry Andreas. You gave her the best life a kitty cat could have :)

[@GeorgeToell]

The concept of prompting sounds a lot like pseudocode. I liked the compiler analogy. Humans will design the functional spec and AI will generates the code. Something to experiment with!

[@enjibkk6850]

And where is the fun in that? If I wanted to be a business analyst I wouldn't be a programmer

[@AndreasSpiess]

Indeed, pseudocode is a way of formulating a solution that is not very precise. Programming is the same but more accurate. And usually with much more details. Prompts can even go one level higher, where you formulate the problem, not the solution.

[@AndreasSpiess]

@enjibkk6850 The question is whether you love the result or the process, I think. As an engineer, I love results.

[@morpheusde]

Eine super Projektidee!

[@AndreasSpiess]

Danke!

[@khaledellefi8325]

thank you! can you make us a video in which you explain how to control an ESP32-CAM card anywhere in the world :)

[@AndreasSpiess]

I would divide this problem into two parts: Control ESP32-cam at home and controll home from everywhere. Then you find many projects that solves this problem.

[@____________________________.x]

Thanks, I was struggling with this api call. I could ask it to add 2+2 and get a reply, but python is so complicated I didn’t know how to use the response. I miss BASIC ☹️

[@AndreasSpiess]

I used a JSON library and ChatGPT ;-)

[@marcbruneel1858]

I think the fontain has standard already AI in mind. When you pass it it suggest that you have to drink a glas of water.

[@AndreasSpiess]

Or go to the toilet if you are old like me ;-)

[@Alex000113]

Just like GPU integration some processors now include AI subsystems ( eg NXP i.MX8 Plus).

[@AndreasSpiess]

I assume the price tags are higher, and the power consumption, too. But they are nice devices to play with.

[@MrZiemwit]

simple rpi or rpi coral should work also, maybe there is some small ofline model on rpi, frigate also should work on one camera that do basically nothing all day

[@AndreasSpiess]

You are right. I would put it in the "expensive" department. Before this project, I tried the same with Frigate (without coral) and the fan of my HA server ran 100%. The CPU usage was very high. That is why I changed to the aproach presented here.

[@MrZiemwit]

@@AndreasSpiess i was hoping that frigate would only work for recognition only after pir detect object not all the time as usual silly but in my mind even esp32 could takie cat ai task, everything is triggered by pir not in constant recognition mode like 30fps ;)

[@csongorvarga]

Small world, I am an SAP CRM consultant :)

[@AndreasSpiess]

Cool!

[@roflchopter11]

This is interesting, though I'm not a fan of sending a near-realtime image of my house to some server somewhere.

[@AndreasSpiess]

It probably depends if the picture is made in the bedroom or if it shows the surrounding of a cat fountain ;-)

[@greenpotatochip]

Nice. Info packed

[@AndreasSpiess]

Thank you!

[@adriyyyy]

Nice Ideas 😁 thanks

[@AndreasSpiess]

You are welcome!

[@efimovv]

In any case I only plan to use local systems in home automation which works without internet.

[@AndreasSpiess]

As said in the video: The human creates the strategy!

[@WEKXSU]

Please do some videos on Arduino portenta

[@AndreasSpiess]

Why should I use such an expensive board?

[@reversetransistor4129]

You can build your own AI code with tensorflow in Phyton working the picture in some one board pc, that a nice project. I tried to do that in C direct in a microcontroller, I broke several fingers and a arm, perhaps next time, lol

[@AndreasSpiess]

This is what ChatGPT proposed. However, this is way more complex (as you write), and so far, I have not seen convincing results from such small models.