Build a smart IoT device with TensorFlow Lite and Raspberry Pi

Learn how to build smart IoT devices using TensorFlow Lite and Raspberry Pi. Discover pre-trained object detection and sound classification models that you can use off-the-shelf to "see" and "hear" the surrounding environments. If the off-the-shelf models don't satisfy your app’s needs, learn how to collect training data and train your own custom models using TensorFlow Lite Model Maker.
Resource:
Download Raspberry Pi OS → goo.gle/3ysjJA9
YAMNET audio classification model → goo.gle/3wlwSbx
Raspberry Pi's audio classification sample → goo.gle/3L2AJQr
Raspberry Pi's object detection sample → goo.gle/3GaABw3
Coral's IO talk → goo.gle/3M4HMJo
Train a custom audio classification model → goo.gle/3wmcMhk
Train a custom object detection model → goo.gle/3wmcVBo
TensorFlow Forum → goo.gle/3vZy9Gh
Speaker: Khanh LeViet
Watch more:
All Google I/O 2022 Sessions → goo.gle/IO22_AllSessions
ML/AI at I/O 2022 playlist → goo.gle/IO22_ML-AI
All Google I/O 2022 technical sessions → goo.gle/IO22_Sessions
Subscribe to TensorFlow → goo.gle/TensorFlow
#GoogleIO

[@KubinatorLP]

Hey, how do I run the picamera on 64 bit OS? For me the cam only works on 32 bit versions

[@ErlTarrega]

Great video! Thanks for posting this guide! 👍

[@elvenkim]

I have problem running the tflite-model-maker, it hangs at the "200kb", anyone else have same issue?

[@shawarma3636]

Is the Raspberry Pi Pico supported?

[@sarathpavithra]

Thank you for this wonderful introduction.

[@miketony2069]

Great introduction!

[@yvansnippeur8971]

Warm greetings, thanks for the video. Please how can i run object detection with an IP camera?

[@khanhleviet5416]

You'll need to figure out how to get images from your IP camera. Usually IP camera provides an network endpoint to get the frames from the camera. You can download the latest frame, convert it to the RGB format in numpy array and the rest would be the same as detecting objects from the Pi camera as I explained in this video.

[@thehappycoder3760]

Brilliant, thank you

[@shaheermubarak]

Model maker installation takes a decade, how to fix this?

[@sam-vv6gl]

when will tensorflow lite support raspberry pi pico micro controller ?

[@aubriewest1135]

You can run TFLite models on RP Pico with proper optimizations. I guess the biggest limitation is it's working memory.

[@aubriewest1135]

The framework is called TFLite Micro and it supports ARM Cortex M4+

[@khanhleviet5416]

Raspberry Pi Pico is a microcontroller without the Linux OS so it needs a different framework to run ML models: TensorFlow Lite for Microcontrollers. Check out our documentation for more details and samples. www.tensorflow.org/lite/microcontrollers Due to microcontrollers' limited computing power, they can only run a limited set of ML models. Both the object detection and audio classification I introduced in this video can't run on microcontrollers.

[@flickmew99]

this is always what I want to learn, btw can we object detect something difficult like dust/debris ? I wonder if later I would pour a flour or any random powder, could our machine find the difference ?

[@khanhleviet5416]

You'll need a custom model for that. Take a look at this video to find an in-depth instructions about training custom object detection model. kzbin.info/www/bejne/Y4vcd4yknqyIqdk

[@flickmew99]

@@khanhleviet5416 thank you for replying sir, correct me if I'm wrong, so basically tensorflow already got bunch of image sample/model on their library, but if we got something particular that might not included in the library we can train a new model/sample, is this true ?

[@khanhleviet5416]

@@flickmew99 That's correct. We published many pretrained computer vision (e.g. object detection) models on TensorFlow Hub (tfhub.dev) that can detect general objects. If you want to detect specific objects that aren't supported by the pretrained models, you'll need to collect training data yourself and use TensorFlow Lite Model Maker to train a custom model.

[@flickmew99]

@@khanhleviet5416 got it, thank you for the explanation, I'm going to study that for my project

[@jevalaggaantamilalagan6111]

@@khanhleviet5416 sir, I just have a question.. What is the model architecture for the pre trained model available on tfhub.. Is it int8/uint8 or float 16.. Because I tried running my own int 8 tflite model that I converted from tensorflow using this program but it shows some errors

[@nvfpv]

finally i can put my pi 2 to work >:)

[@jhonshephard921]

Please don't. I assure you, you need a minimum pi 3, which will also be very slow.

[@nvfpv]

@@jhonshephard921 why did he mention pi 2 though

[@levietgiakhanh9818]

Yes this can work on the Pi 2 but the frame rate will be lower than Pi 3 and Pi 4 for the obvious reason.

[@MauricioRamcerva]

So, no raspberry pi W Zero, where is the fun then

[@YS-lf9xw]

Is he Korean

[@duycuongpham7407]

Vietnamese

[@YS-lf9xw]

@@duycuongpham7407 merci

[@memyselfandi8544]

I can’t believe I used to think this junk was important. Fertilizer is important. Baby formula is important. Gasoline is important. My Raspberry Pi collection, worthless.

[@TradieTrev]

ROFL!! It's true though

[@porcorosso4330]

I guess you can potentially do quality control on baby formula with a raspberry pi. You can also hook up sensor to monitor compose for making fertilizer with a pi... Of course, maybe you don't monitor your fertilizer pile. You just let it sit for x amount of days and ship them off to be processed...

[@memyselfandi8544]

@@porcorosso4330 What good is electronics when everyone has lost their minds and is seeking global thermonuclear war?

[@Peter-oz1oo]

So you´re saying that only one thing can be important? If your pi collection is worthless, would you mind sending it to me free of charge?