Raspberry Pi IoT Server Tutorial: InfluxDB, MQTT, Grafana, Node-RED & Docker

[@kilohertz684]

Brilliant! Thank you so much! I've been puzting around for 3 days watching other outdated channels, trying to get this working and you guided me thru in less than an hour, "how cool is that".

[@nathanpendergrast6917]

Maybe I just hadn't looked hard enough before but I think you're the first person I've seen explain grafana this well.

[@frollard]

After what feels like 45000 tutorials, thank you - you finally gave me the insight needed to strip out unnecessary data and CHANGE the incoming data into a payload that influx was happy with. I now have a happy little chart of how bad the air is during the Canadian wildfire season.

[@familyplans3788]

This is ,by far, the best video I have seen for explaining an IOT server, thank you

[@LincsOracle]

Just a note, when setting up the MQTT server in Node-Red you can type in the name of the docker container as it is running in the same stack as Node-Red.

[@pedromorgan99]

Thanks

[@LearnEmbeddedSystems]

Thank you for the generous super thanks! I really appreciate your support :)

[@JoachimGrossmann-j6w]

Thank you for this excellent project & tutorial. It was my first step into the world of microcontrollers.

[@r4nd0mstuf]

After struggling for a whole day trying to get mosquitto to work on my raspberry pi, following your video had me up and running within half an hour. This is a fantastic tutorial, thank you so much for the help ❤

[@ika9]

Thank you! Your video is straightforward and to the point. Other videos were a complete waste of time. I'm glad I found your videos

[@nosafetyswitch9378]

Amazing guide! I watched it a couple of times and after a few hours I had my Grafana up and running, mind you I had never done anything MQTT before. For now I am just logging the room temp with an ESP32 and an LM35 temp sensor but I will be adding a lot of stuff soon and hopefully some solar production data in the future. Thanks a lot!

[@kylemfwelch]

This is a really excellent tutorial. Every step is clearly explained in detail. Just one issue: it is generally not advised to use a leading forward slash '/' for the root an MQTT topic. But that's a minor quibble

[@SDNTechForum]

Amazing Video, cannot thank you enough. I tried it myself and here is feedback. The github code you shared is having issue with NodeRed, see error below - I managed to install other containers by skipping nodered and later installed nodered as systemd. It would be nice if you can make your github repo public in that way users can report issue and can pull the working code. ############ Start Stack: docker-compose up -d --remove-orphans WARNING: Some networks were defined but are not used by any service: nextcloud Creating network "iotstack_default" with driver "bridge" Building nodered ERROR: Cannot locate specified Dockerfile: Dockerfile

[@statters]

Great helpful video and website. One point, on the website it says to use 'docker-compose up' to check the status of the containers whereas you use 'docker-compose ps' in your video. I really appreciate the effort you are putting into your videos and supporting material.

[@manasXP]

Great tutorial to get started with IoT stack in R-Pi4. I am going to use this in a CM4 and use node red + mosquito talking to IO module downstream. Docker surely makes things flexible and easy.

[@yannkitson116]

Great tutorial... I am going to get the data from my netatmo devices and data from my "solar system" into this.

[@edinetgrunhed6000]

Thank you for this video, i learned about portainer,grafana and influxdb in same video. thank u very much

[@srikumarc]

Fantastic Video. Very crisp and to the point. Helped me learn the whole stack at one go.

[@LearnEmbeddedSystems]

Great to hear!

[@srikumarc]

@@LearnEmbeddedSystems Is it possible to build this stack in the cloud? In other words can this container be ported to the cloud? A video on it would be helpful.

[@ceecee_22]

Really good tutorial. I wish I watched this before setting up my IOT server

[@eveb843]

Great Video, I am bumbling through this on my own system, I intend to use this to monitor my solar install, hopefully remotely, so much to learn so it may be sometime before it goes live. Thanks for posting ;)

[@smaroukis]

I've been debugging my IOTstack install for a week now -- I found that I had a corrupted SD Card. Best video I've seen so far getting right to the point, explaining what **should** happen. It would be nice if it included some more gotchas - I've been battling with my v2 B+ which is pretty slow running the IOTstack menu.

[@LearnEmbeddedSystems]

Maybe I could put an FAQ or common problems and how to solve them in the written article? Would that be a good compromise?

[@stevebot]

Storage problem is my biggest concern with Pi. Consider one with onboard eMMC or a M.2 NVMe. There are a few options out there using the CM4. Seeed minirouter, Waveshare IO boards- base B had a RTC, or the DFRobot mini router to name a few. The latter two require finding a CM4, which everyone says is on back order until Dec 2023.

[@smaroukis]

@@LearnEmbeddedSystems that would be helpful, although I've just went the route of installing locally without docker containers and it is working

[@lpz3sn]

Running a database where data like temperatures etc. are constantly written with an SD card is only for people who enjoy reinstalling Raspis. USB-SSD/HDD is mandatory for this application.

[@olesik]

@@smaroukis I had the same problem, i have 5 corrupted sd cards on my desk now, I was about to try this videos method, I installed each item individually and the docker install script was killing sd cards

[@Darkk6969]

Very cool video on the Pi. For Arduino the ESP8266 NodeMCU is a cheaper option as it already have Wifi built-in and better power management. ESP32 would work too. Both very small form factor.

[@michaeljohnson9758]

100% esp devices are better for 95% of iot projects and they are cheaper and smaller. Only problem is they are 3.3v devices so you might need a logic level shifter.

[@austinsandford]

I use wemos mini clones plus bme280s for very compact temp/hum/pres sensors

[@dang48]

Well done and I have a number of projects in mind that will use what was explained here.

[@AJB2K3]

Thanks for this guide, I have referenced and linked to it because it helped me build a server on an M5Stack CM4 Stack.

[@paulhatfield498]

Very cool!! Thanks a heap for these awesome videos.

[@RichardPaul93]

Thanks for this video, alongside the ESP32 IoT sensor node tutorial, both have helped me to quickly and easily get this connected! I'm looking for next steps - a graphical front end to show a user graphs from just selected nodes, buttons to perform actions, sending mqtt back the other way to the ESP32, etc. Any plans to continue videos on this IOT topic? Or anyone, any keywords you can point me at to research and learn appropriate components for this tech stack, such as a user front end, and user/device management?

[@josetuz]

Incredible! Thanks for that. From Argentina.

[@MashiurRahman-ew8eq]

This is a very useful tutorial and a nice presentation!! Just to be clear regarding viewing the containers using Portainer, 1) do I need to have an account in Portainer? 2) Is there any limitation on device numbers under the free account?

[@chrisrudy1969]

That was absolutely fantastic! Thank you so much!

[@Nokless]

love the video !! can you make another video about the hardware setup plz ? and how the arduino sent data to raspberry ?

[@richsadowsky8580]

Really great video with written instructions. Thanks.

[@navinarajoo7426]

Hello, this was a great tutorial! I got a little lost at the Nodered part. How do I know what is the Nodered Json interpreter that I should use?

[@PLCspec]

awesome tutorial

[@mitchellboswell186]

Great video! Nice work. I would love to see the set up of the sensor node

[@Bob-Fisher-6120]

Hey I also noticed that the "influxdb out" pink dragbox you used in the video at around 7:48 no longer seems to be in the storage tab, or in the left toolbox space at all. What happened to it?

[@pllagunos]

This is a really amazing video. Thank you. Any chance of you doing a tutorial on how to deploy Grafana Cloud connected to the local RPi Server? That way one can monitor devices away from home and even have alerting through Grafana IRM's mobile app.

[@radderek007]

You are inspiring. Thank you for sharing.

[@markshaz8691]

Really well explained and informative video, subbed.

[@patrickpuentelml4958]

Almost every single thing is very good explained in this video an exception of HOW CAN I MAKE THE INTERFACE COMMUNICATION BETWEEN MY ESP32 (WHICH IS READING THE DATA SENSOR) AND NODERED?

[@Lucas-xu5oq]

Well this video was exepsional congrats! It worked fine for me, however for some reason I can't add the libraries of the functions on the pallet in node-red.

[@trevorwatson5041]

Mine did not work as when i want to start the stack it returns an error that three of the apps are obsolete. Just wondering if you had an issue like this or any way to resolve

[@akshatmodi4843]

Hey LES! Superb video thank you so much for the information. Just had one question, I have certain scenario's where people switch of the Raspberry Pi at the end of the day's work. So when they restart will all of the tools namely, Mosquitto, Influx-DB, Node-Red and Graphana or essentially the docker container reboot as well? Thanks again for the fantastic video.

[@abhishekpaul97]

Hi, extremely helpful video! Can one use the sensor data to control GPIO of another raspberry pi using MQTT?

[@PurnimaMishra-g7o]

Absolutely brilliant.

[@ShriyaSinhaBLC]

can even upload these data to an app and control the actuaters from it only?

[@willembos01]

Very good tutorial!! Thank you I will experiment with this.

[@edinetgrunhed6000]

subscribed done, please make updated video for this some additional dashboard components

[@michaelandersen1161]

Great video....when I try to install thsi with a static IP adress set I can not connect to the server. When must I set the static IP adress?

[@feedmemusictv]

Thank you. Nice; quick and succinct. One comment.Why skip over Portainer when - and this is from limited knowledge - I think you could have set everything up from within Portainer

[@jimaustin3608]

(3:29) " fresh install of headless Raspbian OS ... (3:37) we have a video Linked In the cards above ... " Anyone know where this link is?

[@rafaelk2xt]

Well Done. Thank you. Do you know if can I do that on an EC2 AWS instance?

[@simonfunghk]

It is very good idea to let i Pi as IoT server as all IoT things can be made by myself.

[@sanketss84]

Superb video ❤

[@ChenLaw88]

Thank you! Very nice, well explained and easy to follow, bookmarking and saving your website tut too. I've got a couple renewable energy projects that I'm designing, and this should work perfectly for some monitoring hardware (tachometers on wind turbines and voltage readers on solar arrays). Also thinking about a few other metrics around the house I could definitely appreciate collecting :D.

[@jungscott9340]

very useful iot server guide..

[@RodrigoRocha1980]

Resumo do vídeo "Raspberry Pi IoT Server Tutorial: InfluxDB, MQTT, Grafana, Node-RED & Docker" em 5 pontos: 1. *Objetivo do Projeto:* O tutorial visa criar um servidor IoT utilizando um Raspberry Pi para coletar dados de sensores, como temperatura, pressão, umidade e qualidade do ar, através de um Arduino Uno emparelhado com um sensor BME680. 2. *Protocolos e Ferramentas Utilizados:* Utiliza o protocolo MQTT para enviar dados em formato JSON pela rede para o servidor Raspberry Pi. Explora aplicativos como Mosquitto (broker MQTT), Node-RED (interpretação e envio de dados para um banco de dados), InfluxDB (armazenamento de dados do sensor) e Grafana (interface gráfica para visualização dos dados). 3. *Configuração do Ambiente:* Emprega o Docker para rodar os aplicativos em contêineres, facilitando a configuração e implantação dos componentes, tornando possível a execução desses aplicativos em diferentes dispositivos. 4. *Passos para Implementação:* - Atualização do sistema do Raspberry Pi e instalação do "iot stack". - Seleção dos pacotes desejados (Grafana, InfluxDB, Mosquitto, Node-RED, Portainer) usando o menu interativo. - Configuração do InfluxDB para armazenar os dados do sensor. - Configuração do Node-RED para interpretar e enviar os dados para o InfluxDB. - Configuração do Grafana para criar um painel visual dos dados coletados. 5. *Finalização do Projeto:* Após a configuração de todos os componentes, é possível visualizar e monitorar os dados dos sensores em um painel gráfico oferecido pelo Grafana, consolidando assim a criação de um servidor IoT funcional. É importante ressaltar que o vídeo fornece um guia detalhado de passo a passo, incluindo comandos e configurações específicas para implementar esse servidor IoT com Raspberry Pi.

[@spillagonner]

Please note: In the video, you state that you can check what containers are running with the command 'docker-compose ps' while on the related web page you show the command as 'docker-compose up'

[@ivanronda9583]

docker-compose ps shows you status of docker containers, docker-compose up execute the docker-compose.yml file(that is something like a config dile for docker) you also use this command only when starting docker

[@makmakmp]

That's an excellent tutorial !! Congratulations !! I would like to ask in case we have many sensors in /home/sensors. How can we get the unique id of each sensor and show in grafana? I've implemented your project with wemos D1 mini + DHT22 sensor + LCD 16x2 (parallel connection) and raspberry pi 4 (8Gb) as the server. The point is that viewing grafana i only see that data comes from sensor_data which is the db name from influxdb.

[@duckyblenderold]

Very informative video, thanks!

[@237311]

Great tutorial. I have so many Pi 3s lying around, and wanna know if I can use them on this project? Now Im interested to see some videos on how tos with the sensor nodes.

[@LearnEmbeddedSystems]

Pi 3 should work just fine! ESP32 sensor Node video hopefully coming this weekend!

[@austinsandford]

Great tutorial, would be useful if Grafana could easily plot multiple values on one graph so that many sensors could be viewed easily

[@janhatton4405]

Hi, yes it can do this you just add a second query panel when you are setting up the graph (+ Query button) and it outputs to the same graph

[@austinsandford]

@@janhatton4405 Thank you

[@Bob-Fisher-6120]

Hey quick question, so when you get to the menu part and have to select nodered to install in the build list, do you have to download every single package that nodered has or can you just install the default ones shown?

[@energyideas]

Looking into this for a mesh network of environmental sensors including ground level pollution from vehicles. Thanks.

[@AndersJackson]

Great video. But as I understand it, MQTT messages should not have a starting "/".

[@nirojkumarsahoo]

Very informative. Definitely going to try this.

[@LearnEmbeddedSystems]

Let us know how you get on!

[@eveb843]

Can I ask a question, how do you enable the USB serial ports in docker, I want to use Arduino nodes to collect and send data over serial, is there a file I can edit for the Node Red container?

[@thomasgrassmann6724]

Great video! Thanks!

[@SobhitPanda]

Absolutely awesome content 😊 + Your voice 🌟

[@LearnEmbeddedSystems]

Thank you!

[@trevorcook1427]

This is great, going to try and connect the data from my Victron solar gear, Ill get back and let you know how I get on

[@MattMiskinnis]

I actually want to do the same thing. Have you got yours working with victron?

[@contouraerials]

in theory this can be done on a headless pi zero from a pi because youre using the ip address to setup all the interfaces? (im a newb)

[@adjbutler]

love it!😄

[@alsamaraee2]

Hi, Thank you for this professional tutorial. I followed it and my ESP32 client can connect to the MQTT server jus if both the Raspberry pi and ESP32 on same wifi network. What am I missing?

[@oskarbrenner13]

I think MQTT is run upon a network, such as Wifi, bluetooth e.t.c.

[@shivambandekar5329]

Can you do it with websocket and through public internet?

[@thorstenfricke3257]

have you ever used iobroker?

[@uffeibsen8872]

great tutorial

[@TOMTOM-nh3nl]

Thank You

[@KW-hy2bu]

Can I use esp32 instead od arduino with jolly module? Does it have a chance to connect with RPI?

[@Scobbo]

So one topic is for each arduino with the sensor suite, so each room has a different topic? or is it a bunch of sensors sending data to one arduino and it is transmitting different topics per room?

[@toastedregret1601]

Each arduino is an individual sensor node in the video. There are cheaper microcontroller alternatives with more IoT features (timers, power modes, WiFi, BLE), such as the ESP32. Arduino in this case was used as an example.

[@Thomas_Grusz]

Great video, thanks. I would be interested in a sensor node video as well.

[@musandiaye9231]

Very useful. How can I do this on windows or mac? Are the instructions the same?

[@YigalBZ]

Where do you discuss the static IP? Should this be set in the router?

[@toastedregret1601]

It should be set within your router, otherwise it will change the next time the raspberry pi power cycles. The video didn't mention that. Go to the IP address for your router, type the username and password (the very first time, it should be "admin" , "password"), find the raspberry pi in the list of devices on the router, view more info, and set "reserve" IP address.

[@homelessgee957]

Do you have any suggestion to decrease power usage to change it to make a reading every 1 mins instead of few seconds. And using only solar energy from a panel? To make sure its fully autonomous

[@toastedregret1601]

You could look into lower-power mode on the device you are using, and use a watchdog timer or some other peripheral on the microcontroller to power it back on after X time to take another measurement.

[@burstfireno1617]

Super tut bro

[@jesus_songs3005]

0.01 is this the circuit connection i need to make

[@BAXEN]

Has anyone been able to enable and enforce users for the mqtt container?

[@SamirMishra6174]

Very helpful

[@Jannot_Gailer]

Very well done.

[@sansy9568]

Amazing tutorial. May i have a question ? Say i finish all the installation above, is it possible that my system can operate completely offline ? ( reading, workflow, visualization, modifying … ) Like i connect those 2 boards to the same network which doesnt have internet connection.

[@LearnEmbeddedSystems]

Yes, it shouldn't need an active internet connection

[@RudW0lf]

Based on my 3+y expirience InfluxDB wrong choice for time-series data. It can't be used on production setup. The better way - prometheus, or graphite with clickhouse backend.

[@shobhamaurya1886]

If I'm using influxdb on web browser instead of installing it to my pc, will it not work? Because when I'm connecting nodered with influxdb it's not binding with influxdb. I've used the host name for influxdb as my url and port as 8086.

[@SamuelSantos-iq4iz]

Hello, excellent video! I'm Brazilian and I don't speak English (I'm using Google Translate lol) and I would like to know if I can use a conventional computer with the Raspberry Pi system installed in place of the Raspberry Pi board. Thank you in advance!I would also like to know if I can use the ESP8266 ESP-01 WiFi Module for the wireless connection, as I use an Arduino board without a built-in WiFi module.

[@limacs]

Great video, also interested to see different sensor nodes that you can make and setup, thank you!

[@LearnEmbeddedSystems]

Thanks for your feedback! Will look at making a couple of videos on sensor nodes then :)

[@patrickpuentelml4958]

I have a question. This thing (API?) called "IOTstack" was made using docker?

[@RixtronixLAB]

Nice video, thanks :)

[@cernejr]

I am confused about Node-RED. Is it design-time-only tool? Or does it have to run also at runtime, after I deploy my solution?

[@GiuliaLeali]

Hello! Great tutorial, thanks! However, when I build the docker-compose.yml.file containing Mosquitto, Influxdb, Grafana, Node-RED, and Portainer-CE and I type the command "docker-compose ps" I don't get any list of Mosquitto, Influxdb, and so on. I get the following error responses (see the attached picture). I've been trying to troubleshoot it and rewrite everything from the beginning, but the issue persists. I've tried to add some other commands found on the internet, but no luck yet. Let me know if you have any ideas on how I might solve this problem. Thanks :)

[@sy3002]

Which raspberry bi model is used here ?

[@vitusyu9583]

I am a novice in IoT, and appreciate if you could explain how to do the sensor board part. Thanks!

[@toastedregret1601]

The video uses an arduino as the microcontroller unit (MCU), but if you don't want the hassle of connecting a WiFi module to it and managing the communication between the two modules, you can get a microcontroller with wireless capability (bluetooth, WiFi) embedded directly into it. I'd reccomend using an ESP32.

[@sajadmohammadi3155]

Hello, I want to set the parameters of an industrial drive using a node through RS485 serial communication. In this project, I need to use the hex codes of an industrial drive and control the drive using the hex codes. Can you help me with this project with node-red-node-serialport?

[@javieralonso4054]

Hi. This is very good work However, I'm trying to develop this with my raspberry pi and an arduino MKR. It happens that the WiFi library has its init method as private method, so no way you can start the WiFi in AP+STA mode (at least, I've not been able ot do it). Is there any work around. I think this is the cause the sensor node only gets one data through the node red workflow.

[@HolgerEich-k9h]

Hi Javier, I’m interested in using the mkr, too. Have you received any feedback or found a solution? Best, Holger

[@vedantjadhav4525]

after configuring everything the datatbase is not showing any measurements

[@guillaumecuadrado8096]

Hi ! I got the same issue ! Did you managed to fix it ? thx a lot !

[@vedantjadhav4525]

@@guillaumecuadrado8096 yes just experiment by replacing sensor with sensors and vice versa in few area he has put different spelling

[@sanjeevjoshi6441]

it worked fine for a few hours and then started returning this error ( rc=-2Attempting MQTT connection...failed ) what might be the issue