Understanding Logging: Containers & Microservices

Subscribe to show your support! goo.gl/1Ty1Q2 .
Patreon 👉🏽 / marceldempers
Today we're taking a look at the concept of Logging in microservice container environments.
I go over two concepts, standardised and centralised logging.
How to keep your codebase simple by logging to stdout.
How to gather all the logs from a distributed system and bring it to a central place with FluentD.
This is a conceptual video demonstrating the concepts we'll need in this series.
In a future video, we'll dive into FluentD, the ELK + EFK stack and deploying all this stuff on Kubernetes.
Checkout the source code below 👇🏽 and follow along 🤓
Also if you want to support the channel further, become a member 😎
marceldempers.dev/join
Checkout "That DevOps Community" too
marceldempers.dev/community
Source Code 🧐
--------------------------------------------------------------
github.com/marcel-dempers/doc...
If you are new to Kubernetes, check out my getting started playlist on Kubernetes below :)
Kubernetes Guide for Beginners:
---------------------------------------------------
• Kubernetes development...
Kubernetes Monitoring Guide:
-----------------------------------------------
• Kubernetes Monitoring ...
Kubernetes Secret Management Guide:
--------------------------------------------------------------
• Kubernetes Secret Mana...
Like and Subscribe for more :)
Follow me on socials!
marceldempers.dev
Twitter | / marceldempers
GitHub | github.com/marcel-dempers
Facebook | thatdevopsguy
LinkedIn | / marceldempers
Instagram | / thatdevopsguy
Music:
Track: J3bii - Existance | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / existance-prod-j3bi
Track: souKo - souKo - Parallel | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / parallel

[@RonnyBadilla]

Just discovered your channel and I’m impressed, I have seen just two videos and both helped me to have a better understanding. Keep it up, you got a new subscriber!

[@BrianThomas]

This is the 3rd time that I've found your channel when looking for something IT/tech related. I love your content. Thank you very much @That DevOps Guy for taking the time and putting this stuff together. You've made a subscriber out of me. I know this video is a bit but IMO you're doing a great job my friend and keep up the great work.

[@Jakub-yu2vy]

Great video ! I really like the way you present. I am looking forward to watching next part

[@ashwaniahuja]

You are rock in docker and kubernetes , I have not seen any other better tutorials like this

[@Being_Joe]

It is like you have been reading my "things to figure out" list and making videos off of them. Looking forward to the rest of this series.

[@MarcelDempers]

👀👉🏽👉🏽 watchin you! 😎

[@RezwanRahman]

cannot wait for the next video ....pls up as soon as possible ,,its very well explained .

[@ruchk3872]

Very useful and simple way to explain...thank you!

[@carlomagnogonzalezvillalob3255]

Increíble toda la información que compartes, muchas gracias por enseñar tus conocimientos. Saludos desde Costa Rica

[@abeard1]

Love your channel, greetings from Warsaw :)

[@motizukilucas]

Thank you! As always great quality video

[@asifsadiq1693]

Your video are very informative, keep posting

[@imanhpr]

thank you very much this video helped me a lot.

[@amitpadgaonkar8830]

I love your channel. It is truly underrated in terms of number of subscribers. One quick question. Can you please the system that you are using? the processor, memory, desktop brand etc. I want to buy the same system for me.

[@TheMrJoshua]

very helpful like always

[@nicacton]

Lifesaver! Definitely subscribing

[@nicacton]

One nit-pick: when the icons AND the background are moving like 1:50 it can be a little harder to follow. But then again, maybe it forces me to expend more mental energy and focus more 🤷‍♂

[@yovangrbovic7692]

Thanks Marcel!

[@BiohaZd5]

All those fast moving backgrounds hurt my eyes.

[@MarcelDempers]

I was wondering about this, thanks for the feedback, will wind this back in future 💪🏽

[@amlife180]

I like the graphics, helps a lot of explaining complex topics. Great content! Keep up the good work!

[@khoavo5758]

@@amlife180 1:22 That background does anything but help explaining the topic.

[@sujeetkumar.]

Awesome 👍

[@oraculotube]

Only top videos 👌

[@dufferzzzzz]

Arch btw! Great video as always. I am wondering if you would be able to do a video on SocketIO/WS generally on k8s, I initially ran into some roadblocks with Nginx ingress, Redis, and setting affinity correctly. Fixed it now - but others may be curious. Everything 'seemed' to work fine on one instance, but as soon as another ws server instance was running; k8s' affinity would default to selecting another instance on next connection resulting in 400 errors with Unknown Session ID

[@El18Cucuy]

you can send log remotely using json driver or fluentd,etc drivers.

[@muthukumarramu3149]

Thanks for your video, I need to gather all container logs... in that case.. what are the parameter needs to be added in the production compose files. please help me

[@phuongnguyentrong3084]

Awesome

[@jafarsadique2746]

Please make a detail video on collect containers using fluent-bit and redirect to azure log analytics

[@9691886126]

It nice..can you cover a video on sending fluentd logs to cloudwatch ? So developer can watch app logs from there to see if pod crashed for any reason

[@hardikpanchal9726]

Hi sir, I am using Rancher to maintain my all applications in cluster & their logs are stored into container path var/logs/app1.log using log4j file appender. So is it possible to download that logs from container to my local system?

[@trollerninja4356]

hey can you help me out? I have included fluentd in the list of services in the docker-compose file for my node app...it has 3 containers - mongodb,nginx and node app containers...the 4th one being fluentd container..,I see the logs/fluentd directory created but no logs are written to it...m using a windows host machine...I would like to mention that I had excluded the root prop in the compose file for fluentd service...could that be a reason for this issue?

[@xDeadlyWarriorX]

Please, how can i enable Log Rotation? There is a JSON LOG that has 10G already but i know i can not clear it since it can generate some erros with the JSON structure

[@pedro-br1mo]

Any chance you follow up with this + with data coming from rabbitmq? (RELK)

[@rayudu1180]

Hi Bro, can you please make a video on EFK stack on minikube kubernetes 🙏

[@L84netlol]

Good job! But the logging directory not yet on github,plz.

[@8982121541]

Hi there there is one situation i am stuck at inside my electron container there is one reporting service who continuously generates log which inside /var/lib/docker/volumes/electron/electronlogs.log here i want to use this command log-opt max-size=10mb max-file=10 how i can write it in a script

[@sirishabala7423]

do you have any series with data dog?

[@jonathanhurd1565]

Very interesting/useful. Do you have any experience/thoughts on Loki versus fluentd?

[@MarcelDempers]

I've not worked with Loki before, but I'm super keen to cover it in this series 💪🏽

[@ukaszdya223]

could you make a movie about network security in kubernetes e.g. Calico

[@catsgotmytongue]

The movement and colors hurt...

[@_kovshenin]

Great video, but man, the blinking, the moving, so hard to concentrate and watch, can probably cause a seizure even without history of epilepsy :(

[@HieuMinh-ss3ut]

great video. Thanks! but i have problem with time fortmat, could you help me . I have prolem with time zone log fluend! when i run any service and fluentd service. fulend was able to get the service's log but the log time is incorrect time displayed on log is path + time + ".log" (testlog.19700101_0.log)