Solving Engineers 4 Biggest Airflow Problems

In this video I'm going to show you the four biggest problems engineers have with Airflow and how to solve them.
Try out Astronomer 14 days for free: www.astronomer...
Jump marks:
01:09 Managing Airflow Deployments
04:09 Good Development Environment & Easy Deployment
06:52 Source Code Management & CI/CD
09:30 Observing Data Pipelines
If you want to learn more about Data Engineering:
Learn Data Engineering & Airflow at my Academy at learndataengin...
#dataengineering #bigdata #datascience #airflow

[@andreaskayy]

What kind of problems do you have with Airflow? Would love to hear them.

[@timiayoade9953]

Hi Andreas, I am having problems deploying airflow locally on my computer using docker-compose file, it slows down my PC whenever I run it. I think this is because I am on a window machine with 8gig ram. Can deploying Airflow using astronomer solve this problem for me?

[@andreaskayy]

Try it out, but as I said in the video Airflow needs multiple containers to run and that takes memory. 8gb seems low, but maybe the Astronomer dev environment is more optimized than what you have

[@timiayoade9953]

Okay will try it out

[@Girijamma-zr1nh]

​@@andreaskayy9:38

[@Sanusiinc]

do you have this in your Data Engineering course on your site?

[@andreaskayy]

Not yet, but it's coming. I have documented all the steps and going to record it as a course with source codes and everything

[@Sanusiinc]

@@andreaskayy I email you ! I don’t know if you can give me more discount on the lifetime data engineering

[@Shub.ham_meena_5757]

Yes i face this problem