Apache Airflow One Shot- Building End To End ETL Pipeline Using AirFlow And Astro

[@krishnaik06]

Check out my new Udemy complete MLOps Course with 10+ end to end ML Projects: www.udemy.com/course/complete-mlops-bootcamp-with-10-end-to-end-ml-projects/?couponCode=SHARING

[@adityapatel907]

sir, we need new udemy coupon for ' DSA bootcamp + Leetcode Exercise ' . please sir ! do it quickly.🙏

[@tanmaybagul2957]

Realizing the Power of Airflow 😎 this video help me have high level understanding of what i was doing from last 3 years

[@oceanview3165]

It's very straightforward and helpful! Thank you!

[@SandipJadhav-ls1kg]

when will be big data engineer cource we can expect krish sir

[@gagansingh3481]

It's Etl , etl can be done though power query and load it to postgre sql .

[@gagansingh3481]

It's Etl , etl can be done though power query and load it to postgre sql .

[@bp2807]

Krish Sir you really bring very good course, please include kubernetes. Also which tablet and software you use to write and explain sir. Love the way you teach. Kindly answer🙏

[@tusharmodi8344]

Sir for your mlops course do we also need to have aws subscription to do project work or is it not needed we can do something in free tiwr

[@krishnaik06]

U can do in aws free tier

[@harshsinha12]

​@@krishnaik06sir the AWS Environment set-up is giving some errors, i have tried various fixes

[@ARkhan-xw8ud]

@@harshsinha12 i have taken of my friend

[@bralabala]

what I would have preferred to see is how you configured it to work with local files rather than calling an external api. since I posted this comment, I have now solved the problem myself. the short of it is to use astro's docker-compose.overrride.yml file to mount your local drives so it can communicate with the local directories

[@BechirMathlouthi-n9h]

very insightful !! thanks for the effort !

[@gillfromamritsar]

I got the point it's base one but I have some senerios to thinnk about -> What is Retry mechanism if any of the step fails -> If we have a senerio where a large data should be loaded then from one DB to another then how to resume from a point we left of, for example there are 50k records to be loaded from one loc to another then and process broke at 25k then it should resume when next time it runs instead of running 50k from point 0

[@mohitjoshi8984]

Sir talk about Kubernetes please

[@jaggyjut]

Awesome. How is airflow different in comparison to Microsoft fabric

[@dineshbiswal2775]

What tools are you using for the video?

[@ravulapallivenkatagurnadha9605]

Please add more Videos on Data Engineering

[@aibutsimple]

Need more etl tools using project

[@madhuppandey3151]

Hi krish, How many videos you will upload on this series ?

[@THEINDIANSAVIOR]

Amazing sir 👏👏

[@sachinkumar8369]

hey krish! thank you so much for this helping video. Can we use this build this using aws free tier account?

[@Stocks-l4x]

Running Dockers and airflow in local windows? It needs Linux environment right?

[@tajudeensanni9240]

Krish! This is amazing!!! Please is there any way Apache Airflow can be connected with k8

[@maxwellmaxwels9618]

Top content, Classic!

[@tejasgandhi7045]

Thank you for such a nice video. It was really informative. I want to understand from the multiple pods which are coming up, if I want to run the worker node in AKS then what all chants are required and the process ?

[@istiyakahamad2114]

Great work..

[@baophan259]

Can the pipeline still run when i turn off my laptop ?

[@aliensamv3997]

hey Krish please add production-level project in your paid course (399) ... there were only basic project

[@saifilicious1749]

Thanks bhai you listen to me

[@Beliver5894-l8h]

Hello Krish, create a end to end data science project which should involve ETL steps as well with airflow and also deployment

[@MOHIT836]

Krish sir, is this available on MLOps course ??

[@krishnaik06]

yes many projects have been uploaded there :)

[@satyamsubhaprakash]

Hi Sir, can you please add projects on Market mix modelling?

[@ChandanNayak-lq1pd]

Great ContentSir,will it be the part of data engineering course,and when is it available?

[@krishnaik06]

yes a;ready started big data engineering course recording

[@sumitrukadikar3521]

​@@krishnaik06sir when it will launch?

[@ambivert_sigma_motivation]

Thanks for this course! Can you create a Jarvis using Python+AI?

[@aliensamv3997]

hey krish please add production level project in your paid course (399) ...

[@Sankarm018]

Sir pls provide full DATA science course from beginner to advance level on udemy sir

[@hemanthvarma1939]

Your Voice Was Breaking in the Video due to a Connectivity issue in your end...Does this happen in any-part of your Udemy Video's too...Planning to Buy the Data Science Course on Udemy

[@krishnaik06]

U can get it all the videos are recorded properly

[@hemanthvarma1939]

@@krishnaik06 Thanks a Lot Krish Sir

[@RakeshPrajapati-b4z]

Sir machin learning ka paid cours kaise join kare kaise sir please guide me

[@alhasanmohammedbinquraish960]

Hindi video sir airflow 🙏🙏🙏

[@aliensamv3997]

hey krish please add production level project in your paid course (399) ... how we are suppose to know working process at industry lvl ... just 1 project please

[@siddharthtyagi1254]

sir please upload some more tools in udemy course

[@krishnaik06]

yes brother kubernetes will also get added

[@siddharthtyagi1254]

@@krishnaik06 sir i want to learn feature store also from you . please sir

[@mohammadfurquan241]

📌ONE VIDEO ON KAFKA ALSOOOOO

[@njagikevin1069]

Env file ".env" found. Loading... [+] Running 3/4 - Container 20241028-etl-weather_43a1dc-postgres-1 Starting 1.2s - Container 20241028-etl-weather_43a1dc-triggerer-1 Recreated 0.4s - Container 20241028-etl-weather_43a1dc-scheduler-1 Recreated 0.5s - Container 20241028-etl-weather_43a1dc-webserver-1 Recreated 0.4s Error: error building, (re)creating or starting project containers: Error response from daemon: Ports are not available: exposing port TCP 127.0.0.1:5432 -> 0.0.0.0:0: listen tcp 127.0.0.1:5432: bind: An attempt was made to access a socket in a way forbidden by its access permissions. i am getting this error when running astro dev start

[@blazemates]

go to services and turn off you postgress service. or see if any other image is using that port

[@prathmeshatre6251]

S missing in using word

[@AlHasan-sh3il]

sir hindi video please

[@deepop1287]

Hello sir , Actually i want to buy your Udemy course of DATA ANALYST , But the price is currently high , so can you please provide me a Udemy coupon for this course then it will be really helpful sir 🥺🥺

[@KNfarming882]

First comment