Apache Airflow One Shot- Building End To End ETL Pipeline Using AirFlow And Astro

  Рет қаралды 37,326

Krish Naik

Krish Naik

Күн бұрын

Пікірлер: 58
@krishnaik06
@krishnaik06 2 ай бұрын
Check out my new Udemy complete MLOps Course with 10+ end to end ML Projects: www.udemy.com/course/complete-mlops-bootcamp-with-10-end-to-end-ml-projects/?couponCode=SHARING
@adityapatel907
@adityapatel907 2 ай бұрын
sir, we need new udemy coupon for ' DSA bootcamp + Leetcode Exercise ' . please sir ! do it quickly.🙏
@tanmaybagul2957
@tanmaybagul2957 2 ай бұрын
Realizing the Power of Airflow 😎 this video help me have high level understanding of what i was doing from last 3 years
@oceanview3165
@oceanview3165 28 күн бұрын
It's very straightforward and helpful! Thank you!
@SandipJadhav-ls1kg
@SandipJadhav-ls1kg 2 ай бұрын
when will be big data engineer cource we can expect krish sir
@gagansingh3481
@gagansingh3481 2 ай бұрын
It's Etl , etl can be done though power query and load it to postgre sql .
@gagansingh3481
@gagansingh3481 2 ай бұрын
It's Etl , etl can be done though power query and load it to postgre sql .
@jaggyjut
@jaggyjut 2 ай бұрын
Awesome. How is airflow different in comparison to Microsoft fabric
@dineshbiswal2775
@dineshbiswal2775 2 ай бұрын
What tools are you using for the video?
@bp2807
@bp2807 2 ай бұрын
Krish Sir you really bring very good course, please include kubernetes. Also which tablet and software you use to write and explain sir. Love the way you teach. Kindly answer🙏
@madhuppandey3151
@madhuppandey3151 2 ай бұрын
Hi krish, How many videos you will upload on this series ?
@tusharmodi8344
@tusharmodi8344 2 ай бұрын
Sir for your mlops course do we also need to have aws subscription to do project work or is it not needed we can do something in free tiwr
@krishnaik06
@krishnaik06 2 ай бұрын
U can do in aws free tier
@harshsinha12
@harshsinha12 2 ай бұрын
​@@krishnaik06sir the AWS Environment set-up is giving some errors, i have tried various fixes
@ARkhan-xw8ud
@ARkhan-xw8ud 2 ай бұрын
@@harshsinha12 i have taken of my friend
@maxwellmaxwels9618
@maxwellmaxwels9618 29 күн бұрын
Top content, Classic!
@Stocks-l4x
@Stocks-l4x Ай бұрын
Running Dockers and airflow in local windows? It needs Linux environment right?
@tajudeensanni9240
@tajudeensanni9240 2 ай бұрын
Krish! This is amazing!!! Please is there any way Apache Airflow can be connected with k8
@BechirMathlouthi-n9h
@BechirMathlouthi-n9h 25 күн бұрын
very insightful !! thanks for the effort !
@tejasgandhi7045
@tejasgandhi7045 Ай бұрын
Thank you for such a nice video. It was really informative. I want to understand from the multiple pods which are coming up, if I want to run the worker node in AKS then what all chants are required and the process ?
@sachinkumar8369
@sachinkumar8369 2 ай бұрын
hey krish! thank you so much for this helping video. Can we use this build this using aws free tier account?
@baophan259
@baophan259 9 күн бұрын
Can the pipeline still run when i turn off my laptop ?
@mohitjoshi8984
@mohitjoshi8984 2 ай бұрын
Sir talk about Kubernetes please
@satyamsubhaprakash
@satyamsubhaprakash 2 ай бұрын
Hi Sir, can you please add projects on Market mix modelling?
@ambivert_sigma_motivation
@ambivert_sigma_motivation Ай бұрын
Thanks for this course! Can you create a Jarvis using Python+AI?
@THEINDIANSAVIOR
@THEINDIANSAVIOR 2 ай бұрын
Amazing sir 👏👏
@aliensamv3997
@aliensamv3997 2 ай бұрын
hey Krish please add production-level project in your paid course (399) ... there were only basic project
@ravulapallivenkatagurnadha9605
@ravulapallivenkatagurnadha9605 2 ай бұрын
Please add more Videos on Data Engineering
@bralabala
@bralabala 2 ай бұрын
what I would have preferred to see is how you configured it to work with local files rather than calling an external api. since I posted this comment, I have now solved the problem myself. the short of it is to use astro's docker-compose.overrride.yml file to mount your local drives so it can communicate with the local directories
@hemanthvarma1939
@hemanthvarma1939 Ай бұрын
Your Voice Was Breaking in the Video due to a Connectivity issue in your end...Does this happen in any-part of your Udemy Video's too...Planning to Buy the Data Science Course on Udemy
@krishnaik06
@krishnaik06 Ай бұрын
U can get it all the videos are recorded properly
@hemanthvarma1939
@hemanthvarma1939 Ай бұрын
@@krishnaik06 Thanks a Lot Krish Sir
@istiyakahamad2114
@istiyakahamad2114 2 ай бұрын
Great work..
@aibutsimple
@aibutsimple 2 ай бұрын
Need more etl tools using project
@MOHIT836
@MOHIT836 2 ай бұрын
Krish sir, is this available on MLOps course ??
@krishnaik06
@krishnaik06 2 ай бұрын
yes many projects have been uploaded there :)
@saifilicious1749
@saifilicious1749 2 ай бұрын
Thanks bhai you listen to me
@RakeshPrajapati-b4z
@RakeshPrajapati-b4z 2 ай бұрын
Sir machin learning ka paid cours kaise join kare kaise sir please guide me
@Beliver5894-l8h
@Beliver5894-l8h Ай бұрын
Hello Krish, create a end to end data science project which should involve ETL steps as well with airflow and also deployment
@ChandanNayak-lq1pd
@ChandanNayak-lq1pd 2 ай бұрын
Great ContentSir,will it be the part of data engineering course,and when is it available?
@krishnaik06
@krishnaik06 2 ай бұрын
yes a;ready started big data engineering course recording
@sumitrukadikar3521
@sumitrukadikar3521 2 ай бұрын
​@@krishnaik06sir when it will launch?
@aliensamv3997
@aliensamv3997 2 ай бұрын
hey krish please add production level project in your paid course (399) ...
@aliensamv3997
@aliensamv3997 2 ай бұрын
hey krish please add production level project in your paid course (399) ... how we are suppose to know working process at industry lvl ... just 1 project please
@Sankarm018
@Sankarm018 2 ай бұрын
Sir pls provide full DATA science course from beginner to advance level on udemy sir
@alhasanmohammedbinquraish960
@alhasanmohammedbinquraish960 2 ай бұрын
Hindi video sir airflow 🙏🙏🙏
@mohammadfurquan241
@mohammadfurquan241 Ай бұрын
📌ONE VIDEO ON KAFKA ALSOOOOO
@prathmeshatre6251
@prathmeshatre6251 2 ай бұрын
S missing in using word
@siddharthtyagi1254
@siddharthtyagi1254 2 ай бұрын
sir please upload some more tools in udemy course
@krishnaik06
@krishnaik06 2 ай бұрын
yes brother kubernetes will also get added
@siddharthtyagi1254
@siddharthtyagi1254 2 ай бұрын
@@krishnaik06 sir i want to learn feature store also from you . please sir
@gillfromamritsar
@gillfromamritsar 22 күн бұрын
I got the point it's base one but I have some senerios to thinnk about -> What is Retry mechanism if any of the step fails -> If we have a senerio where a large data should be loaded then from one DB to another then how to resume from a point we left of, for example there are 50k records to be loaded from one loc to another then and process broke at 25k then it should resume when next time it runs instead of running 50k from point 0
@njagikevin1069
@njagikevin1069 Ай бұрын
Env file ".env" found. Loading... [+] Running 3/4 - Container 20241028-etl-weather_43a1dc-postgres-1 Starting 1.2s - Container 20241028-etl-weather_43a1dc-triggerer-1 Recreated 0.4s - Container 20241028-etl-weather_43a1dc-scheduler-1 Recreated 0.5s - Container 20241028-etl-weather_43a1dc-webserver-1 Recreated 0.4s Error: error building, (re)creating or starting project containers: Error response from daemon: Ports are not available: exposing port TCP 127.0.0.1:5432 -> 0.0.0.0:0: listen tcp 127.0.0.1:5432: bind: An attempt was made to access a socket in a way forbidden by its access permissions. i am getting this error when running astro dev start
@blazemates
@blazemates 23 күн бұрын
go to services and turn off you postgress service. or see if any other image is using that port
@AlHasan-sh3il
@AlHasan-sh3il Күн бұрын
Sir , please hindi video 😭🙏
@deepop1287
@deepop1287 2 ай бұрын
Hello sir , Actually i want to buy your Udemy course of DATA ANALYST , But the price is currently high , so can you please provide me a Udemy coupon for this course then it will be really helpful sir 🥺🥺
@KNfarming882
@KNfarming882 2 ай бұрын
First comment
@AlHasan-sh3il
@AlHasan-sh3il Ай бұрын
sir hindi video please
What is Agentic AI? Important For GEN AI In 2025
22:36
Krish Naik
Рет қаралды 3,1 М.
“Don’t stop the chances.”
00:44
ISSEI / いっせい
Рет қаралды 62 МЛН
VIP ACCESS
00:47
Natan por Aí
Рет қаралды 30 МЛН
MLOPS Github Action With CICD Pipeline One Shot Tutorial
1:08:33
Krish Naik
Рет қаралды 27 М.
Event-Driven Architecture (EDA) vs Request/Response (RR)
12:00
Confluent
Рет қаралды 177 М.
Anthropic MCP + Ollama. No Claude Needed? Check it out!
18:06
What The Func? w/ Ed Zynda
Рет қаралды 10 М.
Complete Dockers For Data Science Tutorial In One Shot
1:19:29
Krish Naik
Рет қаралды 133 М.
Apache Kafka in 15 minutes
15:33
Gaurav Sen
Рет қаралды 49 М.
Master Apache Airflow: 5 Real-World Projects to Get You Started
13:18
Darshil Parmar
Рет қаралды 12 М.
Airflow DAG: Coding your first DAG for Beginners
20:31
Data with Marc
Рет қаралды 236 М.
Airflow for Beginners: Build Amazon books ETL Job in 10 mins
13:13
Sunjana in Data
Рет қаралды 21 М.