Quick and clear. With your Udemy courses and your KZbin videos, I'm learning a lot about Apache Airflow. From Zero to hero in a few weeks. Thx.
@waodezalmawati4602 жыл бұрын
Whats a clearly explanation! Thankyou so much! Love from Indonesia!❤🔥
@luuquanghuy81932 жыл бұрын
Thank u for your sharing, keep it up because u doing that right clearly and easy to understand. Love from Viet Nam
@guolunli19082 жыл бұрын
Thank you so much! By following your videos closely I not only avoided bugs but also learned in detail how to set up an airflow pipeline!
@quanminfeng8375 Жыл бұрын
Best xcom video I watched
@MarcLamberti Жыл бұрын
Thank you 🙏
@ХеллБой-я1щ Жыл бұрын
thank you! as it gets very often, some detail is missed by me in order to use new functionality and that detail was ti thanks for explaining!
@anatolys9203 Жыл бұрын
great tutorial and explanantion! thank you!
@pyrojoke23 жыл бұрын
Thank you! I spent a few hours trying to figure out how it works, but your tutorial did it in 10 minutes
@victortelles96702 жыл бұрын
Amazing tutorial as always. Love yout channel.
@shantanubugadi31882 жыл бұрын
Very well explained...! Very awesome...! I am just enjoying the series
@GustavoCosta-jr1mh2 жыл бұрын
Thanks for all your videos, Marc. Brilliant content, very straightfoward!
@yunusemrahulucay49532 жыл бұрын
Thank you so much. It was really quick and clear.
@guyfridman44263 жыл бұрын
Thank you marc, I did the course in udemy and it's very recommended
@MarcLamberti3 жыл бұрын
Thank you!
@saritkumarsi41663 жыл бұрын
Thanks Marc. Great info for a beginner like me :)
@MarcLamberti3 жыл бұрын
Great pleasure, let me know what you would like to see next :)
@saritkumarsi41663 жыл бұрын
@@MarcLamberti It would be awesome if you can share some knowledge by taking this tutorial further by adding (if possible) a new service and assigning it a task via DAGs within the Airflow container. Just to give you some background on what I am currently trying to do --> Similar to your compose file, I have Postgres, Airflow Webserver and scheduler services defined in docker compose file and running perfectly. However, I have an extra service (say an Ubuntu server) defined in the same compose file which basically takes an .sh command and runs it. I am trying to send an sh command via DAG DockerOperator, to this 4th container. I took help from your video on DockerOperator as well :). If I understood correctly, the Airflow DAG in that video is running on the host instead from within a docker container. I searched for help on the internet, but haven't come across any. Do let me know if such use case is possible with airflow. Appreciate you help :)
@Jmignet3 жыл бұрын
Pretty enlightening! Thank you very much!
@sodiqafolayan49213 жыл бұрын
Thanks Marc as always
@MarcLamberti3 жыл бұрын
My pleasure!
@dmdv-h1t3 жыл бұрын
thank you for clear explanation
@kyuukev Жыл бұрын
Nice video but is there a way to do it between 2 DockerOperator? And if not with the DockerOperator creating some kind of CustomDockerOperator to make it able to do it?
@АндрейРезниченко-п9ъ Жыл бұрын
Hi all! Please suggest how to implement the following scenario: Have a dag with sshoperator that executes shell script. Shell script returns different return codes. How to analyze which return code is received??
@kipodiha3 жыл бұрын
Very useful video! after reading airflow documentation and example for me this XCOM was still not clear :( Now I m ok! Thanks a lot!!!
@Jamdat332 жыл бұрын
Excellent thank you
@padegalsaigiriraj34593 жыл бұрын
Thanks, Marc. Great info Can You Please Confirm This I have Two Tasks in DAG1 Task1 - Normal Function Which does some process Task2 - TriggerDagRunOperator (Trigger DAG2) Can I Use the return value/XCom of task1 in Task2 and use this value in DAG2? Thanks
@drakdragon2 жыл бұрын
Sir, I tried to do the needful to do good and nice Airflow in the XCom, but my hit chance was still saying 99% chance of to becoming the hit sir, but it still missed most of the time. Sir I do believe you should do the needful by improving the Airflow by using the Minecraft Mushroom Soup Windtunnel.
@prashantjoshi46913 жыл бұрын
add provide_context = True in task which uses ti in older version of airflow
@bigkongenergy60543 жыл бұрын
Commander, the aliens continue to make progress on the Avatar Project. If we're going to slow them, we need to move fast.
@marcuscheng94133 жыл бұрын
ok. we're raiding the blacksite.
@ramjan1352 жыл бұрын
Hi Marc, how to cross communication data (data sharing) between master_dag and child_dag(this is triggered by master dag)?
@wll90852 жыл бұрын
useful!
@thati27922 жыл бұрын
can we pass xcom as params in redshiftsqloperator?
@BULLSHXTYT2 жыл бұрын
How to access xcom in subdag from parent dag?
@djalan845 ай бұрын
what about sharing data between dags?
@MarcLamberti5 ай бұрын
You can. Just specify the dag_id in the xcom_pull method.
@antoniosaldivar41632 жыл бұрын
I have the same xcom key from same task and it does not get replace or overwrite, instead it creates other xcom record with same key and task id
@MarcLamberti2 жыл бұрын
look at the execution date which should not be the same. That's why you see that. XCOMs are created based on their DAGRun too
@antoniosaldivar41632 жыл бұрын
@@MarcLamberti thank you for the info!!, you are right the execution dates (YYYY-mm-dd, HH:mm:ss) are different, I see it has to be executed at the same time to overwrite that XCOM.
@RaviPrakash-vu6hf Жыл бұрын
Hi @@MarcLamberti I have the same issue. I do want to overwrite the value from the last run. I have a DAG that will run every 15 mins and it will push a key to XCOM. Since this key will be stored in the DB, the number of records will keep on growing. Hence, I want to overwrite the value of the key from the previous DAG run. please let me know if that is possible. Thanks!!
@yerkhankabyl62842 жыл бұрын
Does someone have the same issue like me? I got 3 “None” -> [None, None, None]
@yerkhankabyl62842 жыл бұрын
Oops, I solved this problem, the main error was the name of “Xcom” 😅 Thank u, man, u are awesome (:
@BM-uf4pp3 жыл бұрын
The three people who disliked this video are nifi fans
@marcuscheng94133 жыл бұрын
or ADVENT troopers.
@g3rzin Жыл бұрын
Please don't read the underscore!!!! Just read it a "choose model" and not "choose underscore model"
@MarcLamberti Жыл бұрын
Okkkkkkkkkk!!!!!!!! Will do
@g3rzin Жыл бұрын
@@MarcLamberti ahahah great! And thanks for teaching me airflow!