Managed Airflow Instance with Google Cloud Composer | Product Spotlight | Google Cloud Platform

  Рет қаралды 27,660

Sushil Kumar

Sushil Kumar

2 жыл бұрын

In the first product spotlight video, we are covering Google Cloud Composer, a fully managed Airflow service.
Composer provides a highly available and scalable Airflow installation with awesome monitoring and logging out of the box.
Composer leverages lot of other Google Cloud Services such as Google Kubernetes Engine, Google App Engine, Google Cloud Storage, Cloud PubSub and more.
___________________________________________________________________
Cloud Composer Versions : cloud.google.com/composer/doc...
Sample DAG : gist.github.com/kaysush/85f56...
___________________________________________________________________
I regularly blog and post on my other social media channels as well, so do make sure to follow me there as well.
Medium ► lnkd.in/dPu9iKi
GitHub ► github.com/kaysush
LinkedIn ► lnkd.in/d5AtTna
KZbin ► lnkd.in/dt-XAWY
#GoogleCloudPlatform #GoogleComposer #Airflow

Пікірлер: 27
@dileepreddy7469
@dileepreddy7469 Жыл бұрын
Great content and very informative. Keep up the good work. Cheers☺️
@DileepKumar-ko6cy
@DileepKumar-ko6cy Жыл бұрын
Great video and clear explanation ,Thank you so much 👍
@amnihs394
@amnihs394 2 жыл бұрын
Good work, it is really helpful. Thanks a lot
@ATPuKc
@ATPuKc Жыл бұрын
Great video! Thank you very much
@udaypatel83
@udaypatel83 2 жыл бұрын
Good explanation! Thanks a lot
@JenilCalcuttawala
@JenilCalcuttawala 2 жыл бұрын
Great content & explanation.!
@binoypaul4030
@binoypaul4030 Жыл бұрын
Awesome - please make more use cases on this topic
@justdoittomorrow7119
@justdoittomorrow7119 2 жыл бұрын
Really insightful.Thanks Sushil
@dharmendra811
@dharmendra811 2 жыл бұрын
You are amazing man, please make more videos on composer and dataflow
@dineshkakade9979
@dineshkakade9979 2 жыл бұрын
great video to understand composer
@bageshgupta2332
@bageshgupta2332 Жыл бұрын
Great content and explanation
@febriyantabidin
@febriyantabidin 2 жыл бұрын
great explain about the product
@saleenapatel
@saleenapatel 10 ай бұрын
awesome video
@DileepKumar-ko6cy
@DileepKumar-ko6cy Жыл бұрын
Very Good Content ,Could you create more videos on GCP .Thank you so much for putting your efforts 👍
@anonymous-ze5fg
@anonymous-ze5fg 2 жыл бұрын
great content, please make more video on this GCP composer and GKE with airflow,
@khurramkhan5878
@khurramkhan5878 4 ай бұрын
hello sir i m creating composer environment but here is image version please assist me how to create
@leleuf3
@leleuf3 Жыл бұрын
What a video! Thanks Man! You’re the best!!! I catch a legacy project and I’m suffering to build the same environment to run my tests. I don’t know where to seek help. I’m newbie in GCP. I have no idea if I run the tests in my local computer or if I use a copy of the project in the cloud, but the second option I have no idea how to do.
@sudhirkenguva4095
@sudhirkenguva4095 Жыл бұрын
Hi Sushil, Thanks for this great content, Please make video of using git with cloud composer
@khurramkhan5878
@khurramkhan5878 4 ай бұрын
hello sir i m creating composer environment but here is image version please assist me how to create
@pratikshirbhate8026
@pratikshirbhate8026 Жыл бұрын
Very well explained. Just one question, do we need additional compute instance or gke instance to execute compute intensive functions like training a machine learning model, or processing a data from GCS bucket?
@shivakumaryadav8997
@shivakumaryadav8997 Жыл бұрын
Hi Sishil Great video. Can you help or make a video on how to Create a basic Airlfow DAG to move a csv file from one location to another.
@ExploreWithArghya
@ExploreWithArghya Жыл бұрын
Sushil.. I'm really messed up with airflow cost.. Please tell me how configure airflow_monitoring - schedule interval ?
@khurramkhan5878
@khurramkhan5878 4 ай бұрын
hello sir i m creating composer environment but here is image version please assist me how to create
@presha8041
@presha8041 5 ай бұрын
Can you add the provider package here, i am trying to add mongo db but i am failing to add its provider package here
@kakaji_cloud8106
@kakaji_cloud8106 2 ай бұрын
Do we need to pay for Composer , even if any process is not running in composer?
@kaysush
@kaysush 24 күн бұрын
Yes. The machines for running scheduler are always running.
@amalkumar5201
@amalkumar5201 11 ай бұрын
Sir i want to join your session gco
Create first Dag in Cloud Composer(Airflow) - Part 1
33:30
TechTrapture
Рет қаралды 16 М.
Cloud Composer - Orchestrating an ETL Pipeline Using Cloud Dataflow
24:07
Cloud & AI Analytics
Рет қаралды 7 М.
Best father #shorts by Secret Vlog
00:18
Secret Vlog
Рет қаралды 21 МЛН
THE POLICE TAKES ME! feat @PANDAGIRLOFFICIAL #shorts
00:31
PANDA BOI
Рет қаралды 25 МЛН
GCP Cloud Composer Introduction | Airflow managed service
21:13
Anjan GCP Data Engineering
Рет қаралды 19 М.
Cloud Composer: Good development practices
26:33
Google Cloud Tech
Рет қаралды 15 М.
Cloud Composer Architecture
11:32
TechTrapture
Рет қаралды 4 М.
Cloud Composer Best Practices
23:54
Google Cloud Composer
Рет қаралды 3,3 М.
Building stream processing pipelines with Dataflow
15:17
Google Cloud Tech
Рет қаралды 24 М.
GCP Composer | Airflow GCS to BigQuery and BigQuery Operators
19:24
Anjan GCP Data Engineering
Рет қаралды 12 М.