Пікірлер
@apoorvkansal9266 6 күн бұрын
You are doing a great job and please continue creating such useful and informative videos. I am preparing for Sr/Lead Data Engineer interviews and highly appreciate your hard work here. 🙏🙏
@kakaji_cloud8106 2 ай бұрын
Do we need to pay for Composer , even if any process is not running in composer?
@kaysush 24 күн бұрын
Yes. The machines for running scheduler are always running.
@shefalishrivastava8950 2 ай бұрын
These videos are literally SAVING my life, thank you!
@manishpathak6262 4 ай бұрын
Very good work bro!
@khurramkhan5878 4 ай бұрын
hello sir i m creating composer environment but here is image version please assist me how to create
@presha8041 5 ай бұрын
Can you add the provider package here, i am trying to add mongo db but i am failing to add its provider package here
@sureshraina321 6 ай бұрын
Great series
@informationsatellite5155 9 ай бұрын
great.. need more such videos,,,thank you so much.
@dhirendramisra6671 9 ай бұрын
Hi Sushil - Does nodemanager need vCore on the the worker node?
@user-fb1ue1fr8n 9 ай бұрын
You are doing a great job. These videos are very helpful. I had one doubt in this video, how is Airflow able to access dataproc and create cluster and fire spark jobs to it. Don't we need to add some kind of permission in dataproc to allow airflow access dataproc and also some configurations in Airflow?
@sahillohiya7658 10 ай бұрын
you are so underrated
@surajhk31 10 ай бұрын
Unable what ?
@saleenapatel 10 ай бұрын
awesome video
@Amarjeet-fb3lk 11 ай бұрын
Nice video, But,how we can create dataproc cluster with gcs storage and sql metastore using this airflow operators? What if I need to read data from a gcs bucket and write to another bucket?
@amalkumar5201 11 ай бұрын
Sir i want to join your session gco
@ManojKaushik93 Жыл бұрын
"Transaction declined" problem with my Visa debit card. What to do?
@Kondaranjith3 Жыл бұрын
air flow basics needed sir
@karthikeyanr1171 Жыл бұрын
Please keep your mic at distant
@vickydashing196 Жыл бұрын
Please increase the quality of video
@swapnamohankumar6602 Жыл бұрын
@ Sushil Kumar, Thank you so much for this wonderful playlist on GCP... it will really help the beginners... Good explanation and end-to-end data pipeline flow is covered.
@pratikshirbhate8026 Жыл бұрын
Very well explained. Just one question, do we need additional compute instance or gke instance to execute compute intensive functions like training a machine learning model, or processing a data from GCS bucket?
@ten2the6 Жыл бұрын
New to GCP. I found your tutorial very easy to understand and it helped me become productive very quickly! Thank you!
@BriteRoy Жыл бұрын
Question : what wil be the use-case of such a scenario where we are separating computing and storage and later deleting the dataproc cluster. The example you showed here is just for a one time job. What about a daliy batch processing where files are coming to GCS on a daily basis and pyspark job is required to compute. How will be the cluster availablility be managed in such a case ?
@user-qj3uo6ts3v Жыл бұрын
Hi shushil, getting this error while creating shared dataproc cluster. There was no instance found at projects/projectid/instances/hive-mysql or you are not authorized to access it. I have enabled the cloudsql admin api
@rutujapabale5080 Жыл бұрын
Same pinch
@rutujapabale5080 Жыл бұрын
could you help me if you could solve it?
@ExploreWithArghya Жыл бұрын
Sushil.. I'm really messed up with airflow cost.. Please tell me how configure airflow_monitoring - schedule interval ?
@ExploreWithArghya Жыл бұрын
Hi sushil, I am following you videos of GCP . I have two doubts. First how do I set the job_id of dataproc jobs via airflow and second which is very important - how do I add 'additional python files' in dataproc job via airflow?
@SunnyG9 Жыл бұрын
it errors out - Service Hive server2 is not enabled ... localhost port 3306 (tcp) failed: Connection refused ... dataproc cluster is created but not able to connect to Hive metastore ...
@RaviKumar-uu4ro Жыл бұрын
Hardly trying for a different accent , but no luck. Output is cumbersome.
@soumyasourabh8799 Жыл бұрын
Huge props Sushil. The way you simplified the concepts, things just got slotted into my head. Great work, really appreciate it!
@zg7860 Жыл бұрын
thank you !
@ATPuKc Жыл бұрын
Great video! Thank you very much
@prachiagarwal9457 Жыл бұрын
How can I mark the Airflow task as failed when the spark job fails? The operator used to submit the spark job is DataprocSubmitJobOperator
@pushkar17jan Жыл бұрын
It's a great start of the course.. looking forward for new videos and examples with cloud run
@duskbbd Жыл бұрын
Make video on cloud run vs cloud scheduler vs cloud composer vs work flow templet scheduler
@sanketkale6384 Жыл бұрын
Good Job. Thanks
@duskbbd Жыл бұрын
Better you should show how to see spark ui dag after job submission with help of dataproc and writing logs to gcs buckets @sushilkumar
@dileepreddy7469 Жыл бұрын
Great content and very informative. Keep up the good work. Cheers☺️
@luoyufan7771 Жыл бұрын
very useful!
@chetanhirapara7082 Жыл бұрын
Thank you so much Sushil kumar. You are doing awesome job for learner. It would be great and complete playlist if you will add installation of airflow on GCP
@chetanhirapara7082 Жыл бұрын
Thanks Sushil for this wonderful playlist. You're doing awesome work for GCP community.
@DileepKumar-ko6cy Жыл бұрын
Very Good Content ,Could you create more videos on GCP .Thank you so much for putting your efforts 👍
@ankitsingh-ng5cn Жыл бұрын
bhai apka youtube channel hai..aj he dekha ..really looking forward to learn from you..ANKIT here
@harshalpatel3012 Жыл бұрын
Nice tip, Thanks
@shivakumaryadav8997 Жыл бұрын
Hi Sishil Great video. Can you help or make a video on how to Create a basic Airlfow DAG to move a csv file from one location to another.
@Kogulansendhan Жыл бұрын
How to acknowledge the message post processing in Cloud function to avoid messages piled-up in Pub/Sub queue? Please suggest
@kaysush Жыл бұрын
Ideally if your function returns successfully it the message should be acknowledged.
@kaysush Жыл бұрын
Ideally if your function returns successfully it the message should be acknowledged.
@leleuf3 Жыл бұрын
What a video! Thanks Man! You’re the best!!! I catch a legacy project and I’m suffering to build the same environment to run my tests. I don’t know where to seek help. I’m newbie in GCP. I have no idea if I run the tests in my local computer or if I use a copy of the project in the cloud, but the second option I have no idea how to do.
@sudhirkenguva4095 Жыл бұрын
Hi Sushil, Thanks for this great content, Please make video of using git with cloud composer
@khurramkhan5878 4 ай бұрын
hello sir i m creating composer environment but here is image version please assist me how to create
@bageshgupta2332 Жыл бұрын
Great content and explanation
@binoypaul4030 Жыл бұрын
Awesome - please make more use cases on this topic
@DileepKumar-ko6cy Жыл бұрын
Great video and clear explanation ,Thank you so much 👍