YARN - Capacity Scheduler

[@s.baskaravishnu22]

your videos are excellent. Your videos are very much useful to me, Many thanks for that. My warm regards to you.

[@MonuChoudhary-dm4ww]

nice explanation with demo

[@dwhlab5436]

Very nicely done

[@yogeshpawar6068]

Great session sir, i got to know how to configure in gui & cli also.

[@haseebuddinkhan7144]

Thanks sir. Plz take the theory part of capacity scheduler. Sir

[@mdshahalam3010]

Great content!! Thank you sir.

[@HadoopEngineering]

Please subscribe my channel and watch my other play lists as well

[@nareshreddy4563]

I was very confused with the schedulers before seeing this video.i am very clear now.

[@HadoopEngineering]

@Naresh , thanks for your comments.

[@aravindareddy9763]

Good demo. Thanks for posting it.

[@HadoopEngineering]

Thanks for your comments , Please subscribe this channel to get more updates

[@akashkatyarmal1287]

Awesome work bro

[@HadoopEngineering]

Thanks..Please subscribe to this channel

[@rajanthangaswamy4678]

Kindly explain capacity scheduler features.

[@kalyanp1062]

Hi Shijeesh, its really great content what you have posted. We have been following your videos. Appreciate it vmuch. I have couple of Q's : What happens to Running Hive Jobs if it gets into Stale coniguration because of parameter changes? Application container will be common for all Jobs? ( I mean, each job will have a different application container or even for 100 jobs it will be having a one common application master?) When it is Capacity scheduler where is the option to set FAIR as internal scheduler which is non-default and new* feature? Thanks for making Hadoop admin easy for Us... :-)

[@HadoopEngineering]

Thanks Kalyan , If you see the stale configuration it does not effect any of the running jobs .In order to reflect that change you made on configuration the restart of the service is highly recommended. Application master is unique for each job . Answer to your third question - capacity scheduler in back end uses FIFO scheduling , but recent versions have fair share scheduling supported .I dont remember exact configuration in Cloudera distribution.

[@kalyanp1062]

Hearty thanks for your reply Shijeesh. Just curious on 1st Q'n - Lets say a hive query is in progress and we did restart hive server because of stale configuration because of a dependent component config change. Does the running hive query will not be impacted? or CM waits till all existing query are completed then only it proceeds with service bounce?

[@BuddhaCafé-b7k]

Hey, It was a nice video, I got the concept and how to configure it on CDH and on CLI also. Thanks

[@morr8842]

can I apply activities like that with 8GB ram total in my lap

[@HadoopEngineering]

with 8 GRAM it is bit difficult , at least 16G RAM is better ..

[@rameshpokhrel6475]

Hello sir, can we change the value of resource percentage dynamically using "yarn rmadmin" command ? if i want to automate those values within certain time interval, how is it possible to do ?

[@HadoopEngineering]

Queue configuration is normally one time task and changing the configuration is very rare.These queue configuration changes requires changes in scheduler configuration xml files normally done by using Ambari/Cloudera Manager

[@mubashirchamanshaikh419]

hello sir. what is main difference between capacity scheduler and fair scheduler

[@HadoopEngineering]

In capacity scheduler we create queues and allocate resources .Each queue can be allocated to some percentage of resource .Within a queue capacity scheduler uses fifo scheduling for scheduling jobs submitted to that queue.I will be uploading fair scheduling demo video soon

[@chetanbonde2256]

Good presentation, You have applied max capacity kzbin.info/www/bejne/mme9XmCZfrSZq9k to Engineering is that applicable for finance also if we submit the jobs in finance and engineering queue is empty?