Is it possible to a make switch into big data without prior/relevant exp. ?
@nehabansal6775 жыл бұрын
Finally got the concepts cleared
@DataSavvy5 жыл бұрын
Thanks Neha
@DataSavvy5 жыл бұрын
Thanks
@pandurangbhadange257 ай бұрын
repartition: 1. is used to increase or decrease the RDD/DataFrame partitions 2. More shuffle Coalesce : 2. Reduce the partition 2. No shuffle 3. Less expensive
@mohans31435 жыл бұрын
Well explained but it would be explained by using some use cases.. We can get definitions in google. Now a days it is needed to explain everything in practical.
@aparnashrivastava48824 жыл бұрын
in which use case repartition and coalesce be used?
@DataSavvy4 жыл бұрын
Repartition calls full shuffle to create equal size partitions... Coalesce tries to combine existing partitions and reduce no of partitions... Coalesce is used for decreasing no of partitions... Repartition can be used to decrease or increase partitions
@Kassadhy5 жыл бұрын
Well explained!!!!!
@gauravpathak70175 жыл бұрын
Harjeet-On what basis this partition happens?
@surenderraja13044 жыл бұрын
Does Coalesce() happen in map side or reduce side ? Does repartition() happen in map side or reduce side ().
@Nikita-fy7js3 жыл бұрын
there is no map reduce in spark....everything happens in memory so there is no concept of map reduce here
@vkd94424 жыл бұрын
Dude.. Audio is too low.. Can u pls rectify it
@DataSavvy4 жыл бұрын
I tried changing it... Somehow KZbin is not allowing to do so... This is improved in New videos
@ampolusantosh53506 жыл бұрын
how can w know one partiton has high data,one partition has low data
@DataSavvy6 жыл бұрын
Following will give you a new RDD which will help u get size of each partition in terms of records rdd.mapPartitions(iter => Array(iter.size).iterator, true)
@ampolusantosh53506 жыл бұрын
expalin diff between linage vs DAG
@DataSavvy6 жыл бұрын
Here is your video my friend... kzbin.info/www/bejne/hHiydWqAg5uUsK8
@ampolusantosh53506 жыл бұрын
in wide transfermation also we can give no.of partition.so what is diff groupByKey(8) vs repartition(8)