Hadoop Tutorial - Map Reduce Examples - Part 2

[@ScholarNest]

Want to learn more Big Data Technology courses. You can get lifetime access to our courses on the Udemy platform. Visit the below link for Discounts and Coupon Code. www.learningjournal.guru/courses/

[@navinnayak007]

thank sir...i'm always wait 4 your tutorial ...coz you have great concept..

[@sonal2011]

Very informative. Thanks for making this series. Is it possible for you to also make hadoop with Python video series?

[@manishnit4u]

How to download ppts and notes for this course?

[@petrinac.h9484]

Great video! One question, in the example, the map function collects all valid & invalid meta info for these videos. Take the "valid" record as an example, its value contains almost all the original records which are extremely big. Can we read all of them from an Iterable into memory???

[@nish21m]

Great Video, Just had a doubt, with the example of "Valid" and "Invalid" value. Even if we don't set the numberofReducer(2) explicitly, there should be 2 Reducer create by default as we have 2 KEYS. Will this not create two different files?

[@ScholarNest]

By default, there is only one reducer, and it can take more than one Keys. However, test it and let me know.

[@nish21m]

ohh... so when we say that after shuffle/sort each key is processed by the reducer, by default single reducer would do the aggregation recursivly? il try but just wanted to clear the concept

[@nish21m]

ohh... so when we say that after shuffle/sort each key is processed by the reducer, by default single reducer would do the aggregation recursivly? il try but just wanted to clear the concept

[@ScholarNest]

Yes. One key means one reducer record. If there are more than one keys(reducer records), the framework will call the reducer more than once and pass all the data for one key in each iteration. When we create more than one reducers, the partitioner comes into play and divide the work among those reducers.

[@thevijayraj34]

I'm using 2.7.3, It creates 2 output files but writes the output only to the second file (00001) and first one (00000) remains empty, Why? I also tried MultipleTextOutputFormat type in driver class, It only stops creating the empty file.