A Community for Hadoop Users

Output file contains output of mapper, not reducer

Hi Everyone,

I have just started working wit hadoop a few week ago, and am in the process of writing my first map reduce program after completing some tutorials online. I am running on a psuedo-distributed network with one node. Everything seems to be running ok, but for some reason, the output file of my program contains the output from the mapper instead of the reducer. I am trying to use the new API 0.20.2 without deprecated classes. Also, I have not set numReducers to 0, which I have read could cause this.

Any ideas?

Thank,

-Stephen

Attachments:

▶ Reply to This

Replies to This Discussion

Permalink Reply by Jason Venner on July 1, 2010 at 7:18am

The default reducer is the Identity Reducer, which simply orders the output of your Map phase.
Is it possible you have a miss configuration and the Reducer class you have specified is not being engaged.

You could put some log messages into your reduce class methods to verify that the class is being used.

▶ Reply

Permalink Reply by Stephen Giguere on July 2, 2010 at 12:20pm

Thanks for your response. For the time being, I have opted to just utilize the deprecated but much better documented classes. Everything is working fine. I suspect that you are right about the configuration and may revisit this later. Thanks again,

-Stephen