A Community for Hadoop Users

How to disable sort in hadoop

Dear all, If in my application, I do not need the hadoop to sort the intermediate result for me. How can I disable the sort in the application? Because sorting needs time. But actually, I don`t want it to be sorted. Thanks!

▶ Reply to This

Replies to This Discussion

Permalink Reply by Jason Venner on October 8, 2009 at 12:11pm

If you set the number of reduce tasks to 0, there will be no sorting. There will also be no reduce phase.

In hadoop through 19, the JobConf object provides a method setNumReduceTasks, and the parameter behind it is mapred.reduce.tasks.
I do not know the hadoop 20 equivalents.

▶ Reply

Permalink Reply by wang zhengkui on October 8, 2009 at 7:48pm

Thanks Jason. Does this mean, if my numReduceTask doesn`t equal to 0, hadoop must sort the intermediate result? If my reduce number doesn`t equal to 0, is there anyway I do not let it sort my intermediate result?