A Community for Hadoop Users

How to disable sort in hadoop

Dear all, If in my application, I do not need the hadoop to sort the intermediate result for me. How can I disable the sort in the application? Because sorting needs time. But actually, I don`t want it to be sorted. Thanks!

▶ Reply to This

Replies to This Discussion

Permalink Reply by Jason Venner on October 8, 2009 at 12:11pm: If you set the number of reduce tasks to 0, there will be no sorting. There will also be no reduce phase.

In hadoop through 19, the JobConf object provides a method setNumReduceTasks, and the parameter behind it is mapred.reduce.tasks.
I do not know the hadoop 20 equivalents.; ▶ Reply to This

Permalink Reply by wang zhengkui on October 8, 2009 at 7:48pm: Thanks Jason. Does this mean, if my numReduceTask doesn`t equal to 0, hadoop must sort the intermediate result? If my reduce number doesn`t equal to 0, is there anyway I do not let it sort my intermediate result?; ▶ Reply to This

Permalink Reply by Jason Venner on October 8, 2009 at 11:45pm: If your number of reduce tasks is not 0, the hadoop framework will sort your results. there is no way around it.; ▶ Reply to This