Hadoop Professionals

A Community for Hadoop Users

There are two requirements which I want to implement based on Hadoop. But , by now, I do not think that hadoop support them now. I am looking forward to your suggestion how to implement these.

Firstly, if I want to let the reducers to fetch more partitions files from map out put, is that ok? For instance, now reducer one can fetch all the partition 1 from mappers, how I implement that reducer one can fetch all the partition 1 and also 2 to go to reducer 1? If can , How could I implement that?

Secondly, in the map phase, one recorder can only be written into one partition file according to the partitioner function. If I want to write one record to multi-partition files, how can I do that? For example, there are M reducers and there should be M partition files in map phase. Now one recorder can only be output to one of M partition files. If I want to output one record to multi-partition files, is there any way to do this?

Looking forward to your new idea about this.

Comment

You need to be a member of Hadoop Professionals to add comments!

Join Hadoop Professionals

Groups

© 2010   Created by Jason Venner.   Powered by .

Badges  |  Report an Issue  |  Terms of Service

Sign in to chat!