yigang chen's Discussions

Legacy code running in mapper
3 Replies

Hi, I'm trying to move my legacy data procfessing code to hadoop. My issue is the legacy code relies on local file system - it both reads and writes meta data. When the code access local data it ty…

Tagged: cache, distributed

Started this discussion. Last reply by Jason Venner Mar 10.

View All

yigang chen's Page

Gifts Received

yigang chen has not received any gifts yet

Give yigang chen a Gift

Latest Activity

Jason Venner replied to yigang chen's discussion 'Legacy code running in mapper'

You can build your own symbolic link by running a command from java, you just need to verify where the data is unpacked, and then build a link to it. A quick search turned up the following page for sample java code for you: http://www.giannistsakir…

March 9

yigang chen replied to yigang chen's discussion 'Legacy code running in mapper'

So that means I'd need to modify the legacy code, i.e., change the hard coded: "a/relative/path/to/my/file.xml" to: "./mymeta.zip/a/relative/path/to/my/file.xml" Is there a way at all to NOT change the legacy code?

March 8

Jason Venner replied to yigang chen's discussion 'Legacy code running in mapper'

If you pass -archives mymeta.zip there will be a symbolic link in the current working directory for the map or reduce task mymeta.zip, which points to the directory that the archive was unpacked in. so if you use ./mymeta.zip/path_in_archive/file.xm…

March 7

yigang chen added a discussion

Legacy code running in mapper

March 7

yigang chen is now a member of Hadoop Professionals

Welcome Them!

March 7