Hadoop Professionals - A Community for Hadoop Users

A Community for Hadoop Users

This network is a place to discuss and learn Hadoop, Solr, Katta, Map Reduce and a place to discuss hadoop resources such as hadoop books.

Members

View All

Latest Activity

Nils Drews, Tucker Roth and SANKET SHAH joined Hadoop Professionals

3 hours ago

Tucker Roth updated their profile photo

6 hours ago

Elton Tian added a discussion

Question: One instance of Hadoop can only run one Map Reduce Job at once?

I got a question as title. It just popped up in my head. And I think it's right.Unless, Hadoop instance can start multiple JobTrackers, and there's some resource manager coordinates JobTrackers from running into each other.Please correct me if I am…

yesterday

Elton Tian added a discussion

When should I jump on HBase rather than RDBMS?

Hello everyone,I read through some literature and end up with some ideas on HBase and RDBMS. Please correct me if I am wrong:* Use HBase if the application is going to handle large datasets, like Petabytes. That means when scalability is a big conce…

on Tuesday

Elton Tian, eric chastan and samir joined Hadoop Professionals

on Tuesday

Andy Nahapetian is now a member of Hadoop Professionals

Welcome Them!

March 12

Jason Venner added an event

Bay Area Hadoop User Group (HUG) March Meetup at Yahoo Campus Building C, Second Floor, Classroom 5

March 11, 2010 from 6pm to 7pm

Building C, Second Floor, Classroom 5 It's in the same campus, just cross the street and walk pass building D to Building C 6:00 - 6:20 - Socializing and Beers 6:20 - 6:50 - Preview to the Hadoop Security Release Owen O'Malley, Yahoo! 6:50 - 7:2…

March 11

Jason Venner replied to yigang chen's discussion 'Legacy code running in mapper'

You can build your own symbolic link by running a command from java, you just need to verify where the data is unpacked, and then build a link to it. A quick search turned up the following page for sample java code for you: http://www.giannistsakir…

March 9

Jason Venner replied to sanjo cole's discussion 'hadoop or mysql?'

If your data is highly relational, your users will have a simpler time accessing it if it is stored in a more traditional data warehouse. The sizes you are talking about are very small, I have some of the higher end solid state devices for storage,…

March 9

Rodrigo Carvalho Rezende and sanjo cole joined Hadoop Professionals

March 9

yigang chen replied to yigang chen's discussion 'Legacy code running in mapper'

So that means I'd need to modify the legacy code, i.e., change the hard coded: "a/relative/path/to/my/file.xml" to: "./mymeta.zip/a/relative/path/to/my/file.xml" Is there a way at all to NOT change the legacy code?

March 8

sanjo cole added a discussion

hadoop or mysql?

hi,i'm working on a data warehouse and am deciding whether to use hadoop or mysql.the dataset is currently likely to be no bigger than 40gb for the first year, then perhaps 80gb for the next year, and possibly 120gb the year after.we want to be able…

March 8

Anand updated their profile

March 8

Anand and yigang chen joined Hadoop Professionals

March 8

dachuan huang replied to dachuan huang's discussion 'how could i setup a hadoop source project?'

where can i find this code bundle? at present, i just want to run some simple examples which just some internal classes are involed, it's not necessary to re-build the whole hadoop source project. thank you.

March 7

Jason Venner replied to yigang chen's discussion 'Legacy code running in mapper'

If you pass -archives mymeta.zip there will be a symbolic link in the current working directory for the map or reduce task mymeta.zip, which points to the directory that the archive was unpacked in. so if you use ./mymeta.zip/path_in_archive/file.xm…

March 7

Photos

Loading…

Help With Hadoop

A great place to learn Hadoop, and to tune your map reduce jobs.

Ask specific Hadoop questions here to get help from an expert :)

Forum

Question: One instance of Hadoop can only run one Map Reduce Job at once?

Started by Elton Tian 1 day ago.

When should I jump on HBase rather than RDBMS?

Started by Elton Tian Mar 16.

Legacy code running in mapper 3 Replies

Started by yigang chen. Last reply by Jason Venner Mar 10.

Events

Add an Event

Blog Posts

datanode can not connect to the namenode in a small hadoop cluster

Hey there I have a hadoop cluster build on 2 servers (2 laptops). One node (A)

contains the namenode, a datanode, the jobtraker and a tasktraker.

The other node(B) just has a datanode and a tasktraker.

I set up correctly hdfs with ./start-hdfs.sh

When I try to set up MapReduce with ./start-mapred.sh the
TaskTraker of node (B) can not connect to the namenode. The tasktracker log will

keep throwing:

INFO org.apache.hadoop.ipc.Client: Retrying conne… Continue

Posted by Marc Sturlese on February 15, 2010 at 7:00am — 3 Comments

hello...is this site a bit inactive or am i the only one feeling it...please make sm noise

....

Posted by nijil on February 14, 2010 at 6:40am — 1 Comment

seeking advice on word vectors

Hello all,

Hope all is well in the community. I am inquiring on how to apply hadoop to retrieve information from various blogs, news feeds, etc.. in a particular fashion.

I have identified three groups of word pairs that are valuable to me. I would like to explore the clustering patterns among particular URL's of these particular word pairs in their respective blog spaces, news feeds, etc.

So, given that I have an expec

… Continue

Posted by Mark Cejas on February 13, 2010 at 10:41am — 2 Comments

.bashrc file error

Hello all,

I hope that the holidays are going well,
I finally have my graduate school work behind me and have more time to learn about this wonderful Hadoop tool. I work on a Fedora 11 distribution and upon getting my JAVA_HOME and HADOOP_HOME paths set, I started to encouter the following error. The error is is observed upon establishing root user as follows:

[rasaan@rasaan ~]$ su
Password:
bash: /root/.bashrc: line 9: unexpected EOF while looking for matching `)'
bash: /root/.bashrc: line 14… Continue

Posted by Mark Cejas on December 31, 2009 at 12:23pm — 1 Comment