A Community for Hadoop Users

Saranyu Netmanee: Share on Facebook Share Twitter

Blog Posts
Discussions (10)
Events
Groups
Photos
Photo Albums

Saranyu Netmanee's Apps

Saranyu Netmanee's Discussions

How to run and writting program on cluster?

Started Feb 4, 2010

How to set hadoop cluster?
13 Replies

Started this discussion. Last reply by Saranyu Netmanee Jan 11, 2010.

View All

Saranyu Netmanee's Page

Gifts Received

Saranyu Netmanee has not received any gifts yet

Give Saranyu Netmanee a Gift

Latest Activity

How to run and writting program on cluster?

I writed program count website from log file which I send parameter when run program( bin/hadoop jar webcount.jar WebCount webtudom.log output tudom ) tudom is third parameters which in wordcount when run program will send 2 parameter are in and out . I test run my program with single node then have result and no error but I test run my program with multiple node then no result. I don't know why so that. I attach my program for you check.Thak for advance.See More

Discussion posted by Saranyu Netmanee Feb 4, 2010

0 Comments

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

And how to configure all files in folder conf. I read from book and website apache but I don't understand to configure file in folder conf. And I think cause of error due to wrong configuration file in folder conf. Thank you.

Jan 11, 2010

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

I have run the wordcount program on single node setup and multiple node setup. The file size was approximately 800 MB. The multinode set up comprises 1 master and 3 slave machines connected to a single L2 switch with 10Mbps links. Each machine has a…

Jan 11, 2010

Jason Venner replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

The other reason this can happen is if the hostname to ip address translation for the various task tracker nodes return incorrect ip addresses. verify that on each task tracker, the lookup of the hostnames of the other tasktrackers actually returns…

Jan 9, 2010

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

I'll check firewall again. And file in folder me configure correct or incorrect ? I thought wordcount program is good for run on single node because had low complexity then use time for run less than. But when run on multi node then use very ?…

Jan 8, 2010

Jason Venner replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

It appears that the reduce tasks are unable to fetch the map outputs. 10/01/07 14:39:29 WARN mapred.JobClient: Error reading task outputSlaves-02 10/01/07 14:39:29 WARN mapred.JobClient: Error reading task outputSlaves-02 10/01/07 14:39:29 INFO…

Jan 8, 2010

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

Now I try run word count on single node file 800 MB use 5.3 Minutes but run on cluster ( 1 master and 1 slave ) use 9.18 minutes. I want to known how to develop application for run on hadoop cluster. And I want to known WordCount example can run on…

Jan 7, 2010

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

I'am sorry. Have error when wordcount file size about 800 MB. But run wordcount file 1 MB no problems. I attach error result. And please check configuration in folder conf.

Jan 7, 2010

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

No task failures. After run wordcount success will show in table Completes.

Jan 7, 2010

Jason Venner replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

Did you have many task failures?

Jan 5, 2010

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

Now I'm can run word count on hadoop cluster successful but used long time. But run word count on single- node is less than.Single node used 14 seconds and Multinode used 2 minutes 17 seconds. (In file .txt about 38,000 words)

Jan 5, 2010

Jason Venner replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

What is the error when you run the word count example?

Dec 27, 2009

Saranyu Netmanee replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

Yes I can set ssh to access each machines and I use command bin/hadoop namenode -format and bin/start-all.sh at Master node and use command jps it show Jps Namenode Jobtracker and SecondaryNamenode. And I use http:masternode:50030 it show State…

Dec 27, 2009

Jason Venner replied to Saranyu Netmanee's discussion 'How to set hadoop cluster?'

my very first question is: as the user that the hadoop server processes run, on the machine that runs the namenode/jobtracker can you ssh to each of your datanode/task tracker machines without needing to enter a password.

Dec 27, 2009

How to set hadoop cluster?

I set hadoop cluster follow chapter3-4 in Pro Hadoop. I use 4 machines; 1 master and 3 slaves , Ubuntu os , hadoop 0.19.2. I use command bin/hadoop namenode -format and bin/start-all.sh and jps will show namenode datanode jobtracker tasktracker secondarynamenode and jps . And in hadoop-site.xml I copy from book. But I running wordcount it cannot to run. I don't understand to confige file .xml. I need to configuration hadoop cluster step by step, please.Thank you.See More

Discussion posted by Saranyu Netmanee Dec 25, 2009

13 Comments

Saranyu Netmanee is now a member of Hadoop Professionals Dec 25, 2009

Welcome Them!