A Community for Hadoop Users

clarification on hadoop....

i have benn looking into hadoop as i am interested in serch engines..and i learnt that it is pretty good at parallelising jobs and map reduce etc
my doubt is can we use hadoop as a sort of platform for cloud computing based webhosting......does that make any sense
owing to the fact of data replication and virtualisation and resource sharing.....can we share load and stuff using this.....i hav just started with it....so please ignore if it doesnt make any sense

thanx 4 the reply 4 previous post

▶ Reply to This

Replies to This Discussion

Permalink Reply by Jason Venner on February 15, 2010 at 6:55am

I don't really understand your question.

▶ Reply

Permalink Reply by nijil on February 16, 2010 at 5:09am

i meant can we use hadoop in a cloud computing setup were the cloud is used to provide web hosting or Iaas(Infrastructure as a service)... for users in a way of "pay as you use" model .....so we need to manage lot of things in such a network right...like load sharing,data replication for safety,sharing of resources,scaling.....so can we use it for such purposes or is it only for map reduce jobs.......i have heard of eucalyptus cloud software for such purpose........one more thing i would like to know is can hadoop fe run on top of another file system only??

▶ Reply

Permalink Reply by Jason Venner on February 19, 2010 at 8:30pm

Hadoop runs quite happily on top of any shared file system, if the file system provides information about the locality of data, hadoop will attempt to schedule tasks close to the source of the data.

I have personally run hadoop on top of a lustre file system and had very good results. I did have very high band width network pipes into the storage nodes.

clarification on hadoop....

Replies to This Discussion

Groups

NoSql

Karma Sphere Studio Users

HBase Users

Hadoop Cascading

Hadoop Administrators