Hadoop Professionals

A Community for Hadoop Users

All Blog Posts (39)

Chaula Ganatra Hadoop MapReduce to compare Relational Data Stored in text File

Hi All,

We are reading two txt files which contains relational data and then we generate java objects from these data.

Then we compare the objects of two files. We want to find which objects differ in both the files.

Note : We can not compare Strings (line by line) of one file to another file because one line contains reference of another line of the same file and this way it has tree of references.

Again each line may have different data structure (we…

Continue

Added by Chaula Ganatra on January 13, 2012 at 1:37am — No Comments

Bhavesh Shah Query related Hadoop's Map-reduce

Scenario:

I have one subset of database and one dataware house. I have bring this both things on HDFS. I want to analyse the result based on subset and datawarehouse. (In short, for one record in subset I have to scan each and every record in dataware house).

Question:

I want to do this task using Map-Reduce algo. I am not getting that how to take both files as a input in mapper and also how to handle both files in map phase of map-reduce. Pls suggest me some idea so…

Continue

Added by Bhavesh Shah on January 3, 2012 at 12:35am — 2 Comments

radhakrishnan_cse Need help ?? Can u say this ??

I am currently doing project in energy efficiency and reliability in hadoop.. Can any one tell me which class in hadoop doing block spliting and block allocation and block replication.. Also whether is it possible to dynamically add nodes in cluster..if it is possible tell me the steps.. Can any one give the answer..? Plz soon..

Added by radhakrishnan_cse on December 28, 2011 at 8:30am — 2 Comments

Dmitriy Goldin Hadoop architect/developer needed for a premier brokerage firm

A major brokerage firm is looking for a senior Hadoop architect/developer. You will work with a variety of financial data, trading and back-office-related. Security Master knowledge is helpful.

You will work on the infrastructure end, working on logical and physical architecture. Your main responsibility will be working with Terabytes of data and organizing it into data domains. You should have extensive experience with data implementations, data storage, data access and…

Continue

Added by Dmitriy Goldin on June 20, 2011 at 6:55am — No Comments

Muhammed Irshad Loading tables Using Serde

Can any one give an explanation on Serde option of loading data into tables ....

Added by Muhammed Irshad on May 31, 2011 at 10:12pm — No Comments

Amy Caruso Flurry is Hiring!

Flurry is Hiring! Here are just a couple of the open positions!

Please go to our website www.flurry.com to apply!

About Us

Flurry is the world's leading mobile analytics and advertising platform. We track over 11 Billion user sessions each month while serving more than 37K mobile app developers and over 70 K mobile applications across the iOS, Android, Blackberry, and Windows Phone 7 platforms. The company…
Continue

Added by Amy Caruso on April 18, 2011 at 3:45pm — No Comments

Mendeil Bailey Living in the Era of Hadoop and Large-Scale Data

It’s clear that now we are living in the era of big data. The stores of data on which modern businesses rely are already vast and increasing at an unprecedented pace. Organizations are capturing data at deeper levels of detail and keeping more history than they ever have before. Managing all of the data is thus emerging as one of the key challenges of the new decade.



The solutions to this challenge vary, but interest in them seems to be universal. The largest database vendors and…

Continue

Added by Mendeil Bailey on April 11, 2011 at 11:20am — No Comments

MIT Hadoop Developer position in Israel

Hands-on experience with installing, developing and integrating with Hadoop
BSC / MSC in computer science

Please contact Neta: netak@mit.co.il

Added by MIT on December 29, 2010 at 8:12am — No Comments

Anantha Srivathsa N How Big Data is analyzed using Hadoop





 

Introduction

Hadoop is rapidly becoming the technology of choice for enterprises that need to effectively collect, store and process large amounts of structured and complex data.

 

The Pentaho BI project is open…

Continue

Added by Anantha Srivathsa N on December 19, 2010 at 11:00pm — No Comments

Adarsh Measure shuffle time and bandwidth

Hi,

I am trying to evaluate the dependency of hadoop on the underlying networks and hence need to find the bandwidth and shuffle
time during a hadoop job. Can anyone please let me know how to get the
bandwidth and the shuffle time during a job.

Thanks in advance

regards,
Adarsh

Added by Adarsh on November 18, 2010 at 3:18pm — No Comments

Adarsh Upper limit for bandwidth in hadoop

Hi,

I was going through the hdfs-default.xml and found an entry dfs.balance.bandwidthPerSec. So my question was, is there an upper limit set for bandwidth in hadoop. Please let me know if my understanding is correct.

Thanks in advance,
Adarsh

Added by Adarsh on November 14, 2010 at 9:57pm — 1 Comment

Amy Tierney Java Developers needed in Scottsdale, AZ with strong Hadoop knowledge and experience



Conde Group, Inc. (www.condegroup.com) a consulting and staffing firm based in southern California focuses on helping our clients to acquire and retain…

Continue

Added by Amy Tierney on November 10, 2010 at 7:36am — No Comments

Vincent Garrigoux Hadoop Permanent positions in Switzerland

Hello there !
I am actively looking for Hadoop Specialists. One of my customers, an US famous company with offices in Switzerland, is offering several permanent positions.
Please drop me a line in case of interest and I will tell you more.
Regards.
Vincent, Perl-Ressources SA

Added by Vincent Garrigoux on August 17, 2010 at 10:41pm — 1 Comment

Vincent Garrigoux Open Hadoop Positions in Switzerland

Hello dear Hadoop Professionals,


Perl-Ressources is an IT / IS specialized sourcing company.


One of our main customers, a very well known American company with offices here in Switzerland, is actively looking for experienced Hadoop specialists. The job descriptions are ready and can be sent by email in case of strong interest.


Please contact me first on:…
Continue

Added by Vincent Garrigoux on August 17, 2010 at 8:47am — 1 Comment

Louisa Landry Problem with activation

Hi there, I dont know if I am writing in a proper board but I have got a problem with activation, link i receive in email is not working...
http://www.prohadoopbook.com/?09a262b1d40cab9a32dd7416e68,

Added by Louisa Landry on March 23, 2010 at 8:45am — No Comments

Marc Sturlese datanode can not connect to the namenode in a small hadoop cluster

Hey there I have a hadoop cluster build on 2 servers (2 laptops). One node (A)

contains the namenode, a datanode, the jobtraker and a tasktraker.



The other node(B) just has a datanode and a tasktraker.

I set up correctly hdfs with ./start-hdfs.sh



When I try to set up MapReduce with ./start-mapred.sh the TaskTraker of node (B) can not connect to the namenode. The tasktracker log will

keep throwing:





INFO org.apache.hadoop.ipc.Client: Retrying… Continue

Added by Marc Sturlese on February 15, 2010 at 7:00am — 4 Comments

Mark Cejas seeking advice on word vectors

Hello all,


Hope all is well in the community. I am inquiring on how to apply hadoop to retrieve information from various blogs, news feeds, etc.. in a particular fashion.


I have identified three groups of word pairs that are valuable to me. I would like to explore the clustering patterns among particular URL's of these particular word pairs in their respective blog spaces, news feeds, etc.


So, given that I have an…
Continue

Added by Mark Cejas on February 13, 2010 at 10:41am — 2 Comments

Mark Cejas .bashrc file error

Hello all,



I hope that the holidays are going well,

I finally have my graduate school work behind me and have more time to learn about this wonderful Hadoop tool. I work on a Fedora 11 distribution and upon getting my JAVA_HOME and HADOOP_HOME paths set, I started to encouter the following error. The error is is observed upon establishing root user as follows:



[rasaan@rasaan ~]$ su

Password:

bash: /root/.bashrc: line 9: unexpected EOF while looking for… Continue

Added by Mark Cejas on December 31, 2009 at 12:23pm — 1 Comment

Jason Venner I am giving a talk at the HUG on Wed, scaling search with hadoop, katta and solr

Jason Rutherglen will be providing the in depth lucene/solr pieces.

Hope to see you there.

Added by Jason Venner on November 17, 2009 at 12:57pm — No Comments




Groups

© 2012   Created by Jason Venner.

Badges  |  Report an Issue  |  Terms of Service