Friday, March 28, 2008

This Yahoo!/CRL announcement comes on the eve of the first ever Hadoop Summit. Apache Hadoop is a Free Java software framework that supports data intensive distributed applications running on large clusters of commodity computers. It enables applications to easily scale out to thousands of nodes and petabytes of data. In Feb 2008, Yahoo announced that its Search Webmap application has been converted to a Hadoop application that runs on a more than 10,000 core Linux cluster and produces data that is now used in every Yahoo! Web search query.

CRL’s EKA supercomputer is a Hewlett-Packard based system with 14,400 processors, 28 terabytes of memory, 140 terabytes of disks, a peak performance of 180 trillion calculations per second (180 teraflops), and sustained computation capacity of 120 teraflops for the LINPACK benchmark. Of the top ten supercomputers in the world, EKA is the only supercomputer funded by the private sector that is available for use on commercial terms.

The actual supercomputer was built in just 6 weeks. However, other aspects took much longer. It took an year of convincing to get the project funded. And another year to build the physical building and the rest of the infrastructure. Eka uses

2.5MW of electricity
400ton cooling capacity
10km of electrical cabling
10km of ethernet cabling
15km of infiniband cabling
The computing infrastructure itself consists of:

1800 blades, 4 cores each. 3Ghz for each core.
HP SFS clusters
28TB memory
80TB storage. Simple SATA disks. 5.2Gbps throughput.
Lustre distributed file-system
20Gbps infiniband DDR. Eka was on the cutting edge of Infiniband technology. They sourced their infiniband hardware from an Israeli company and where amongst the first users of their releases - including beta, and even alpha quality stuff.
Multiple Gigabit ethernets
Linux is the underlying OS. Any Linux will work - RedHat, SuSe, your favorite distribution.

Actual article: http://punetech.com/yahoo-and-crl-eka-to-collaborate-on-cloud-computing-research/

No comments: