Oak Ridge National Laboratory Tests the Limits of Long-Haul InfiniBand over UltraScienceNet using Obsidian Longbow Range Extenders
RDMA Performance Demo - Supercomputing 2007 - ORNL Booth #538
RENO, Nevada - October 31, 2007 — Obsidian, the leader in InfiniBand range-extension, is providing Oak Ridge National Labs (ORNL) with a means of testing long-haul InfiniBand over UltraScience NET (USN). Dr. Nageswara Rao, USN's co-Principal Investigator said, "We are interested in technologies that will help us move increasingly large datasets between Department of Energy (DOE) labs and facilities. The Longbows on USN are posting remarkable performance numbers and offer a very attractive alternative to TCP/IP".
The DOE operates many of the world's largest "capability" supercomputers, producing and consuming data files that are growing exponentially in size as ever larger machines are deployed - increased data capacities enable larger, higher-fidelity physics simulations. However, this growth trend presents a critical problem to the DOE - "When supercomputer users need to move super-sized files, shared-infrastructure TCP/IP-based networks are unable to handle the bandwidth overload.", remarked Dr. Rao, "Clustering techniques for processors and storage have allowed the computing machinery to scale, but the conventional approaches to networking them together have not kept pace".
ORNL has been evaluating Obsidian Longbow XR devices since September 2006, running a variety of tests over USN, which is well suited as a next-generation networks testbed. Comprising a pair of OC-192 SONET channels between Oak Ridge, Atlanta, Chicago, Seattle and Sunnyvale, it can be configured to provide loops of varying lengths. Results have been highly encouraging according to Dr. Rao - "Longbow XRs configure trivially easily, simply appearing as two-port InfiniBand switches that completely hide the optical Wide Area Network (WAN) from the InfiniBand servers and storage. Using InfiniBand's Remote Direct Memory Access (RDMA), we found we could stream data across all 8,600 miles of USN with an essentially flat bandwidth-distance curve, right out of the box. To see this kind of plug-and-play performance with no in-situ tuning, and independent of distance, is in striking contrast to normal network performance".
Dr. David Southwell, President and CEO of Obsidian Research, said "We are most pleased with Dr. Rao's results using Longbow XRs across USN, and look forward to working with the DOE in the future". Work is already underway to integrate the InfiniBand-enabled Lustre file system to provide file level native InfiniBand transport between DOE locations at very high speeds.
Dr. Rao is providing live demonstrations of this capability across USN between ORNL in Tennessee and Reno during SC|07 in the Oak Ridge National Labs booth.
Obsidian Research Corporation and the Obsidian Longbow LP are the developers of Longbow, a series of InfiniBand range extension products. Longbow technology allows an InfiniBand fabric, normally a short-range network used in high-performance computing, to be extended via optical fiber over varying distances. Longbow connects across Campus, Metro or Global networks to offer unparalleled high-bandwidth, low-latency access to InfiniBand compute and storage resources. For more information, see www.obsidianresearch.com.
About UltraScience Net:
The next generation DOE large-scale science applications have requirements that will drive extreme networking, in terms of sheer high throughputs as well as dynamically stable connections. The UltraScience Net is (USN) an experimental research testbed funded by DOE Office of Science to enable the development of hybrid optical networking and associated technologies to meet the unprecedented demands of large-scale science applications. For more information, see www.csm.ornl.gov/ultranet/.