US energy agency demos blazingly fast network

The U.S. Department of Energy will debut in November one of the fastest networks ever built: a 100Gbps Ethernet network that will enable researchers to create more complex, real-world simulations in climate change, particle physics, astronomy and other scientific fields.

DOE is demonstrating its 100G upgrade to the Energy Sciences Network (ESnet) at a supercomputing conference being held in Seattle from Nov. 12-18. Previously, ESnet provided 10Gbps links.

"Since 1990, our traffic has grown by a factor of 10 every 47 months on average," explained Steve Cotter, ESnet department head. "We are in the age of observation right now in science. All of these scientific instruments and experiments are collecting significantly more data than they ever have in the past. This data needs to be stored ... and moved around. We've been using this 10 Gigabit network ... but we knew that if we didn't start planning now for a 100 Gigabit network, the demand would overwhelm us.''

PLANNING GUIDE: High-speed Ethernet

NETWORK RESEARCH: Follow our Alpha Doggs blog

The 100G ESnet upgrade was funded through the Obama administration's economic stimulus package in February 2009.

"We received stimulus funds to build this 100G network with the intention of doing two things. One was accelerating the deployment of 100G so that the equipment manufacturers didn't shelve the technology on fears that there wouldn't be demand. ... The other reason they gave us the money was to build a next-generation network test bed and to fund network research," Cotter said.

Costing an estimated $62 million, the 100G network will link three DOE research centers: Lawrence Berkeley National Laboratory in California, Argonne National Laboratory in Illinois and Oak Ridge National Laboratory in Tennessee. It also will connect with a key Internet exchange point in New York for connecting to research networks overseas.

The 100G network is being built using dark fiber from Level 3 Communications, high-speed services through the Internet2 research consortium and routers from LGS Innovations, the federal sales arm of Alcatel-Lucent. The systems integrator on the deal is a small, minority-owned business called Synchronized Networking Solutions in Penrose, Colo.

LGS Innovations is providing 14 Alcatel-Lucent 7750 Service Routers. Other router manufacturers that bid on the 100G ESnet upgrade are believed to be Ciena and Juniper. Juniper was the incumbent router provider for ESnet.

Steve Gilke, who handles technical sales for LGS Innovations, said the ESnet upgrade is one of the first networks to handle 100G at the edge. "Nobody has done that before," he added. "They've run 100G on the National Lambda Rail backbone, but they've never approached that speed in Ethernet at the edge."

Winning the ESnet upgrade is significant for LGS Innovations, which is marketing its 100G routers and hinting at a future of 400G within the next few years.

"This [deal] is cutting-edge technology, it's a client on the leading edge, and it's one of Alcatel-Lucent's newest and greatest products," said Robert Farr, chief marketing officer for LGS Innovations. "This is good for the federal channel. Agencies are looking for new and exciting technologies, not just for the federal labs but for federal business."

One unique aspect of the 100G ESnet backbone is that instrumentation will be deployed to measure how much energy the network itself uses across its 12,924 miles of dark fiber.

"We are going to instrument the network for energy consumption measurements," Cotter said. "We believe we are the first network at this scale to collect that information and make it available to researchers so we can begin to look at the energy consumption of networks. There's been a lot of effort into making data centers more energy efficient but very little has been done to look at the networks that connect data centers. ... Maybe we'll be able to provide dynamic routing based on the source of energy and how green it is."

Cotter said the financial industry has deployed 100G on a smaller scale and that content providers such as Facebook and Google are evaluating it. He believes the ESnet upgrade helped motivate router manufacturers to bring 100G products to market despite the recession.

"100GE is not really revolutionary. I look at it as evolutionary," Cotter said. "Science is hinging on the ability to analyze the data. If we can make it easier to move that data to the supercomputers where it can be analyzed, we're hoping to speed up the cycle of scientific discovery. That, in turn, will lead to a better understanding of the origins of the universe or accelerate advances in medicine or healthcare."

Read more about lan and wan in Network World's LAN & WAN section.