Online trading company Scottrade recently moved its data center to a new US$25 million IT facility.
The new data center, located in a St. Louis suburb, is the largest technology investment that Scottrade has made since it was founded in 1980, according to CIO Ian Patterson.
"This wasn't as much a move as it was a revamping of the technology," Patterson said. "We actually redesigned the entire network core and all the infrastructure on the trading floor. It was a greenfield approach."
The 34,000-square-foot data center boasts a 10 Gigabit Ethernet backbone, versus Gigabit Ethernet in the old facility, and the PowerEdge blade servers that support Scottrade's trading floor have been upgraded to Dell Inc.'s so-called 9th Generation models. The raised-floor portion of the facility comprises about 11,000 square feet, large enough to hold about 100 people in addition to all the systems, Patterson said.
The data center includes exterior security cameras that prescreen visitors, while internal security measures include the use of card keys, motion-activated surveillance cameras and biometric palm scanners.
The facility also features fully redundant electrical and cooling systems. Two liquid chillers do the cooling, with a third that can be added if the raised-floor portion of the data center ever needs to be expanded, Patterson said. Dual-power feeds are connected to each server rack, and some have quadruple electricity hookups. In addition, each of Scottrade's two backup generators can support the entire data center if the other goes down.
Because of the situation with the electrical grid in St. Louis, Scottrade wasn't able to tap into multiple power suppliers -- a common strategy in some data centers. "That's why we went with dual generators and a battery backup system," Patterson explained. "Our battery system is rated for seven minutes but in testing [ran] for 10. And our generators take about one-and-a-half minutes to power up, so we're covered."
Another improvement is the addition of a loading dock that includes a large storage area, plus an equipment burn-in room directly between the dock and the data center. "In the old facility, we didn't have enough dock space to get new products in, so we made sure to improve that flow," Patterson said.
The new data center is located about 8 miles away from the old one, so personnel issues weren't an issue in the move. Patterson said that the upgraded facility was needed to avoid trading downtime and latency, and that it should be able to handle Scottrade's processing requirements for the next five years.
"Seconds mean dollars to us," he said. "When you look at what's happening in the stock market in terms of volumes, it was clear we needed to do something. When I started, a big day was 110,000 trades. Now it's over 250,000."
He added that given the company's growth, it was clear that the old data center would have sufficed for only another two years before Scottrade began running out of space and power. Instead of spending money to revamp the old data center, the company figured it would cost about the same to build a new one.
In the new setup, all systems monitoring -- of computers as well as network gear -- is handled on a separate network, to keep that traffic away from the transactions processed by the trading and other production systems.
Patterson gives a lot of credit to his IT team for the 12-month data center construction and migration effort. But he said that Scottrade also worked very closely with all of the networking and systems vendors involved in the project, particularly Cisco Systems Inc. and EMC Corp.
"I contracted with them so they could spend a few months learning our business," said Patterson, who worked as a consultant himself for 17 years. "We had them sit with us and go through our current system architecture, uptime metrics and so forth. Then we were able to talk about coming up with a new design. These guys lived and breathed this with us -- they helped us design a better mousetrap, and they all have skin in the game."