When US petroleum giants Ultramar Diamond Shamrock Corp. and Valero Energy Corp. finalized their megamerger in December, the task of protecting two gargantuan data storage architectures became urgent.
Storage specialists at the merged company, the new Valero, knew they needed to craft a plan for a single, integrated back-up and restore system. The goal is to ensure data availability yet lower administrative costs.
Such integration will be no easy task. The data involved in operating the third-largest U.S. petroleum refining and marketing firm, a Fortune 50 company with annual revenue approaching US$30 billion, is staggering.
The new Valero pumps 2 million barrels per day out of refineries and supplies gasoline to nearly 5,000 service stations nationwide. In so doing, it generates terabytes of data, which reside on hundreds of NetWare, Unix and Windows servers.
Integrating backup is the No. 1 storage management priority for the merged storage team, says James Brents, technical specialist at Valero in San Antonio, Texas. "Our goal is to be able to restore the data first and foremost. Secondly, we have to continue meeting our back-up window as our data grows," he says.
The new Valero backs up an average of 12 terabytes of data nightly across a LAN or storage-area network (SAN). This is storage from 35 AIX and 40 other Unix boxes (running HP-UX or Solaris), 32 NetWare servers and 350 Windows NT or 2000 servers. The largest databases, those in the 100-plus gigabyte range, are backed up over a SAN.
Integrating the previously disparate back-up efforts promises easier maintenance and reduced costs. "We will be able to focus our administrators, and they will be able to share training and experiences," Brents says.
The new Valero currently uses IBM Corp./Tivoli Systems Inc. Storage Manager (TSM) and Hewlett-Packard Co.'s Storage Omniback for backups. TSM handles most backups at the Valero North campus and the NT backups at the Valero corporate offices, where the Omniback handles Unix backups.
"Part of our challenge is evaluating the technologies used by the two companies previously and determining whether it makes sense to integrate and how to do that, or whether it makes sense to standardize on one," says Mark Calkins, senior IT manager at the new Valero. "We are looking for synergies."
Brents won't disclose the cost of the back-up integration project. But he stresses the need for sound data retention policies for backups and archives when it comes to managing storage volumes as large as the new Valero's. His advice is to include back-up, archive and disaster-recovery costs into initial storage plans. In so doing, he says, "don't believe the users when they say they'll only need X amount of space. They always need two times, three times and more."
The storage team says the integration project should be complete in September.
Speed freaks
To meet its second goal of speeding back-up time, the team is investigating LAN-free backups, in which Valero's TSM clients back up data directly to tape drives on the company's SAN. Control information, such as storage policies and metadata, would continue flowing over the LAN to the TSM server.
Transferring data over the SAN rather than the LAN "will help avoid network bottlenecks and take a load off the LAN]," says Pablo Clifton, technical specialist at the new Valero. Today, the new Valero backs up its 1-terabyte SAP ERP database at a speed of 400G bytes per hour. In a recent test, Valero backed up that database in two-and-a-half hours. While the storage team isn't unhappy with that rate, it believes it can improve that time by at least 200G bytes per hour with LAN-free mode, Clifton says.
While Valero seeks out new back-up technologies, it also presses to achieve backups at the published speed of its tape back-up devices. Brents is particularly proud of that practice. "Reducing the back-up window is always the goal, but [you have to] keep in mind that a tape library also needs time to perform the daily administration processes, such as reclamation, off-site copies, expiration and the TSM database backup," he says. "The more time available for those processes to complete daily, the quicker the restores."
Valero tests its back-up and restore procedures often. It conducts major back-up tests four times per year, with project tests multiple times per month and hot data backups six times per week, Brents says.
Testing is a challenge, but absolutely necessary, he adds. "It's really tough to have a duplicate test environment when you have such a large operation."
Remember, Brents says, "you must manage your data, and not let your data manage you."
In Valero's pipeline
The storage team plans to tackle these projects next.
-- Disaster recovery: The new Valero's storage team is conducting a business-impact analysis that it will use to craft a comprehensive disaster-recovery plan for the merged company. The plan will specify back-up and restore procedures on an application-by-application basis. Valero might back up certain applications to the SAN, while it moves others to the LAN.
-- SAN management: The new Valero uses point products for storage management now, but would like to find an umbrella SAN manager for zone configuration, performance reporting and storage allocation. It has evaluated McData Corp.'s SANavigator and Storage Technology Corp.'s SANView software, and expects to make a decision by year-end.
-- Storage reporting: The storage team would like to tie SAN management data into its Hewlett-Packard Co. OpenView and SAS Institute Inc. IT Service Vision management tools.
Variety at Valero
By using NAS and SAN technologies, the company avoids reliance on a single vendor.
While the new Valero considers whether to standardize on a back-up solution, its overall storage environment will remain as diversified as ever.
The company uses network-attached storage (NAS) devices and storage-area networks (SAN), and so has a mixed bag of products. Those include:
-- McData's ED-6064 Fibre Channel Galaxy-Class Director as its SAN backbone at the Valero North campus.
-- Brocade Communications Inc.'s SilkWorm switch for the SAN at the Valero South corporate offices.
-- IBM Corp.'s Shark Enterprise Storage Server for disk storage.
-- Compaq's StorageWorks for SAN.
-- Network Appliance Inc.'s NetApps for NAS.
"We've learned that it's not necessary to stick to one vendor or be afraid to choose an alternate product if it better fits the requirement," says James Brents, technical specialist at Valero.
Likewise, Valero's storage team has designed the SAN to work with a heterogeneous network, Brents says. "We can accommodate a variety of servers, switches, operating systems, applications, security models, etc. Our chosen products have capacity to spare and clear upgrade road maps."