Boston's Suffolk University launched one of the first online MBA programs in 1999. Initially, the school used an application service provider to host its courses, but in 2004, it brought the program fully in-house, using an e-learning system from Blackboard in Washington. This meant changing the IT infrastructure to boost reliability.
"When you have an online program running in-house, you have to make sure the uptime is high enough so professors and students can do their jobs," says Praneeth Machettira, online technical director at Suffolk University's Sawyer School of Management. "Virtualization came in as a way to do disaster recovery."
The school used ESX Server from VMware to create a set of five virtual machines running on ProLiant servers from Hewlett-Packard. But although using virtual servers provided some redundancy, it wasn't enough. A standard backup application wasn't adequate either, because of problems with open files and CPU spikes. So instead, Machettira went with Double-Take from NSI Software to provide full replication in real time of everything taking place on the virtual servers.
"The typical load balancing, clustering and tape backup is not enough," says Machettira. "But by using a combination of virtualization and replication, when a server goes down, we can have Blackboard back up and running in three to five minutes."
A different animal
Server virtualization is cropping up just about everywhere these days. According to IDC, more than three quarters of companies with 500 or more employees use virtual servers, and 45 percent of all new servers purchased this year will be virtualized. Once limited to mainframes and large Unix boxes, the technology is now moving into two- and four-way Linux and Windows servers.
But backing up virtual servers isn't like backing up physical ones. Backup software vendors are doing their part to develop tools that meet the challenges of these new environments, such as avoiding conflicts and resource bottlenecks when several virtual servers are trying to use the same hardware. And companies have found that they need to take a multilayered approach to achieve adequate uptime and reliability.
Companies have several options for creating virtual servers, including Microsoft's Virtual Server 2005, SWsoft's Virtuozzo and the open-source Xen, which is supported by XenSource. Then there's the industry's 800-pound gorilla, VMware, which was bought last year by EMC.
With the expansion in virtualization comes the need to back up virtual servers, and some users were concerned that EMC would try to use its ownership of VMware to promote its own backup products to the detriment of other vendors. But so far, that hasn't been the case.
"I don't see a lot of politics in that area," says Andi Mann, an analyst at Enterprise Management Associates. "They compete against and partner with the same companies."
Charles Keiper, senior NetVault product manager at BakBone Software, says he hasn't seen any changes as a result of the EMC buyout and doesn't expect to. "In the long term, they may provide some level of integration we can't match," he says. "But I do not see them changing the current model, because it would restrict the growth of VMware if they restricted the ability of products to interoperate."
Backup for all
Backup vendors are ensuring that the latest versions of their software can address both physical and virtual machines, and from an administrator's viewpoint, there is often no difference.
"For the most part, a virtual machine works and acts the same way a physical machine does," says Bob Roudebush, director of solutions engineering at NSI Software.
But the backup software has to treat the virtual machines differently. For one thing, there is the matter of resource consumption. Running multiple virtual servers on a single physical server results in better resource utilization during normal operations but can saturate those resources during backup.
"The problem is compounded when you have multiple virtual machines competing for resources from the host system during backup cycles," says Tricia Jiang, technical attache for IBM Tivoli storage systems. "Backups from one virtual machine can starve resources from applications running in other virtual machines." To address this, Tivoli Storage Manager can stagger the backups across low-peak windows.
Then there is the matter of what to back up -- each virtual server individually, or the physical server on which they run. Syncsort Inc.'s Backup Express lets users select either mode.
There is also the option of backing up the entire virtual server as a single file. "This method requires fewer backup agents on the virtual machine but is not application-aware," says Kelly Harriman-Polanski, director of product marketing at CommVault. "It also requires backup of very large files, which are typically 2GB in size or larger, unless the administrator takes the time to execute an export command to convert the file and zero-out the unused portions of the file."
According to Brian Wistisen, senior product manager in Symantec's data management group, the main challenge lies not with the backup itself, but with the process of converting between the virtual and physical environments.
"This is where many solutions face the realities -- and dependencies -- of dealing with all the various low-level hardware devices and drivers necessary to operate the system effectively," he says, "particularly when converting from a virtual state to a physical one."
Most companies are adopting a multi-layered approach to backup. Suffolk University, in addition to using replication, has tape backup for off-line storage and is testing True Image software from Acronis as a way to achieve real-time imaging. Machettira says traditional tape is the university's third or fourth layer of backup.
"Now that we back up to the SAN, we do it disk-to-disk-to-tape," Machettira explains. "We figure tape is the backup of backups when you send something outside for storage."
Terence Choy, network manager at frozen quiche manufacturer Nancy's Specialty Foods, has three VMware virtual servers on a single box running Microsoft's SQL Server. He replicates data instantly between 1TB primary and secondary IP SANs, both from StoneFly. Daily incremental backups are sent to online backup service provider EVault. After the initial data upload to EVault, the daily data changes might be as little as 100MB. Choy uses EVault's management console to configure all the backup and restore jobs.
"It operates the same way, whether you are backing up virtual servers or physical servers," he says.
John Buchanon, senior network engineer at components manufacturer Sypris Solutions, runs 55 servers, including five SAN-connected production VMware ESX servers and 30 production virtual machines running various flavors of Windows and Unix. He runs four to 14 virtual machines on each ESX server.
Buchanon is planning for both disk-to-disk and disk-to-tape backup once he gets additional space in the SAN. But in the meantime, he backs up data nightly to three tape drives using Backup Express from Syncsort. All of the virtual servers communicate with the Syncsort master server by TCP/IP via virtual network interface cards. All of the virtual servers share one or two physical Gigabit Ethernet connections on each ESX server.
"One would expect slower backups, but we haven't seen any significant difference in throughput or backup behavior compared to physical servers of the same class of CPU, RAM and storage," says Buchanon. "Since the master server tests the available throughput per backup task per server, it migrates to the fastest-responding virtual machine at any given time, just as it does with the physical servers."
Virtual perfection
Robert Carver is manager of IT operations at the Southcentral Foundation, a nonprofit organization in Anchorage, Alaska, that operates 65 programs providing health care services to 50,000 Alaska Native and American Indian people. He runs most of his servers on VMware, with the exception of his Microsoft Exchange and main database server. All the rest run on Dell PowerEdge 1855 blades with dual-core Intel Xeon CPUs. It stores the data on a 10TB 920c NAS cluster from Network Appliance and mirrors the data to a NetApp NearStore R200 appliance located across the street from the data center. In addition, the virtual servers store their own files on an iSCSI drive.
"It's not tiered storage, just really redundant," Carver says.
He does a full backup across the street during every downtime window -- about once every two weeks - using BakBone's NetVault software, which then backs up the NearStore data onto tape. Carver also uses NetApp's Open Systems SnapVault (OSSV) to provide up-to-the-minute backups at a file level. He uses PlateSpin's Power P2V migration software to move files from physical to virtual servers. When a virtual or physical server goes down, he then uses NetVault to restore the server from the backup and OSSV to load any additional files from the iSCSI drive. Carver
says backup and restore procedures work equally well for the physical and virtual servers.
"We've had everything from a server erroring out in the middle of a Windows service pack installation to a virtual machine having trouble booting up," says Carver. "But once it was restored, we have never had one come up corrupt."
Server virtualization itself is designed to add redundancy and reliability to enterprise systems. With adequate backup mechanisms in place, it moves one step closer to becoming a fully supported enterprise architecture.
How many licenses?
Backup vendors follow different licensing schemes for backing up virtual servers. Here's a sampling of them:
Symantec LiveState Recovery: On a per-system basis. One license for each instance of an operating system, virtual or physical.
BakBone NetVault: One license per client. If multiple virtual machines are being individually backed up, each requires a license. If the physical server as a whole is being backed up, one license covers the server, regardless how many virtual machines are running on it. n
CommVault Galaxy: Priced per agent.
NSI Software Double-Take for Virtual Servers: One license covers up to five virtual machines running on a single host.
Tivoli Storage Manager: Per processor. Each component installed on a virtual machine is licensed according to the number of processors on the host machine.
Syncsort Backup Express: Same as physical servers -- each system on each node requires a separate client license.
CA ARCserve: One license for the host service and additional licenses for virtual machines.
EMC NetWorker: Separate licenses for each virtual server, but EMC is considering a model where a license would cover both the host and all its virtual servers.