Betting on the IT department

Marshall Andrew has two big reasons to be nervous.

The former Chicago resident's favorite football team will be vying for its first Super Bowl championship in two decades. But Andrew will scarcely have time to enjoy the game, because he is the CIO and vice president of technology for Station Casinos , which operates 15 Las Vegas casinos. Super Sunday is potentially the toughest day of the year for a casino IT department.

"It's our largest betting day of the year, and you hope that everything just stays together and you don't have any issues," Andrew says. "If you do have issues you don't have enough time to respond. If your [systems are] out, you can't take bets. I would say in my job here I really haven't had a chance to enjoy a Super Bowl."

Betting increases tremendously several hours before the big game and at half time, Andrew notes. Forty or 50 of Andrew's 120 employees will work on Sunday. Andrew will be stationed at what he calls a "command center" that links the 15 casinos, but says he can't say where this central location is because of security concerns.

"We bring extra staff in for both Saturday and Sunday, and we have other staff on call from both hardware support and network support and software application support," he says.

Each of the 15 casinos typically has two IT staffers on site but will have more this weekend.

What could go wrong on Super Bowl Sunday? There could be problems with the network linking the 15 casinos, routers, switches, or the applications that handle betting, for example.

"We've had problems with third-party software two or three times on Super Bowl, where it couldn't handle the volume," Andrew says. "It'll just lock up and people can't place their bets. We call the vendor and see if they can help us out."

Another potential concern is a pager system that handles remote bets customers place on touch-tone phones. By law, Station Casinos can only take bets from within Nevada. A patron who wants to use the phone betting service is given a pager by the casino chain. This pager verifies the gambler is inside Nevada because it does not work outside the state's borders. When a customer wants to place a bet, he or she calls the remote betting service, which sends a password to the pager. Then the customer calls back with a touch-tone phone and enters the password.

Station Casinos also allows betting from home computers through an intranet, which customers can access with software provided by the casino chain. The remote betting systems have been in place six years.

"Those are another level of complexity we always have to watch for and we're concerned with," Andrew says.

When Andrew took his current job 12 years ago, the company had one casino and he had five employees. Now there are 15 casinos located off the Vegas strip, including Red Rock, Green Valley Ranch and Palace. The IT department supports 270 servers, about 3,500 desktop computers, about 220 electronic gaming machines, and a variety of kiosks where players can redeem points similar to frequent flyer miles.

"It's amazing the amount of technology that's used in our business. There's almost nothing you do here that is not technology related," he says. "When we first got here we didn't have computers on gaming tables, we didn't have kiosks, we couldn't do this type of player card and player tracking."

Andrew previously worked for a large paper company, as well as Ingress Software and in Germany for the Esprit clothing retailer. Working for a casino chain is far more complicated not only because of strict gaming regulations but also because Station Casinos is essentially working in five industries: gaming, entertainment, retail, food and beverages, and hotels. "We have to link all the systems together for these five different types of industries," he says.

Page Break

Part of that job is done by IBM's System i platform, which handles applications for property management, casino management, point-of-sale transactions, and financial and human resources management.

The system is involved in probably 50% of Station Casinos' business activities, including player tracking, data warehousing and purchasing, Andrew says. It helps address Andrew's biggest concern as CIO: making sure the systems run smoothly 24 hours a day, because every minute of downtime means lost revenue.

"We try to build in a lot of redundancy," he says. "For networks we have dual switches and dual paths of fiber for our hardware. We use a lot of Microsoft clustering for the IBM side. . . . Redundancy and high availability is a key factor."

Security is also a big concern in a casino IT department, obviously. Protecting revenue from embezzlement is Andrew's top focus in this area, he says.

"This is closely audited and controlled. There can't be any leakage. It's not a casual thing. Every betting machine is like a point-of-sale cashier terminal," he says.

There are checks and balances to make sure employees can't take money or access sensitive information they're not supposed to see, he notes.

"Basically, we handle it by limiting who has access to what systems and what information," he says. "It's closely controlled and guarded. We're controlled here by the Nevada Gaming Commission , which has to approve all gaming systems. So it's gone through a lot of scrutiny and testing."

IT employees are not allowed to gamble at Station Casinos properties because they might have the ability to write code and cheat the system, or may have knowledge about the slot system's performance and know which machines pay out more often. Table games are off limits, too, to prevent collusion between dealers and other employees.

The surveillance department has the primary responsibility for finding cheaters among casino customers, but Andrew's staff has some involvement there too. "Our job is to make sure that there's security in place and that it's maintained, and that if there's any suspicion of anything happening, we'll be asked to assist," he says.

Beyond running day-to-day operations, maintenance and upgrades are key to the casino IT department's success. Currently, IT employees are writing a new race and sports system to get up to date with modern technology. The system will improve reporting on who customers are, allow betting on touch screens, and improve remote gaming and security. Andrew says the sports portion has been installed in about five properties and he hopes to have state approval to install the whole race and sports system in all 15 casinos by the end of 2007.

"That's a large project. We've been working on it for a while now," he says.

As for Super Sunday, Andrew is hoping nothing unexpected goes wrong but is also confident in his casino chain's technology. "I think the Bears and the Colts are spending more time preparing," he says. "We're pretty comfortable . . . we can handle the volume."

Andrew is also confident the Chicago Bears will stop Peyton Manning and the Indianapolis Colts' high-octane offense.

"I'm a Bears fan, I used to live in Chicago," Andrew says. "I'm also a big football fan and I think defense always wins out in Super Bowls."