As business intelligence becomes a critical component of daily operations, real-time data warehouses that can provide end users with rapid updates from transactional systems are increasingly sprouting up at companies.
For example, online retailer Overstock.com has begun connecting users to a real-time data warehouse it completed last month. The project's goal is to help employees gain insight into the effectiveness of the company's online and e-mail advertising campaigns.
Overstock is using transactional data management tools from GoldenGate Software to pull information directly from its business systems into the data warehouse, said Jack Garcella, the Salt Lake City-based retailer's vice president of data warehousing, analytics and reporting.
The data warehouse, which is based on NCR's Teradata software, will replace a process that used traditional extract, transform and load tools to build reports directly from Overstock's back-end systems. As the retailer grew, the reports stressed the systems and gave employees day-old data, Garcella said. Now the data warehouse receives Web site clickstream data in real time, financial and product-sales data every 15 minutes and other information hourly.
"When we launch campaigns now, we can look within five minutes and see if they are producing lift or revenue that would not normally have happened," Garcella said. "You can't wait until the next day or three hours later to get that data." He declined to specify how much Overstock is spending on the warehousing project, other than to say the cost is in the millions of dollars.
Harrah's Entertainment is testing a real-time data warehouse that combines operational and historical customer data, said Tim Stanley, the Las Vegas-based gaming company's CIO.
The new setup is based on an architecture that Harrah's developed in mid-2002. The company is using adapters from Tibco Software to feed information from transactional systems into its Teradata warehouse to help workers interact with customers at Harrah's properties, on the phone or on the Harrah's Web site.
"It uses Teradata's transactional database and also has direct access to all the historical data," Stanley said. "You don't have to have two databases talk to each other."
Changing Needs
Eric Rogge, an analyst at Ventana Research, said that because business intelligence tools are being used more often for operational decision-making, many companies are finding that they need to refresh their data warehouses more frequently than on a nightly basis.
"It's not about loading a data warehouse so a small department of business analysts can forecast two years out -- it's for daily decisions," he said.
For the past 18 months, Avnet Electronics Marketing has been using a near-real-time data warehouse that captures orders and updates of logistics data from its back-end system every 15 minutes, said Kevin Harrington, director of IT delivery for global information solutions at the Phoenix-based electronics distributor.
Avnet uses tools from Informatica to move the data into the warehouse. Because of the integration infrastructure, it took only 24 hours in late July to begin populating the warehouse with order and customer information from a company that Avnet recently acquired, Harrington said.
But not all users find they need real-time data warehouses. Merial, which makes medications for pets and livestock, last year ditched efforts to create a real-time system for updating sales and inventory data from its 33 ERP systems worldwide. Although some divisions updated invoicing information daily, others did so only weekly or at the end of the month, said Steve Lerner, director of information systems, global finance applications and integration.
In the end, the company decided to use data warehousing tools from Kalido to pull data from its ERP systems once a week. "The consensus among the business users was that there was no way they were prepared to make business decisions based on sales other than on a weekly basis," Lerner said.