Business battles are fought in real time, and IS must keep pace. Real-time business intelligence infrastructures promise a never-ending stream of fresh information, insight and decision support to frontline knowledge workers.
Nevertheless, real-time business intelligence has not graduated to enterprise primetime yet. Most production business-intelligence implementations rely on data warehouses, which consolidate operational data loaded via scheduled batch transmissions rather than real-time updates from source databases. As a result, many organizations have rich stores of historical data in their data warehouses, but few contain information that is refreshed continuously.
A traditional data warehouse operates in store-and-forward mode, introducing latency into data delivery to reports, dashboards and other business intelligence applications. Most of today's data warehouses have been optimized for specific latency-producing operations: extraction, transformation and loading (ETL) of data from operational database management systems (DBMS); retention of that data in persistent repositories; and retrieval of that stored data into reports, graphical dashboards, multidimensional online analytical processing cubes and other business intelligence outputs.
It is possible to retool data warehouses to support real-time business intelligence. Some data warehousing vendors have begun to address these requirements in their products. Doing so requires that data warehouses -- as enterprises' master data management hubs -- be redesigned to serve also as real-time, application-layer data routers (in the broad sense of that term). For example, NCR Teradata's active data warehousing adds support for near-real-time ETL and data delivery. Just as important, the vendor has added the policy-driven event detection, processing and notification features needed to manage the flow of real-time events between data sources and consumers, as brokered through the data warehouse.
Though organizations are beginning to use active data warehouses for real-time business intelligence, no one is seriously considering deploying them as general-purpose, application-layer routers, because data warehouses usually are deployed in hub-and-spoke configurations and thus can become significant bottlenecks. Some in the industry have proposed data warehouse federation to alleviate the potential bottleneck, but most federation scenarios are fundamentally hub-and-spoke, relying on common ETL tools, metadata repositories and data staging areas.
Fortunately, other architectural approaches for real-time business intelligence are being explored. Some firms deploy an operational data store, which is similar to a data warehouse but contains only the most current consolidated data fed in through ETL tools. Another popular approach is enterprise information integration (EII), which supports real-time, federated query and update across distributed source DBMSs.
Unfortunately, there are no industry best practices for real-time business-intelligence requirements. Companies must sort through diverse approaches and try to implement them to leverage and extend their traditional, data-warehouse-based business intelligence environments.
Going forward, the data-management industry should define a clear set of real-time businessintelligence implementation best practices based on open industry standards, such as the various eventing, metadata and other interoperability specifications subsumed under the WS-* umbrella. Vendors should converge all real-time business intelligence approaches in a common service-oriented architecture framework, so that customers can deploy easily any mix of real-time, near-real-time and lagged-time business intelligence that suits their needs.
The stakes in all this are more than low-level protocol plumbing. Real-time business intelligence enables business agility through improved reporting, analysis and response to changing events. It also supports regulatory compliance. Section 409 of the Sarbanes-Oxley Act specifically mandates that companies perform real-time disclosure of material changes in their financial conditions "on a rapid and current basis."
How's that for a market driver? From a technical standpoint, how companies meet that real-time reporting requirement is entirely up to them.
Kobielus is a principal analyst at Current Analysis in Alexandria, Va. He can be reached at (703) 340-8134 or jkobielus@currentanalysis.com.