It's no secret to IT people, or any business executive worth his Beemer, that an amazing wealth of actionable business intelligence is coursing through enterprise applications, databases, and even system logs nearly every moment of every day. The problem has not only been plucking the meaningful events from the unimportant ones but also finding the often seemingly unrelated patterns between them, and doing so before it's too late to make a difference -- before the supplier raises the price, the shopper leaves the Web site, or the scammer transfers the funds.
CEP (complex event processing) promises to help you capitalize on the rich information hidden in your data streams -- information typically tossed away in log files and data warehouses. CEP helps users correlate complex events (often seemingly unconnected ones), spot trends, trap anomalies, and trigger alerts that can be used to seize opportunities or mitigate risks.
Tapping into distributed message buses to unearth important combinations and patterns of events that span multiple, disparate systems and lengthy, dynamic business processes, CEP meets challenges that traditional analysis and automation tools are inadequately suited to address. As a result, CEP is catching on in industries such as banking and insurance, where early adopters are using it to detect fraud and compliance violations. Other promising applications of CEP are RFID and CRM, where the objects, actions, and rules for handling them are constantly changing.
The science behind CEP is the ability to create high-level "business" events from numerous low-level system events. CEP events are created by filtering real-time data (from middleware, applications, databases, and Web services, for example) and infusing it with defining detail such as dependencies or causal relationships discovered by correlating other events.
For example, you could construct a CRM event from an "abandoned shopping cart" message (parsing out the transaction, customer ID, and time), using other filters to extract the value of goods in the cart and tapping the correlation capabilities of the CEP system to add causal indicators such as whether the commerce site was suffering performance problems. Your CRM event might also include customer value or rank from the customer database.
CEP solutions don't discover meaningful events and patterns on their own. You need to define what you're looking for. One of the biggest challenges for CEP vendors has been to develop an EPL (event processing language) that is lightweight enough to run in real time, powerful enough to correlate complex data and manage anomalies, and yet easy enough to be used by business analysts.
Regardless of whether the EPL is accessible to humans, the ongoing participation of business analysts is key to the success of any CEP project. Configuring meaningful CEP rules and filters -- and adapting them to evolving business conditions -- will require a deep understanding of the underlying business process.
If you get it right, CEP can be far more flexible, responsive, and cost-effective than using traditional BAM (business activity monitoring) and BPM solutions. CEP doesn't require ripping out or recoding existing systems, and it is actually best applied in modest increments -- allowing for frequent and easy comparison of ROI metrics to rule sets.
Following is a look at products from two pioneers offering business-grade CEP applications. AptSoft Director for CEP 4.0 provides an infrastructure for CEP well-suited to apps such as fraud detection and CRM, whereas iSpheres Event Server 5.2 focuses on financial apps, RFID, and CRM. Both are doing great work applying CEP to their respective areas of expertise.