jump to navigation

What is Complex Event Processing (CEP) March 25, 2012

Posted by Mich Talebzadeh in Complex Event Processing.
trackback

Those who follow the general trends in the industry may recall the advent of Complex Event Processing (CEP). In a nutshell CEP involves the continuous processing and analysis of high-volume, high-speed data streams from inside and outside of an enterprise to detect business-critical issues as they happen in real time.

Contrast this to the traditional processes involving database systems, which provide delayed analysis. An example of CEP would be real-time financial market data analysis and decision process allowing traders or anyone else to make a decision on the spot based on real time data. Prime example is a Forex System where on the basis of certain indicators (say moving averages for the past 14 periods) you make a decision to buy or sell.

As an example Sybase got into CEP through acquisition, rolling up 2 of the leading independent CEP providers: Aleri and Coral8. As such Sybase now has  two CEP products. These are Sybase Aleri Streaming Platform, and Sybase CEP. The latter is based I believe on Coral8 product. Later on these two products may be integrated to one as one product. Other vendors like Oracle, much like IMDBs have their own pet CEPs.

CEP software offers two major components: a high-level language for programmers to easily describe how to process the streams/messages, and an infrastructure engine for processing and analyzing high-volume data streams. Although CEP software performs different functions, the component structure is somehow analogous to database software, where there is a language (SQL) and an engine (the database server). The objectives of CEP is to get the product and save on development cycle traditionally done by in-house developers.

I believe that in the next year or so CEP will be a product that will be considered by many shops and hence there will be inevitable infra-structure/DBA involvement. Both Sybase CEP products integrate with ASE, IQ and RAP, but they can be deployed on their own. They do not require the use of a database – both have their own purpose-build in-memory data manager as well as disk-based storage manager for (optional) data recovery capability. But when integrated with an external DB (from Sybase or other), the DB can be a source of data into the CEP engine, can be integrated “in line” with the CEP engine calling out to the DB to lookup info, and can be a repository for output.

An interesting book on the subject is “The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems” by David Luckham (ISBN 978-0201727890).

Comments»

1. Ashok Kumar - March 27, 2012

Thanks Mich. Very informative


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: