Recently, there has been a lot of talk about CEP and the benefits that it can bring to the industry. CEP is the analysis of event data in real-time to generate immediate insight and enable instant response to changing conditions. Likewise, a seemingly similar concept is that of Business Rule Engines. These again work in enterprises and help in triggering actions on the basis of the set of rules defined. Let us try to see where each of these fit in.
What are Events?
Events are generated all around us with every action which happens. Airplane taking off is an event. The butterfly fluttering its wings is an event. The car honking in a traffic jam is an event. There could be multiple seemingly unrelated events which could make sense to a CEP engine. A combination of all these events is passed into a processing engine which does real-time detection of the situation and generates a proactive output. As an example from Wikipedia consider the following events. Church bells ringing, appearance of a man in a tuxedo with a woman in a flowing white gown, and rice flying through the air.
A complex event is what one infers from the simple events: a wedding is happening.
We would go into the details of Complex Event Processing in the next post, but here is a quick summary.
So how is CEP different from a Rules Engine?
- One main difference is that RE are stateless. An input payload is expected for a RE which is processed and output is produced. CEP engines are stateful and can be persisted and in case of failure, recovery is supported as opposed to rules engines.
- Another difference is that RE expect a single payload of data where as CEP can accept data and output data to multiple channels.
- CEP works on real-time event data which is coming into the system where as RE would working on stored data or payload data (now some RE’s support pseudo real-time event data too)
Hence RE should be used where the situation does not need to be stateful and there is no need for either real-time events or time based event correlation. CEP and Business Rules can work together and complement each other.
And a bonus question, How is CEP different from ESP?
ESP is Event Stream Processing. Logically it is very similar to the event processing as in CEP because it is also processing events. But, the difference is that the incoming events are supposed to be a part of the stream and not an event cloud. There’s a fundamental difference between a stream and a cloud. An event stream is a sequence of events ordered by time, such as a stock-market feed. An event cloud is the result of many event generating activities going on at different places in an IT system. A cloud might contain many streams, A stream is a special case of a cloud. CEP is more suited for cloud events and ESP is useful in streams.
Stream can assume the events coming in a particular order and being processed in that order. In a cloud the ordering of events is not ensured. Event stream processing is focused more on high-speed querying of data in streams of events and applying mathematical algorithms to the event data. Stock-market feeds is suited for this type of processing. CEP is focused more on extracting information from clouds of events created in enterprise IT and business systems. CEP includes event data analysis, but places emphasis on patterns of events, and abstracting and simplifying information in the patterns.
CEP has a complex set of patterns for data analysis because it also focuses on the causality of events. Event2 can be casued because of Event1 or they could have occured independently.
Stepping into the future,
As the data volume is growing with each stock ticker update, each ball thrown in the baseball game and each digital photo being clicked, the need for processing events and to predict, forecast and analyse events as they happen becomes a necessity. CEP has a wide array of usages from analyzing a lot of event sources for early prediction of epidemic outbreaks, predicting the stock market movement, forecasting sales and manufacturing for industries etc.
More details on CEP in the next post.
