The two types of Event Processing
I was thinking this morning about two types of Event Processing: “Detection Oriented” and “Operations Oriented”. It looks like these are the main categories of Event Processing, both in terms of business value and requirements for a COTS product. Today the Event Processing Technical Society does not make a big deal out of this distinction, but I think that will change.
Detection Oriented Event Processing applications want to locate interesting information in a flow of data. Operations Oriented Event Processing applications want to take an action (which involves making decisions) based on incoming events.
Detection Oriented Event Processing is the Event Processing equivalent of data analysis. It is driven by a need to detect faster than was previously possible and is important to understand that Event Processing adoption is not driven by the need to detect more accurately. More accurate detection requires better detection methods. Detection methods are not part of Event Processing, although Event Processing may make it easier to implement certain methods.
Detection Oriented Event Processing suffers from a significant disadvantage compared to traditional data analysis: the data to be analyzed is not all available at once. Rather, the data arrives incrementally as events.
Detection Oriented applications are developed by data analysts. They require a good understanding of what is being detected and how the events contribute data. They may benefit from statistics, data mining or machine learning. Their challenge is to balance the goal of detecting with their performance requirements. They are tested by running various real-life or simulated data scenarios through the Event Processing Network.
Operations Oriented Event Processing is driven by a need to react faster (with lower latency or higher throughput) than was previously possible. Again we must distinguish faster from “better”. Better reaction requires better decision-making. And better decision-making is not a part of Event Processing, although Event Processing may help make decisions with lower latency or higher throughput.
Operations Oriented Event Processing applications are developed by logic coders. They require a good understanding of how the business should react and what the events mean to the operation of the business. They may benefit from decision management, decision-making under uncertainty or other areas of operations research. Their challenge is to balance the goal of deciding with their performance requirements. They are tested by running specific sequences of events through the Event Processing Network.
The two types of event processing put different requirements on Event Processing software. For example:
Detection Oriented applications will change as the data changes, as the detection methods change or as the detection goals change. It is common for these applications to regularly add more detection logic, while keeping the old logic.
Operations Oriented applications will change as the business optimizes or changes its decision-making strategies. It is more common for these applications to update their decision-making logic and get rid of the old logic.
Of course, every Detection Oriented application will do something with its detection results. And every Operations Oriented application will involve some amount of detection. But the focus of the application remains on one or the other goal.
There are applications that require both Detection Oriented and Operations Oriented Event Processing. These are applications with both complex detection and complex reaction requirements. In my experience, is common for these applications to be cleanly divided between the two goals. This division is often so complete that the two goals are implemented essentially by separate systems and often by separate teams. And when this division is not so complete, everyone usually wishes that it were.
So if these types of Event Processing are separate in terms of their value and their implementation, maybe they should each use separate Event Processing software?