Event Processing example: Foot Traffic Management

This post continues the idea that Event Processing (EP) is more easily understood after dividing the field into two distinct areas. I started  with a couple of posts using this perspective to analyzing the business value of EP. Those posts are available from my EP reference page.

The most common request since then is to better explain the Two Types of EP concept. To that end, I first posted an analysis of two blog posts using the Two Types of EP.

Now I will introduce an Event Processing use case and then analyze it from the perspectives of Detection Oriented vs. Operations Oriented Event Processing. This is not a use case from an implemented project, but it’s plausible and a good example. Real use cases are currently hard to analyze because no one is willing to publish a detailed description of their use of Event Processing.

My background is in statistics, not marketing, and there are flaws in the marketing side of this use case. Anyone who has really looked at this particular marketing scenario will find many problems and questions to be addressed before the idea could be considered for implementation.

The use case: Let’s say that a chain of coffee shops wants to boost its work-hours sales to the urban business crowd. They have many chain stores in urban business centers and they want strategies to increase sales during normal work hours.

The chain has two large data sets available in-house to drive this effort:

• the (electronic) register tape  from every store
• data captured by a retail traffic counter (Google the term if it’s unfamiliar), which counts foot traffic entering and leaving each store

Let’s say that analyzing this data using basic queuing theory, produces the following results (this is not real data, I’m making it up to suit the example):

• Even during “peak times”, the length of the lines at most stores fluctuates greatly during any 30 minute period. There are 10 minute periods with very long lines followed by 10 minute periods with short lines.
• During peak periods, customers seem put off by long lines
  • They may walk out before making it to the register
  • Foot traffic may tail off when the line gets too long, and it seems likely that interested customers see the long line and never enter the store.

The coffee chain sees a group of customers that would like to make  a purchase, but are put off by long lines. Some of these customers may  go to another of the chain’s stores, but many won’t because there are now so many choices for the urban working crowd.

To sell to these impatient customers, the chain can come up with some traditional marketing ideas:

• They can open more shops in the area. Now the customers can spill over from a crowded chain shop into a less crowded one. But it may not be practical to open a new store just to capture peak traffic spillover.
• They can offer price incentives (discounts) during off-peak times. But this has a very negative effect. If the incentives are low, customers pay no attention. If the incentives are high, the result is to move the peak time but lower profit.

Since neither of these ideas works, let’s find a different and creative way to sell to these impatient customers.

We want to help customers avoid lines. This is a valuable service, but doesn’t have negative profit impact of price incentives. Maybe we can do this in a way that will “smooth out” the foot traffic to each store. In other words, try to get customers to pick just the right moment when the store has a shorter line, rather than a longer line. We know that even during peak selling times, a customer arriving at any random time has a chance to find either a long or a short line – can we direct some customers to the short lines?

To summarize, we would like to:

• Help interested customers to find a store with a shorter line
• Help interested customers who would like some coffee, but are willing to come at any point within a half hour window, to find the best time to arrive at a store
• Attract undecided customers by giving them the confidence and convenience of quickly finding a store with a short line
• As much as possible, prevent the very longest lines and wait times that discourage some customers

We will be creating some kind information service to direct customers, as much as possible, to shorter lines. That information might be delivered on a query basis or through push notifications.

Since this concept relies on customers believing in the system, we need to be reasonably accurate with any help we give. We don’t need to be perfect, but if customers don’t believe the information, they will quickly abandon the service. And this is where Event Processing comes in. Without some element of on-the-fly data analysis, there is no way to give good advice on line length. Because line length at any particular time relies on a very random effect of foot traffic, it’s not enough to consider seasonality (even combining annual, monthly, weekly and intra-day components of seasonality) combined with day-to-day time series. The extra step of real-time data analysis is required.

The meat of this use case will have to wait for a future post. But here are the roles of each type of EP.

Detection Oriented Event Processing

• Inputs are real-time information from registers and foot traffic counters
• Output events an estimate of the current line length and wait time
• Outputs short term prediction of line length and wait time by combining real-time data, with stored data that results from analyzed historical data:  multiple components of seasonality and day-to-day time series
• Considers the effects of this marketing campaign on predictions

Operations Oriented Event Processing

• Inputs are events containing line length and wait time info (current and predicted) from Detection Oriented EP components
• Integrate this information with the marketing strategy
  • Stores data for access by query-based systems like web or smart phone apps
  • Distribute the proper alerts and information to interested parties (customers, managers, etc)
• Interacts with customers, so is part of the customer relationship as well

We’ll see that it is interesting to look at this use case through the lens of Detection vs. Operations Oriented components is for a few reasons:

• The accuracy or effectiveness of the components are measured differently. The detection might work perfectly, but the operational rules make poor use of the data.
• The components are tightly integrated to the value of the overall concept, but still the business value of the two types can be viewed independently. The detection logic provides a data source that can be valuable to customers. The operations logic drives foot traffic and even becomes a part of the customer’s relationship with the brand.
• The development process is different. Developing the detection portion involves tweaking an algorithm by back-testing using large amounts of historical data. Development of the operational portion involves planning out a business strategy and analyzing the impact of various decisions, but no historical data is required to test the logic.
• The logic itself is different. One is focussed on analyzing raw data, the other on making business decisions.
• The maintenance and evolution over time will be different. The operational decision making will evolve over time, it might grow to consider (or at least not interfere with) many components of the marketing strategy. The detection logic will evolve if the model for foot traffic and line prediction changes, but the chain may find additional information that can be mined from the same real-time data sources.