Compliance, front running and CEP

Here’s an interesting post about front running and why banks don’t have better analytics looking at their trading activity. In the comments, Marc mentions that CEP is good for surveillance.

In the past, banks have made structural changes in response to front running concerns. For example, Prop traders don’t have access to order flow from the rest of the bank or the customers. And traders all over are trained not to give out data that could be used for front running. So the question is valid: why not back this up with analytics?

Well, a bank is only going to enter into a big project for two reasons: profit or cost savings. The structural changes were brought on by penalties, threats of legal action or rules changes or other potential costs. And apparently, those changes worked to stop those costs. So until another round of costs comes up, there’s no major motivation for an innovation in this area of compliance.

Improving detection of front running is actually a big project for the business. Sure, you can use some pretty simple statistics to begin detecting front running. And one can imagine more complex techniques as well. But what happens when you get an alert from the detector? There’s got to be a process for understanding whether it’s a false alarm. And all the desks will need to sign off on that. So there is a huge amount of testing to show the desks what previous trades would have been flagged. And the desks will have to devote significant effort to looking at these results and at the analytics in general. Not a small task, overall. And not cheap.

Also, this kind of project would be a powerful magnet for attracting the interest of regulatory agencies. If those agencies decided to get involved and to compare the analytics between institutions… Ugh.

So again, what’s needed is some new round compliance related costs that would justify a big project like this.

Now about using CEP for this kind of thing. As I posted here, real-time monitoring is not all rosy. This kind of change to compliance rules is a very big project for the business and honestly, if I were running the tech side of such a project, I would not go for real-time monitoring in the initial stage. Detecting front running in real time actually doesn’t provide an enormous benefit. Sure you might find some rogue trader during the day, but so what? One trader front running for one day won’t cost the bank very much. If it’s caught by analytics that night and stopped the next day, regulators will be fine with this. And knowing that your activities will be caught that night is as much of a deterrent as knowing that your activity will be caught in 10 seconds. So the big win comes simply from having the analytics and not from real-time detection.

At the same time, per my post, real-time analytics introduces big risks. So I would see how batch analytics work and then think about identifying opportunities for improvement with real-time processing. I would not start a project like this with real-time processing.

Having said that, though, this is one area in which streaming SQL engines like StreamBase, Coral8 and Esper have a lot to offer. Even if the processing is not done in real-time, streaming SQL engines have features that are not available with traditional SQL. For example, streaming SQL engines offer interesting and high performance pattern detection operators as well as the ability to do analytics over sliding time windows (although Oracle is catching up with the time windows). So even if the compliance project does not go for real-time monitoring, it might be interesting to look at the analytical capabilities of these engines.

Pondering EP engines for simple order matching

This post is about a recent conversation on matching bond orders using an EP engine. The conversation started with a question: how well would an EP engine work for order matching and related services like quoting and trade reporting? This is a somewhat general question and my first instinct was to answer: “you’re best off doing an evaluation to find out.” That answer, though, is not entirely satisfying.

The following question was asked as part of this conversation: What about just the order matching logic, forgetting about any other functionality? Hmmm…

Well, first off, can you really implement the order matching logic and forget about the rest of the functionality? I don’t think that’s practical, but wouldn’t that be an interesting solution: to use an EP product just for matching orders and something completely different to implement the rest of the engine.

I’ve come to the conclusion that just asking this question should throw up a big, glaring red flag. Asking this question means that you, or someone you know, can picture a path to implement some particular piece of logic in the system (the order matching logic). But when you try to picture how to implement the whole rest of the matching engine, things become fuzzy.

I suggest that it would not be a good scenario to find that you have coded your order matching logic in an EP engine, but that engine makes it very hard to implement the rest of what you need. Order matching, even with the many variant features like combinatorial matching, is not so complex as to justify an entire software package just to simplify the code. If you’re going to use an EP engine, I suggest that you want it to do most or all of the job. Adding a little Java or C++ code here and there won’t kill you, but if most of your project’s won’t be done using that EP engine, then you should seriously ask yourself whether the engine will save more money than it costs.

This leaves the question open: which of the vendors on the market today would be a good choice for an order matching engine? If I had to pick a product at gunpoint, I would pick Apama. Their language is the most general and I know that their product would provide value to the project while having the least chance of imposing an unforeseen constraint that would force one to write a significant portion of the logic in Java or C++. Of course, their language could easily be mistaken for Java or C++ and is missing features that might be available with streaming SQL or a rules engine.

In terms of streaming SQL or a rules-engine EPL, though, I have to recommend a more thorough evaluation before deciding on one of these products for an order matching project. Both of these types of EPL have particular features that make them highly suited for particular kinds of EP logic. But at the same time, the languages impose restrictions and it’s very important to understand whether you’ll spend half of your project working around language restrictions.

I’ve devoted several blog posts (here and here) to defending streaming SQL, so you can be sure that I like many of the features that this approach to EP has to offer. Unfortunately, it seems that at this stage in the development of EP products, we can’t have it all: there is no product yet that offers both the flexibility of a traditional programming language and the full feature set of streaming SQL. One day, maybe.