Predictive analytics, CEP and your local mechanic
Let the indignant comments begin, as a loonie states that “Predictive analytics is a capability based on complex event processing (CEP).” Post is here.
They say that one way to tell that the stock market is over-hyped is when the guys repairing cars down at the local garage are talking about it. The assumption being that your local mechanic doesn’t know the first thing about the stock market and is just quoting some stuff from the news.
Now recently, I’ve been reading blog posts all about predictive analytics coming from people who have clearly never used a predictive analytic in their lives. This is particularly true in the CEP space, where there is so much posting about predictive analytics and machine learning from people who don’t seem to know the first thing about these topics. The above post is a case in point.
So when people who don’t know much about the subject are talking so much about predictive analytics, making statements and assessing their strategic value – I wonder if the public is getting the wrong idea about this stuff. Remember that, to a large degree, over reliance on analytics caused the credit crisis.
Sure analytics are the future, but let’s not jump directly to so many applications. Before you dive into analytics, you need to set a solid foundation for ongoing critical analysis of the effectiveness of your analytics.
A little knowledge about analytics is a dangerous thing. Don’t assume that you can set up some analytics, let them run, and get good data forever. Think before you jump into analytics. And maybe also before you post about them.
Hi Hans… although my understanding of Predictive Analytics is also based on the historic definition (e.g. what tools like SAS, SPSS, our S+ Miner etc provide), if you check the 1st line of the Wikipedia definition at http://en.wikipedia.org/wiki/Predictive_Analysis it says:
“Predictive analytics encompasses a variety of techniques from statistics and data mining that analyze current and historical data to make predictions about future events.”
Arguably, one of those techniques could (and probably is) some use of CEP.
Personally, I prefer the term “event based analytics” for the CEP-based analytics though (e.g. rules determining trends and manipulating predictive scores etc as a result).
Cheers
Paul, maybe CEP bloggers could hire you to edit their posts.
Since you’re so happy to see CEP and analytics appear together, I will guess that you get lots of prospects asking about this connection. I’ll go a step further and guess that there are your favorite kind of prospects: business types who don’t know much about what they’re asking for and just want to get involved with something that will further their career. Maybe I’m wrong about that last part, but it’s certainly the trend I see in posts.
Blogger: “I love analytics, I believe in them and I’m a supporter. Analytics can give you cost savings and strategic impact. I think that analytics are the future of business.”
Me: “Have you ever used analytics of any kind?”
Blogger: “Yes, I took a very good class as part of my MBA program.”
Me: “What color do you prefer your analytics?”
Blogger: “I believe that mauve has the most RAM.”
“Since you’re so happy to see CEP and analytics appear together, I will guess that you get lots of prospects asking about this connection. …”. Actually customers just want efficient solutions – if you could use a goat and cabbage to improve operations, they’d use that.
I’m not sure there is a huge demand for traditional SAS-type analytics though – very specialized skills reliant on expensive data warehouses etc. It will be interesting to see how/if “event based analytics” improve the score, so to speak.
Cheers
[...] 2, 2008 Paul Vincent, of TIBCO (blog is here), has been kind enough to respond to my recent post about the very strange discussion surrounding analytics in Event Processing (EP) [...]
“A little knowledge about analytics is a dangerous thing.”
I agree very much, and in my experience, this is a widespread problem.
“…to a large degree, over reliance on analytics caused the credit crisis.”
I disagree. Modern lending cannot be done based on one’s gut, hence serious predictive analysis is necessary.
Getting back to the previous point, analytics as performed in the financial and insurance worlds is woefully underdeveloped, in this data miner’s opinion. The tools used are antiquated and senior management shows little sign of allowing this function to progress. As I like to say of companies who have recently suffered catastrophically, “They should have hired a better class of statistician.
Hi Will,
I was not clear, of course the lending industry must rely on formal methods and not gut feelings.
I meant that, in part, an over reliance on assumptions that were known to no longer be invalid, caused the problem.
But I think it’s even more complex than this – I ended up writing a new post on the topic, “Lessons on probability from the credit crisis”.
Hans
Correction:
…in part, an over reliance on assumptions that were known to no longer be *valid*, caused the problem.