Big Data and a Big Dinner

The big event for me last week was introductory training in TwinFin i-Class. What is does is explained in the link, what was most interesting for me was the discussion at dinner afterwards. I’m still learning the technology and the products, but the business case for what we’re doing just seemed so obvious to me right from the start. It starts with, but goes way beyond what Netezza insiders call the ‘speed and feed’ story. Speed and feed just means feed the Netezza box your data and it will deliver greater speed than your existing warehouse infrastructure. The key question then is what do you do with all that speed?

What we’re seeing is a world where more and more data gets collected from more and more sources and we’re seeing our customers thinking about some very smart things to do with that data. One example is retailers tying point-of-sale data to customer loyalty cards or branded payment cards, so they can analyse individual spending patterns - scary or cool, depending on how well they turn that knowledge into really helping me with individualized advertising.

For sure, i’ve had enough of so-called eCommerce leaders spamming me with mail. Apparently there’s one global bookseller that thinks i’m interested in fantasy, gardening, learning Italian and technology. A little better analysis of the data they have about me would tell them that i only buy Italian, gardening & fantasy books in November, December or around dates that represent family birthdays. They could even work those dates out if they traced my purchases to the wishlists i pick from. Now it doesn’t matter, i can unsubscribe or i can just delete the mails unread. But in that case we’re both losers and it makes me cross to see the knowledge represented by that data go to waste. And why should it? It’s not expensive or rocket engineering to unify wishlists, purchase records, products and categories and run some simple analytics, though it might need some horsepower and a lot of historical data available.

I hear a lot of discussion about the intelligent economy, and i’m a big believer. But sometimes i think the development of the information economy it’s a bit like the development of the human being. As babies we have all the sense data of an adult but at first we just use it for basic purposes – your hand jumps back from something sharp and your brain registers distress. Later we get smarter and remember that data so we resist the urge to poke the cat in the eye. Later still we learn to coordinate so we can decide to stroke the cat instead. So we use the same data for increasingly sophisticated processes as we retain the data and analyse how we want to use it. And unlike humans, organizations can decide how fast they want to develop. How well they want to exploit those data resources.

There’s always more data you might collect and keep, but i’m thinking that right now the challenge is more about analytics. Our customers , telcos, retailers and others are pushing on hard with smarter ways to use their data. That’s why we’re focussing on moving analytical capability into the database with the i-Class, trying keep ahead of the curve as customers really start to ramp up their deep analytics. I also saw a presentation from one of our partners last week (TEOCO) who provide specific high-value applications for telcos. Exactly the same trend - doing smarter things with more data.