By Paul Morris
I’ve been loving the Olympics and coupled with an amazingly geeky/ interesting digital data chat last week inspired me to write an article about how the 5 rings have been showcased by Big Data and how in turn Big Data will transform Digital Media.
Over the last 2 weeks it’s been great to see how the data vaults are starting to open (ajar rather than fully open but I digress). Journalism has certainly been transformed as a result, inspired by research such as the McKinsey report, that showcased big data as being the next frontier for innovation, competition and productivity.
Data has been showcased in infographics and interactive infographics (the next frontier as I posted 6 months ago) in amazing formats such as those from the BBC, French medal interactivity, New York Times, Twitter Olympic sentiment analysis (take the sentiment with a pinch of salt but still innovative) and geek-tastic tables from an Argentian stats site. You even have amazing competitions that have stimulated Olympic visualisations that couple data with UX and beautiful design.
So what is the movement that underpins all this Olympics shenanigans? For me it’s Big Data.
Big Data is data that generally exceeds the processing capacity of conventional database systems.
Big Data is still a nightmare to harness due to the difficulty/ time to extract, the fact data has historically been poorly categorised and stored, the cost of harnessing it and then brain power required to analyse/ enable (e.g. new product development) it.
…However as many of these barriers come down (data = entrepreneur/ start up opportunity, cloud computing and reduced processing costs coupled with a recognition that Big Data produces great £££Opportunity) the hope of Big Data utilisation becomes a reality. We only have to look as far as Google (search signal algorithm) and Facebook (user engagement = highly personalised user experience = new advertising proposition) to see how Big Data produces competitive advantage.
So how do we understand and characterise Big Data? Often through the lens of a V.
Well 3 X V’s to be precise = Volume (often a problem of processing large data sets without utilising new solutions such as Greenplum or Apache Hadoop, Velocity (the increasing rate at which data flows, often called ‘streaming data’ or ‘complex event processing’ in to an organisation and showcased nicely in IBM’s commercial as there being the need for a super quick feedback loop) and Variety (diverse data with little neat conventional relational structure = the need to be tidied up with as little data discarded as possible)
I therefore believe Bid Data needs to be accompanied by 5 other Big’s to make it a reality in your organisation:
Big Clean Up, Big System Infrastructure, Big Multi Structuring (of the data itself), Big Forensic Analysis and Big Judgment (please do not look these up on the web as I just made them up!)
So whilst I have demonstrated Big Data usage in the Olympics how do we bring it back to our Digital Media day jobs?
Well to begin with we need to listen to Christer Johnson, IBM’s leader for advanced analytics in North America. He says (I’m paraphrasing) patterns and clues are cool but you first need to ascertain what problem you want to solve. Well here goes…
e.g large data sets that improve ppc; such as emerging PPC environmental factors I blogged about a few months ago could show the way for improved targeting on the fly with large data sets being processed in milliseconds.
e.g. Display following (the PPC) suit by utilising data on the fly by incorporating in to smart versioning/ dynamic creative and sequential targeting.
If we take sequential targeting as an example it would mean not only retargeting down the sales funnel with cookie data but also incorporating persona knowledge utilisation, past brand interaction along the marcoms journey, contextual and behavioural data that would in turn lead to truly amazing engagement, CTR and sales results.
e.g. Start simple with killer interactive infographics that are of the moment and utilise your data in tandem with other data sets that truly engage your audience in real time.
Big Data Future
However the founding brain behind this, the Big G, has now moved on to newer technologies such as Caffeine and in 2010 showcased the amazing Dremel. Dremel is similar to SQL, in a really user friendly way, and allows you to run queries on multiple petabytes (milions of gigabytes) in seconds. You can even try it via Googles BigQuery.
If you are after more information on Big Data Trends then visit the nice presentation from David Feinleib at Forbes (there is a presentation transcript at the bottom of the page).