Big data: verbatim

This is a somewhat lengthy annotation that relates to the “Big data: should marketing really care?” post.

I have the tremendous privilege of having one of the most diverse and responsive Twitter following. I thought it only fitting to cite their verbatim input, as I have left out some content, which may offer better context to the quotes I’ve chosen to cite on said post.

Thank you again to the individuals below. You rock.

So I asked everyone, this one  question:

What in your opinion is the significance (if any) of the big data trend to marketers?

I may not be the best person to ask because, personally, I think the term is as meaningless as web 2.0.  Moreover, big data has been with us since the emergence of the commercial net in 1995 (and of course, before that too), and we’ve always analyzed large datasets both in academia and industry since then.  It’s really more like sensemaking than anything else because once we analyze big data, we extract it from its context, so we can only have very little to say about its structure and semantics.  As a result, for marketing purposes, it’s only as good as polling, except that polling measures people’s attitudes and has longitudinal limitations, whereas big data can enable you to see past and current behavior and track it in real time to see dynamic changes. But as you know, predicting future behavior on the basis of past behavior requires a set of assumptions about a static world that doesn’t exist in practice.  So to make a long story short, if you’re a marketeer, and you want to see shifts in the opinions and sentiments of your consumers in real time, big data is a good tool.   e.g.,

As with many concepts, “big data” means something different depending on whom you’re talking with. If you’re talking to an IT storage admin, big data really impacts their life in handling all of the terabytes created in today’s world. If you’re talking to a marketer, big data isn’t so much the IT storage required but the analytics they can glean from the data found within. Soon, IT storage and analytics software will be so sophisticated that marketers can make *measured* decisions unlike ever before. For example, a chain retailer handles millions of customer transactions per day, manifesting really big data — multiple petabytes. Imagine analytics software that could help yield mind blowing insights from those petabytes. what products, when, how, where. today, firms like dunnhumby analyze this data for huge retailers/brands. but, soon, such b.d. insights will be avail. to the little guy too.

I do believe that Big Data will be the hottest buzzword of 2012 (dare I say hotter than cloud computing).

From the product and solution standpoint, marketers from those companies will have a field day as companies scramble to understand all the offerings and get pressure from their executives to understand their particular big data.  These marketers should and will ride the wave while they can because just like all hot trends will slow down as companies become more educated to what the real opportunity is and what they really need ( or do not need) to leverage their big data.

The other side of this coin is how well marketers will be able to leverage all this new intelligence from their customers as well as potentially their competition’s customers.  Having access to this level of  information has to be a differentiator if companies are able to not only process and manage it but  more importantly incorporate it
into really strong processes such as their

  • Pricing strategies and actions
  • New product development
  • Product exit strategies

If these processes are not solid, all this additional intelligence will likely overwhelm marketers, create confusion and do more harm than good.

This is an extremely nuanced question, but the gist is …

Big Data is a Big Deal for Marketeers
Never before has so much information been made available on such a broad scale basis. The smart marketeers will slice and dice this information along so many pivot points that it will make a traditional maketeers head spin.

Think about it. Internally generated information is growing at an exponential pace. The same is true with every organization. One idea behind Big Data is that these internal and external data stores can be combined – or at least cross-searched and indexed. Advances in traditional Business Intelligence capabilities is driving Big Data. BI is not the only thing that is driving Big Data, but many of the tools and techniques that have been developed to create Data Marts and Data Cubes are being using to parse and process the ever growing repositories of Big Information … Or Big Data.

Here are a few data points to consider:

  • Data analysis: Your Data, My Data, Their Data. If you have access to the data the possibilities are endless on how to slice and dice it. (There are some ethical and perhaps legal implications to consider).
  • On and Off Premise: where the data is located is not as important as that the access to the data is enabled.
  • Real time monitoring: I think a fee structure (I.e. Way to monetize access and timeliness) will be imposed to provide access in real-time to the highest bidders. Some people don’t need access in real time and should not have to pay a premium.
  • Complex event processing: advanced analysis in a myriad of ways across numerous pivot points and spanning multiple data repositories.

Forgive me, but I haven’t been paying enough attention to the big data trend of the last couple of years. 🙂

If we’re talking about the rapid growth in data volume, the demand for large-scale databases to hold all the data, and the tools+talent required to make sense of everything – great! If not, I’ve missed the point entirely.

So, assuming the former, here are my thoughts:

I worry that “big data” will distract marketers. I worry that businesses will rely too heavily on big data to find insights and opportunities and answers to all of their problems. I worry that the big data trend will be used as a scapegoat, with lines like “according to the data” being used as an excuse for when things go wrong.

I worry that the big data trend will throw too much weight behind database marketing at the expense of other marketing strategies, and the pitfalls of database marketing will be amplified.

Don’t get me wrong – I think that having access to this much information is great. But we need to maintain a balance. There are things that marketers can’t glean from databases.

Unless you’re analyzing traffic patterns for Facebook or looking to make sense of the entire Twitter fire hose, most marketers today don’t deal with data sets that qualify as “big data.” That being said, with online media, the number of data points is growing exponentially, so I fully expect that in time, marketers need to be equipped to deal with big data. I see this as an opportunity for technology start-ups to build software-based analysis tools for marketers to help them make sense of relevant [big] data.


Generally speaking, big data has enormous implications on our ability as marketers to deliver relevant messages on the web. And from a B2B perspective, it enables us to measure ROI. We can use big data to help others understand the effectiveness of those messages. There are a bunch of us at big tech companies who analyze the data to tell business stories.

I can’t speak as an official representative of my company. But at Yahoo! we see the influence and impact of big data every day. Our cloud, for instance, has grown to more than 200 petabytes and grows by about 50 terabytes a day (to put that in perspective, the Library of Congress has about 200 terabytes of data—we get that every four days). It’s one of the largest data infrastructures in the world. And that big data allows us to serve really relevant content and ad experiences consumers. We have this thing called a Core Optimization Relevance Engine (CORE). It’s able to personalize 45,000 variations of the homepage every five minutes—that’s 13 million in a single day. And this particular kind of science drives huge clickthroughs for our content. So big data has huge impact on our ability to market and sell. You can actually see CORE at work here:

There are two caveats on this, though:

1) How we use big data will depend in part on legislation. There are still big questions the U.S. government is interested in answering: Who owns the data? So marketers’ potential to use the big data will naturally be impacted by the laws that end up governing it.

2) The ability to use big data is only as good as your ability to synthesize it. So it naturally it depends on the tools you have or create to leverage the content. The future is going to be more than web behavior and demographics. We’ll see the interest graph in addition to the social graph and all that.

Regarding our discussion on big #data #trends see this fascinating video by @byronreese (at the 12:12 mark) – Angela, I was at a loss to find a proper reference for your input. Nonetheless, I do appreciate you offering a source 😉

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s