Big Data represents the next not-completely-understood got-to-have strategy. This first dawned on me about a year ago and has continued to become clearer as the phenomenon has gained momentum. Contributing to Big Data-mania is Hadoop, today’s weapon of choice in the taming and harnessing of mountains of unstructured data, a project that has its own immense gravitational pull of celebrity.
So what
But what is the value of slogging through these mountains of data? In a recent Forrester blog, Brian Hopkins lays it out very simply:
We estimate that firms effectively utilize less than 5% of available data. Why so little? The rest is simply too expensive to deal with. Big data is new because it lets firms affordably dip into that other 95%. If two companies use data with the same effectiveness but one can handle 15% of available data and one is stuck at 5%, who do you think will win?
The only problem is that while unstructured data (email, clickstream data, photos, web logs, etc.) makes up the vast majority of today’s data, the majority of the incumbent data solutions aren’t designed to handle it. So what do you do?
Deal with it
Hadoop, which I mentioned above, is your first line of offense when attacking big data. Hadoop is an open source highly scalable compute and storage platform. It can be used to collect, tidy up and store boatloads of structure and unstructured data. In the case of enterprises it can be combined with a data warehouse and then linked to analytics (in the case web companies they forgo the warehouse).
And speaking of web companies Hopkins explains
Google, Yahoo, and Facebook used big data to deal with web scale search, content relevance, and social connections, and we see what happened to those markets. If you are not thinking about how to leverage big data to get the value from the other 95%, your competition is.
So will Big Data truly displace Cloud as the current must-have buzz-tastic phenomenon in IT? I’m thinking in many circles it will. While less of a tectonic shift, Big Data’s more “modest” goals and concrete application make it easier to draw a direct line between effort and business return. This in turn will drive greater interest, tire kicking and then implementation. But I wouldn’t kick the tires for too long for as the web players have learned, Big Data is a mountain of straw just waiting to be spun into gold.
Extra-credit reading:
- Now available: Dell | Cloudera solution for Apache Hadoop
- Microsoft climbs onto Hadoop bandwagon – ComputerWorld
- Big Data Will Help Shape Your Market’s Next Big Winners – Brian Hopkins, Forrester
- Big Data: Sorting Reality From the Hype – NYT Bits
Pau for now…
Barton's Blog 