Sector RoadMap: Hadoop platforms 2012

1. Summary

For years technologists have promised us software that will make it easier and cheaper to analyze vast amounts of data in order to revolutionize business. Some of this software has helped. None of it has yet blown anyone’s socks off.

The hottest contender to this throne today is Hadoop, which emerged from Google and Yahoo almost a decade ago. There are now more than half a dozen commercial Hadoop distributions in the market, and almost every enterprise with big data challenges is tinkering with Hadoop software.

Big data refers to any data that doesn’t fit neatly into a traditional database due to three things: its unstructured format, the high speed at which it comes in, or the sheer volume of it. Hadoop, which is open-source software licensed by the Apache Foundation, was designed to store and analyze big data. It aims to lower costs by storing data in chunks across many commodity servers and storage, and it can speed up the analysis of the data by sending many queries to multiple machines at the same time.

Hadoop is certainly not the only game in town for big data, but finding alternatives feels a bit like changing the channel on Super Bowl Sunday: Everything else gets drowned out.

Market forecasters are hyping big data, which is adding to the Hadoop frenzy. IDC has pegged the big data market for technology and services at $16.9 billion by 2015, up from $3.2 billion in 2010. Any market that is growing at 40 percent per year can only be considered a solid one to be a part of. And there’s a hunger on the enterprise side for solutions to manage and extract value from big data, which goes some way toward explaining IDC’s whopping numbers.

This report examines the key disruptive trends shaping the Hadoop platform market, and where companies will position themselves to gain share and increase revenue.