This maturation process in the data world will involve developments that are less exciting than those we have seen over the last few years. But while 2015 won’t be a breakthrough year, it certainly will be a year of the much-needed building of credibility for newer data and analytics technologies. As such, it will be a year where these technologies become more standardized, more broadly adopted, and more successful. And for vendors in the space, it may be a more lucrative year than those which have preceded it.

Hadoop will become more:

1. Usable by end-users as it transitions to being more embedded and less-directly exposed
2. Enterprise-adoptable through such additions as better tooling and true role-based access controls (RBAC)
3. Developer-friendly through the addition of tools and comprehensive APIs

Not coincidentally, the third point will help advance the first two.

NoSQL databases will mature as well. In fact, so will relational databases. Each will continue to acquire qualities of the other, with a likely outcome that the two categories will converge.

This report will examine the following likely trends affecting the analytics space in 2015:

• Data governance will become a higher priority, and the conundrum of enforcing governance over ever-growing, real-time streaming data sets will have to be confronted. Current approaches to governance, based on years of data warehousing and OLAP work, won’t be sufficient to the task, and continuing to avoid the governance question won’t work either. New thinking will be required.
• Hadoop will morph from an infrastructural entity that users must care for as they work with it to a running service that they can use in a task-oriented way. Discrete components, like Hive and Pig, will continue to be useful in their own right but will become constituent services that higher-level, cloud-based self-service platforms will leverage for their own functionality.
• In much the same way that new user platforms will make Hadoop more self-service, so too will new developer platforms make Hadoop more developer-friendly and more embeddable in line-of-business applications. But proper evangelism and mentoring will be required to get enterprise developers over the big data “hump.”
• Relational databases will become more adept at handling semi-structured data. Such capability will need to extend beyond support for XML and JSON as datatypes to accommodating schema-less tables alongside conventional ones, in the same databases and the same SQL queries. Likewise, today’s NoSQL databases may need to allow schema-based tables, albeit within the key-value, column family, document store, or graph storage formats that they use today.

Thunbmail image courtesy of nadia/iStock.

The post Outlook: Big data and analytics in 2015 appeared first on Gigaom.

As open source data project momentum increases, the value add of commercial offerings that integrate with, abstract away, and simplify those underlying technologies has grown. Incumbent vendors are moving as quickly as they can to adjust to the changes. NewSQL and NoSQL vendors are carving out niches for themselves as they move beyond the market entry stage.

This report will examine the following areas of activity from the third quarter and their impact on the near term outlook in the analytics space:

• The continued growth of YARN, combined with increased momentum for Spark, leads a banner quarter in new releases and shorter release cycles for numerous Apache Software Foundation projects.
• Incumbent vendors made huge strides in the big data arena, with Teradata making acquisitions, Oracle announcing new technologies, new releases and new cloud services, and Microsoft coming to market with three important new releases of its own. Meanwhile, IBM has doubled-, nay tripled-down on Watson.
• Several data discovery vendors, which Gigaom Research reviewed in a Sector Roadmap earlier this year, have come out with new releases and previews that range from evolutionary to innovative to game-changing.
• Meanwhile, horizontal competitors in the NewSQL and NoSQL fields have started to go vertical, as standing out amongst the crowd starts to become a major imperative.

Thumbnail image courtesy: iStock/Thinkstock

The post Big data and analytics third quarter 2014: analysis and outlook appeared first on Gigaom.

Thus is the paradox of the data analytics market right now. New techniques and technologies emerge, but things are nonetheless becoming more integrated. This extends to companies too as the venture capital dollars continue to flow in, but at the same time, a few acquisitions presage a consolidation for the industry.

This report will examine the following areas of activity this quarter and their impact on the near-term outlook in the analytics space:

• Funding rounds were closed by all three major Hadoop vendors, and there were also smaller rounds by companies in the in-memory analytics and business intelligence spheres. Vendors are still going back to the well, and self-sufficiency is a ways off.
• Several acquisitions, also spanning the field of Hadoop players and business intelligence competitors, took place this quarter. Consolidation is starting to take root.
• Each of the three major public cloud providers have either shored up analytics offerings or announced entirely new ones. The cloud, as a data analytics platform, may be belatedly blooming, with customers’ trepidation subsiding.
• Apache Spark and YARN, two open source projects that are at once different and yet competitive, reached critical mass in vendor support. The fundamental Hadoop stack is changing, and MapReduce’s importance is waning.
• Streaming data processing and machine learning are now on-radar for the broader data analytics marketplace, making this one of the next frontiers for big data and a focus area for vendors.
• Suddenly several transactional databases are integrating analytical capabilities in a quest to serve in mixed-workload capacities. The sprawl of data across multiple databases may be coming to an end.

The post Big data and analytics second-quarter 2014: analysis and outlook appeared first on Gigaom.

• Ad tech has exacting requirements that push the analytics envelope. It is heavily reliant on precision and real-time and predictive analytics, with high data volume workloads and demanding SLAs.
• Scale-out architectures and in-memory technology featured in newer database products work well for ad-tech work and other analytics workloads with high-volume, high-velocity data.
• Meanwhile sacrificing the relational data model, transactional consistency, and SQL compatibility of conventional databases brings risks, productivity hits, and staffing challenges.
• NewSQL databases combine facets of different database types such that newer technologies and older standards coexist for maximum benefit.

The post How to solve big data challenges in the ad-tech industry appeared first on Gigaom.