GigaOm Radar for Streaming Data Platformsv4.0

1. Executive Summary

Streaming data platforms ingest, process, transform, analyze, and render action from streaming data in real time. The best tools can do so before data is written to a database, to support use cases requiring low latency. This technology has become imperative because of the influx of machine-generated sensor data, data generated from the Internet, and data continually produced by applications across a plethora of verticals, including customer-facing businesses.

These developments are responsible for the pace of business operations—and the data management requisites for facilitating it—accelerating to real-time processing that belies time-consuming, traditional batch processing. Consequently, streaming data platforms are indispensable to an array of business end users (such as stock traders, portfolio managers, and health care practitioners), IT and operations personnel monitoring telemetry data and other data types, and data scientists building and refining machine learning (ML) models.

Organizations need these solutions because the alternative batch methods of ingesting, transforming, and analyzing data were designed for historical data and conventional business intelligence (BI) use cases. Accomplished vendors in this space couple these batch paradigms–including extract, transform, and load (ETL); extract, load, and transform (ELT); and change data capture (CDC)–with streaming data paradigms designed to expedite meaningful action as indicated by the latest data available for a specific use case. Doing so allows organizations to enrich their streaming data intelligence with historical data to fully understand the latest developments impacting a use case—such as a customer’s most recent behavior affecting a real-time, personalized offer—as well as the appropriate historical data doing the same.

The most significant benefit of adoption is the increased responsiveness to data-driven events that streaming data platforms directly support. This capability manifests itself in predictive analytics, applications of artificial intelligence (AI)—including generative AI—and enhanced customer interactions. The urgency surrounding adoption pertains to the inexorable shift to real time, which is frequently necessary to either supersede or supplement batch processing, for continued capitalization of data-driven investments. The impact of streaming data processing, and its potential to act on data in the moment, is considerable for nearly every use case supported by streaming data, from application observability to the internet of things (IoT) and data science.

This is the third year that GigaOm has reported on the streaming data platform space in the context of our Key Criteria and Radar reports. This report builds on our previous analysis and considers how the market has evolved over the last year.

This GigaOm Key Criteria report highlights the capabilities (table stakes, key features, and emerging features) and nonfunctional requirements (business criteria) for selecting an effective streaming data platform. The companion GigaOm Radar report identifies vendors and products that excel in those decision criteria. Together, these reports provide an overview of the market, identify leading streaming data platforms, and help decision-makers evaluate these solutions so they can make a more informed investment decision.