GigaOm Radar for Data Observabilityv1.0

1. Summary

As an enterprise discipline, data observability provides prolonged, up-to-the-moment status updates about the overall health of an organization’s data. It reinforces data health so data is available, accurate, comprehensible, governable, and ultimately usable for the numerous data-centric applications, operational systems, and organizations that depend on it.

Data observability is critical for countering, if not eliminating, data downtime, in which the results of analytics or the performance of applications are compromised because of unhealthy, inaccurate data. Well-implemented solutions in this space are also useful for detecting and mitigating the effects of data drift, which occurs, for example, when data powering predictive models in production begins varying in shape or statistical profile from that in training settings, skewing data science efforts. Downstream impacts of these and other effects of poor data health include inordinate delays, difficulty complying with regulations, churn, and increased litigation and penalties.

As their names suggest, there are obvious similarities between data observability and classic observability, the latter of which is closer to application performance monitoring. However, data observability is almost solely focused on the state of data itself, as opposed to the logs, traces, and metric information that is central to observability about applications and systems.

That said, it’s worth noting that many developments fueling innovation in data observability seem to borrow constructs from the classic form—yet tailor them to a data-driven focus. There are a wealth of AI techniques that can infer the state of data, monitor it, deliver alerts about issues, and remediate problems. One of the benefits of data observability vendors is they employ these and other measures for data whether it’s at rest, in-motion, behind enterprise firewalls, or anywhere in hybrid and multicloud environments.

Of additional value is the fact that data observability applies to data at a granular level. This discipline supports inferencing, monitoring, alerting, and corrective capabilities for schema particulars and the data quality dimensions of completeness, uniqueness, timeliness, validity, and others. These mechanisms are equally adept for data in pipelines or repositories, and they rely on cutting-edge dashboards and visualizations that show the state of data in ways that are understandable even to nontechnical users. This area also utilizes both predictive capabilities and detailed root cause analysis via data lineage constructs that reveal where breakdowns occurred–and how to prevent them from reoccurring.

This GigaOm Radar report highlights key data observability vendors and equips IT decision-makers with the information needed to select the best fit for their business and use case requirements. In the corresponding GigaOm report “Key Criteria for Evaluating Data Observability Solutions,” we describe in more detail the capabilities and metrics that are used to evaluate vendors in this market.

All solutions included in this Radar report meet the following table stakes—capabilities widely adopted and well implemented in the sector:

• Data profiling
• Dashboards and data visualizations
• Analytics
• Data lineage
• Continuous monitoring and alerting