Proactive Services Data Management in the Age of Hyper-Distribution

1. Summary

This report is aimed at VPs of engineering, CTOs, principal architects, SREs, and heads of operations and development in organizations looking to deliver technology-based innovation at scale.

Across the board today, organizations are looking towards digital transformation; that is, the ability to deliver technology-based innovation at scale. Enterprises are adopting multiple approaches—microservices, serverless, and multi-cloud architectures—alongside best practices such as DevOps. Each approach brings a great deal of potential but also creates headaches for ongoing operations and management.

While innovation teams may benefit from increased flexibility and agility, consequences can be felt in terms of operational visibility and overall business risk. Resulting applications are highly fragmented and dynamic yet still need to be treated as manageable units of function.

Performance data generated by applications, services, and infrastructure is increasing exponentially. The question is not whether such information exists, but how to make the best use of it. Traditional approaches based around siloed data stores impact efficiency and increase risk. Meanwhile, breaking down these siloes, both technically and culturally, creates an opportunity for proactive, predictive operations, driving innovation and transformation.

Figure 1. More Mature DevOps Organizations Make Greater Use of Performance Data

Figure 1 represents data from a GigaOm survey of enterprise technology decision-makers, assessing an organization’s use cases for monitoring performance data and categorizing the results by level of DevOps experience. In general, we can see that more mature organizations make greater use of performance data, across technical and business-facing measures. Performance data is cumulative, with the ultimate goal being the ability to provide the information stakeholders need to do their jobs. For example, monitoring and metrics data can be used to see problematic events or trends, but often the underlying log data needs to be accessed to identify the root cause.

To deliver on this goal, organizations need an approach that brings together centralized performance data management capabilities and a performance improvement culture. This paper looks to help organizations understand the benefits of this approach and what to look for in solutions.