Observability in the Development Pipeline

1. Setup and Benefits Summary

After very fast growth, collaboration tools provider Slack found its application testing teams weighed down under increasingly complex infrastructure, impacting test reliability. Up to 50% of tests were deemed inconclusive—or, in the parlance, flaky—which meant development teams could no longer rely on the outputs of the testing process.

When Slack brought its observability solution to bear on the infrastructure used for software development and testing, the company found it an effective way to pinpoint, diagnose, and solve individual infrastructure issues one by one. But, given the lack of trust in testing results, gaining buy-in across multiple teams was a challenge. This was addressed following a results-driven, show-don’t-tell approach that involved the teams in decision-making.

Overall benefits of deploying observability pre-deployment were:

• A reduction of flaky test results from 50% to 5%
• A removal of 45,000 hours of wasted server time per week
• Increased developer confidence and productivity