Scaling Testing Across the DevOps Pipeline

1. Summary

All companies are software companies, goes the adage. While definitions of digital transformation may vary, enterprise boards agree that digital technologies unlock new opportunities for closer customer engagement and new business models. This being the case, time to ROI has become an essential metric; if you cannot develop something as fast as the competition, then you will lose out. The added complication is that nobody knows exactly what customers want; therefore, giving people the ability to try new things has become more important than delivering a gold standard.

Building on the notion of continuous integration and delivery, DevOps responds to this need for agility and speed. For many organizations, however, software testing practices contradict these goals. Software quality is as important today as it ever was. However, the way in which software is tested needs to evolve to fit with innovation practice.

In this report, we look at how quality and testing can become an integral element of DevOps, starting and ending with customer needs and covering every stage along the way. We review how software quality and testing practice is evolving to meet the needs of innovation as characterized by DevOps: an evolution we are calling DevQualOps.

Key findings of this report are:

• Business pressure to deliver new software functionality is forcing legacy testing practices onto the back burner, increasing cost and risk.
• Modern cloud-native applications, business speed, agility drivers, and DevOps approaches to delivery and operations influence how testing needs to take place.
• The rationale for software testing remains the same whether an application is developed quickly or slowly, by a line of business or IT, customer-facing or not.
• DevQualOps encompasses activities that assure an appropriate level of quality across the DevOps lifecycle, across software testing, environment delivery, collaborative planning, and reporting.
• Automation, enablement, visibility, collaboration, and control are key to delivering DevQualOps across the application estate.
• An organization’s ability to integrate testing successfully into the development and operations lifecycle can be seen as a symptom of DevOps maturity.
• A “start, measure, manage” approach enables the organization to break from “anything-goes” approaches and start the journey towardsDevQualOps maturity.
• The ability to measure quality is the ultimate test for gauging how well quality investments are benefiting the business and its customers.

Overall, we review how to turn application testing into a catalyst for innovation and value creation rather than being perceived as a constraint.