GigaOm Radar for API Functional Automated Testingv2.01

1. Executive Summary

Functional testing has been a standard part of application development for a very long time. However, traditional functional methods can’t be applied to many current projects or products, which are increasingly composed solely of APIs performing specific functions or accessing specific data, because that testing is based on the use of UIs or subroutines as entry points, neither of which can work for APIs.

Automation has long been key to the adoption of testing software. Slowing deployments while a test team runs testing manually could never be an option for most busy IT departments. Such constraints became even more of a burden as agile development and DevOps increased the cadence of software delivery and, by extension, increased the burden of testing. Fortunately, now there are tools that can run test suites automatically, filter results without user input, and offer results that are manageable and actionable for both testing and development teams.

The growth in both API usage and DevOps drove the need for automated API functional testing. With increasing numbers of applications turned out at an ever-quicker rate, the need for quality assurance of all kinds is also growing. The best tools evaluated in this analysis can offer testing, support for shifting testing and resolutions left, and functionality unique to API testing.

An API client that facilitates test development, for example, is relatively standard in these tools but not necessary in other functional testing toolsets. This feature can help developers and testers to develop tests interactively and save those tests for repeatable testing.

Tools in this market facilitate test-driven development (TDD) for organizations wishing to use this methodology. The tools enable developers to use the API contract as the starting point for tests, then add test functionality as the project progresses and the code behind the contract is developed. While a full suite of functional testing tools is not necessary to perform TDD, they are useful tools to have.

Our reasoning about what makes a product in this space innovative, and what constitutes a platform, has changed a bit since our last iteration. Our use of the “Maturity” designation more heavily relies upon release cadences and product stability than in the last iteration. Likewise, nearly every tool in this analysis uses AI at this point, so we’ve reduced its bearing on the “Innovation” rating. On the other axis, declaring a group of products a platform does not make it so, and we were more discerning as to what we accepted as a true Platform Play.

Application quality is important to all organizations, and today, applications are often built from a collection of API calls with user interfaces overlaid to enable them on different targets. API functional automated test tools help ensure the quality of the underlying APIs and to increase stability and quality of the application itself, regardless of where it is deployed.

This is our second year evaluating the API functional automated testing space in the context of our Key Criteria and Radar reports. This report builds on our previous analysis and considers how the market has evolved over the last year.

This GigaOm Radar report examines nine of the top API functional automated testing solutions in the market and compares offerings against the capabilities (table stakes, key features, and emerging features) and non-functional requirements (business criteria) outlined in the companion Key Criteria report. Together, these reports provide an overview of the category and its underlying technology, identify leading API functional automated testing offerings, and help decision-makers evaluate these solutions so they can make a more informed investment decision.