GigaOm Key Criteria for Evaluating API Functional Automated Testing Solutionsv2.0

1. Executive Summary

API functional automated testing brings testing rigor to APIs. This is important because applications today are increasingly collections of APIs with a UI, and many applications do not even have a UI element. Without a UI, traditional application test tools can’t validate the APIs at all, and with a UI, such tools can only validate the end result, not the intermediate steps that most APIs represent.

No matter the API standard or data transport language in use, APIs need to be exercised the same as any other function, method, or class generated during the software development lifecycle (SDLC). Functional automated testing tools enable developers to test their code without requiring that an application be developed to encompass the APIs. In modern enterprises, DevOps teams normally want to integrate API functional automated testing into their processes, and most teams actually utilize these tools to interactively develop APIs. Given a UI designed to call APIs with all available variables and evaluate results, these testing tools serve as excellent build/test tools as well.

Business Imperative
As the importance of APIs grows, functional automated testing tools are the primary interface to them for the entire SDLC. Developers use the clients in these tools to write APIs, the mocking capability to develop a client or server in the absence of the other, and the testing ability to ensure quality. Better applications result from tools in this technology category.

Sector Adoption Score
To help executives and decision-makers assess the potential impact and value of an API functional automated testing solution deployment to the business, this GigaOm Key Criteria report provides a structured assessment of the API functional automated testing sector across five factors: benefit, maturity, urgency, impact, and effort. By scoring each factor based on how strongly it compels or deters adoption of API functional automated testing, we provide an overall Sector Adoption Score for API functional automated testing of 3.2 out of 5, with 5 indicating the strongest possible recommendation to adopt. This indicates that API functional automated testing is a credible candidate for deployment and worth thoughtful consideration.

The factors contributing to the Sector Adoption Score for API functional automated testing are explained in more detail in the Sector Brief section that follows.

Figure 1. Sector Adoption Score for API Functional Automated Testing

This is the second year that GigaOm has reported on the API functional automated testing space in the context of our Key Criteria and Radar reports. This report builds on our previous analysis and considers how the market has evolved over the last year.

This GigaOm Key Criteria report highlights the capabilities (table stakes, key features, and emerging features) and non-functional requirements (business criteria) for selecting an effective API functional automated testing solution. The companion GigaOm Radar report identifies vendors and products that excel in those capabilities and metrics. Together, these reports provide an overview of the category and its underlying technology, identify leading API functional automated testing offerings, and help decision-makers evaluate these solutions so they can make a more informed investment decision.