API and Microservices Management Benchmarkv2.0

1. Executive Summary

Application programming interfaces, or APIs, are a ubiquitous method and de facto standard of communication among modern information technologies. The information ecosystems within large companies and complex organizations encompass a vast array of applications and systems, many of which have turned to APIs for exchanging data as the glue that holds these heterogeneous artifacts together. APIs have begun to replace older, more cumbersome methods of information sharing with lightweight, loosely-coupled microservices. This change allows organizations to knit together disparate systems and applications without creating technical debt from tight coupling with custom code or proprietary, unwieldy vendor tools.

APIs and microservices also give companies an opportunity to create standards and govern the interoperability of applications—both new and old—creating modularity. Additionally, they broaden the scope of data exchange with the outside world, particularly mobile technology, smart devices, and the Internet of Things (IoT), because organizations can share data securely with non-fixed-location consumers and producers of information.

The popularity and proliferation of APIs and microservices have created a need to manage the multitude of services a company relies on—both internally and externally. APIs vary greatly in protocols, methods, authorization/authentication schemes, and usage patterns. Additionally, IT teams need greater control over their hosted APIs, such as rate limiting, quotas, policy enforcement, and user identification, to ensure high availability while preventing abuse and security breaches. APIs also have enabled their own economy by allowing the transformation of businesses into a platform (and even a platform into a business). Exposing APIs opens the door to many partners who can co-create and expand the core platform without knowing anything about the underlying technology.

Still, many organizations depend on their apps, APIs, and microservices for high performance and availability. For this report, we define “high performance” as companies who experience workloads of more than 1,000 transactions per second (tps) and need a maximum latency below 30 milliseconds across their landscape. For these organizations, their need for performance is tantamount to their need for management because they rely on these API transaction rates to keep up with the speed of their business operations. For them, an API management solution must not become a performance bottleneck. On the contrary, many of these companies are looking for a solution to load balance across redundant API endpoints and enable high transaction volumes. Imagine a financial institution with 1,000 transactions happening per second—that translates to 86 million API calls in a single 24-hour day! Therefore, performance is a critical factor when choosing an API management solution.

In this report, we reveal the results of performance testing we completed on two API and Microservices Management platforms: Kong Enterprise and Google Cloud Apigee X.

In this performance benchmark, Kong came out a clear winner—particularly because of its higher rate of transactions per second. Kong’s maximum transactions per second throughput, achieved with 100% success (no 5xx or 429 errors) and with less than 30ms maximum latency, was 54,250. By contrast, Apigee X’s maximum throughput was 1,750.

Testing hardware and software in the cloud is very challenging. Configurations may favor one vendor over another in feature availability, virtual machine processor generations, memory amounts, storage configurations for optimal input/output, network latencies, software and operating system versions, and the workload itself. Even more challenging is testing fully managed, as-a-service offerings for which the underlying configurations (processing power, memory, networking, et cetera) are unknown. Our testing demonstrates a narrow slice of potential configurations and workloads.

As the report’s sponsor, Kong opted for a default Kong installation and API gateway configuration out of the box—the solution was not tuned or altered for performance. The fully managed Apigee X was used “as-is,” since, by virtue of being fully managed, we have no access to, visibility into, or control of its respective infrastructure.

We hope this report is informative and helpful in uncovering some of the challenges and nuances of API management platforms.

We have provided enough information in the report for anyone to reproduce this test. You are encouraged to compile your own representative workloads and test compatible configurations applicable to your requirements.