Figure 1. GigaOm Radar for Evaluating Performance Testing Tools
As we noted in a previous article, the market for performance testing tools is heating up. Why? Because performance testing—which is a process that tests software applications under particular loads and provides feedback—is now seen as an integral element of a quality-first approach in cloud-native environments. Performance testing is necessary to give management and development staff information on the quality of performance of an IT business solution.
Another influencing factor in the growth of performance testing is the move to shift left in DevOps. Testing can kill innovation and increase the complexity of delivering solutions. So if a test tool is difficult to use, it can become a bottleneck due to the limited number of available skilled testers. And this can mean that teams either skip performance tests or accept a reduction in quality.
In his recent GigaOm Radar Report for Performance Testing Tools, Analyst Michael Delzer looked at 11 performance testing solutions based on criteria and a decision-making framework established in his earlier Key Criteria Report for Evaluating Performance Testing Tools.
Delzer says he made a number of discoveries around market changes that will interest technology buyers. The first was with vendor Micro Focus, which made two major changes in the last couple of years to address its ability to compete with other vendors.
“They changed their pricing to make it more in line with their competitors,” says Delzer. “They used to have a high price premium, so they would lose deals over price.”
Micro Focus is also embracing developer use of open-source tools to generate test cases, Delzer says. “Before you had to have a specially-trained tester write all of the test cases.”
Other observations include Apica extending its reach into more of Europe and North America, a move he calls “a solid choice if they can get the awareness of buyers.”
Startup outfit Loadster has gotten into the act, appealing to organizations with solutions for what Delzer calls the “low-hanging fruit of web site and API testing.” He says that the combination of a fast test creation process and compelling pricing could be enough for Loadster to make gains in the market.
One vendor Delzer is watching closely is SmartBear, which he says is poised to make a move after acquiring the SOAPUI solution, which it has since rebranded as SmartBear ReadyAPI.
“SmartBear is knocking at the door, as it is poised to enter the leadership space with Micro Focus and IBM,” Delzer says, adding that the ReadyAPI solution “completes the coverage gap to get them in the same ballpark as the other leaders.”