Making the Business Case for Low-Code Software Testing

Effective testing that meets the monthly, or even weekly, release cycles of developed and packaged software is exerting business-critical pressure. Leading organizations look into traditional test automation for the execution cycle of functional and non-functional testing. However, this increases operating complexity and costs within an organization’s test team, as new technical skills are needed to develop, run, and maintain automation assets for the unit, API consumption, and regression testing.

This has led CIOs to consider how they scale their testing teams to meet business demand without a spiraling increase in cost and complexity. Low-code software testing enables organizations to empower citizen testers through the use of artificial intelligence (AI) techniques, such as natural language processing (NLP), to compress testing cycles, increase the velocity and quality of testing, and provide faster feedback. They allow organizations to scale their testing capability without disproportionately increasing headcount, maintenance, and management costs. Low-code software testing tools, as seen in Figure 1 below, provide an operational step change that can significantly reduce time and effort.

Figure 1: Benefits of Low-Code Testing Software over Traditional Approaches

It is clear that low-code software testing can improve upon the common benefits of traditional test automation and address many of its limitations. To this end, low-code software testing provides a range of key benefits that address scale, accuracy, coverage, and complexity, such as:

• Citizen testers: A low-code software testing tool uses natural language processing (NLP) to allow anyone to automate tests. By expressing test intent in simple English, the tool generates and executes a script in a matter of minutes. It allows a community of non-technical citizen testers to reduce “test fatigue” and scale test coverage.

• Accelerate test creation: A skilled test engineer can create up to two test scripts per day using traditional test automation. A low-code software testing tool can increase that by a factor of four to five for both technical and non-technical testers.

• Autonomous maintainability: A manual maintenance burden exists where testers can spend between 25% to 40% of their time fixing previously working test scripts. Low-code testing software can reduce this burden by up to 90% by leveraging artificial intelligence to detect changes and heal test scripts autonomously.

Non-AI test automation provides substantial long-term benefits for highly repetitive forms of testing. However, this still requires testing resources skilled in creating and maintaining manual and automated test assets. The onset of low-code software testing provides autonomous capabilities to discover what testing assets are needed, generate said testing assets, and maintain/update assets through iterative discovery. These sophisticated capabilities provide for the following use cases:

• Increased test coverage. Low-code software testing can perform a broader range of repetitive tests at higher frequency and velocity. The more tests can run at scale, the greater the opportunity to increase test coverage, leading to an increase in software quality and faster delivery.

• Data compliance adherence. CIOs are accountable for demonstrating adherence to data privacy and compliance requirements across the full range of software applications and APIs in the organization. Testing at scale with AI-generated synthetic data (which syntactically represents real data) is an essential use case to ensure data privacy, security, and compliance.

• Continuous improvement. The highly iterative nature of testing continuously generates synthetic test data that improves the accuracy and effectiveness of low-code test scenarios across non-functional and functional test types.

• Accelerating unit and regression testing. Agile and DevOps have driven a culture of continuous delivery (CD), integration (CI), and testing (CT) in many organizations. Through the use of artificial intelligence (AI), low-code software testing generates assets—quickly and at scale—from actual code (not just stubs). Reducing time and effort, by allowing developers to focus on modifying rather than authoring their test assets.

• User-interface testing. Validating the visual correctness of an application, or series of API-connected applications, through image-based testing of a user interface (UI) at scale is an essential use case. It is a highly detail-oriented task (prone to accuracy issues) as every UI element has to be assessed in regard to color, size, position, shape, and whether an element is obscured or hidden.

Applying automation techniques, technology, and approaches to the operating model for testing software was inevitable. There is no alternative to the critical role of testing, though now there is an alternative to purely relying on manual techniques. If a CIO does not consider AI-driven test automation to be a fit for their business, then they only have two options:

• One option would be to maintain a manual approach but increase the internal test team’s head-count to meet demand.

• The other option would be to engage an external service provider and outsource the test function.

The first option is not practical or sustainable for large and/or complex application estates. The second option can often add operational complexity if not managed firmly through supplier and cost management.

The costs that should be factored into the adoption of a low-code software testing tool are as follows:

• Software License Costs. Most low-code software testing tools offer different pricing tiers based on a scale of deployment metrics, including number of test assets to be managed, volume of tests, and number of applications/products to be supported. There may also be SaaS and on-premise licensing options, where an organization will incur separate additional infrastructure costs to create a hosting environment.

• Technical Training. Adopting a low-code software testing tool will still require both technical and non-technical/citizen testers to be up-skilled to use the software.

• Configuration Costs. All tools require a period of adoption to configure for an organization’s specific needs. However, this should be a one-off cost for a short period.

• Existing Obligations. If an organization is already using an outsourced model for testing, the adoption of a tool outside of the contract may affect costs with one or more suppliers.

• Administrative Costs. Test engineers will be required to maintain, debug, and support the tool to ensure it operates optimally and address any issues or configuration changes. This is considered as a cost relating to internal testing support-line behavior before engaging the vendor.

Selecting a low-code software testing tool comes with a medium level of risk. As with any technology investment, it’s important to define the right functional requirements and consider the non-functional requirements required for the planned usage scenario. The following challenges and risks should be considered when adopting a low-code software test automation tool:

• Unrealistic Expectations. Low-code software testing, supported by artificial intelligence (AI), may not cover all the test needs of an organization, and a degree of manual testing will still be required. Developers, and high-code testers, will need to write tests for complex business logic, uncommon and infrequent test scenarios, and address misclassification issues with ML test models.

• API and RPA Testing. Cataloging, orchestration, and version control of APIs and RPA bots is challenging for most organizations before considering test scenarios. Therefore, application, API, RPA, and test engineers must model a composite business service to drive change management when using low-code software testing.

• Automation Limitations. No matter how sophisticated and rich in features a tool is, it will not be foolproof. All tools have limitations, so it is essential to thoroughly engage with a software vendor to determine the optimal use cases for your organization.

• Adoption Curve. When adopting a tool, changes are always required across existing operational and testing processes. Training will be required for technical and citizen testers to use and maintain the tool. Also, organizations must ensure the tool’s compatibility to interact with target applications/services and related dev/test software tools.

As with pretty much all technology solutions, adoption requires preparation to realize the value in their investment. The common cadence of a software delivery lifecycle (SDLC) leverages both Agile and DevOps. SDLC also relies on and integrates with a wide range of software tools across development, operations, and security/compliance teams.

So it is essential to plan for integrating a low-code software testing tool to reflect this. The adoption of any tool has technical, process, and cultural implications. By not focusing equally on each, the use and value of the tool might falter before it has a chance to demonstrate its potential fully. To achieve this, we recommend the following 30/60/90 day plan:

• 30 Days: Planning and strategy. Perform a low-code software testing feasibility study to shortlist relevant test scenarios. Develop a strategy to choose a framework that focuses on linear, data-driven, and keyword-driven testing. Determine metrics and risk profiles that measure value in regard to leveraging low-code software testing to manage velocity and volume.

• 60 Days: Technical configuration and training. Deploy the low-code software testing tool, run discovery processes, and generate test assets from AI and testers. Provide relevant training to both technical and citizen testers. Perform a focused test cycle to determine the operational effectiveness of the tool and make adjustments accordingly.

• 90 Days: Adoption into software development lifecycle. Define, and fully document, best practices for using a low-code test automation tool, including failure analysis and result reporting mechanism. Fully incorporate it into development sprint, continuous delivery (CD), and continuous integration (CI) operating cycles.

GigOm believes that an organization can build a foundation to realize the efficiency cost benefits by taking this approach and addressing the above mentioned risks. However, it should be noted that the use of AI across all forms of testing is still relatively new. To this end, GigaOm recommends that an organization establishes at least the following key performance indicators (KPI) to measure success:

• Percentage of test assets suitable for low-code software testing
• Equivalent manual test effort (EMTE)
• Low-code software testing coverage

Over the coming years, the use of AI for software testing across both functional and non-functional forms will continue to grow and accelerate. Start small and focused, then grow its usage based on the types of testing best suited for AI test automation to optimally manage velocity and volume without further increasing cost and complexity.