AI testing with Functionize’s intelligent test agent
Automated testing is an essential element of any modern QA strategy. However, most automated testing is based on Selenium scripts, which creates a number of problems. In solving these problems, Functionize has transformed automated testing into AI testing.
The problem with script-based testing
Selenium was revolutionary back when it was created almost 2 decades ago. But it has always had a number of key flaws. These have limited its utility and prevent most companies from being able to get the full benefits of automated testing. Let’s look at a few of these, and then show how AI testing solved the issues.
Creating a set of test scripts for a UI is effectively a complete software development project in its own right. Each test script has to be written step-by-step, tested and debugged. And this process is entirely manual. Sometimes, it can even be impossible to test some functions, especially if your UI imports 3rd party content. As a result, test scripting is extremely slow and inefficient. Even under the best conditions, it can take days to create a single script.
Few UIs are expected to only run on one browser. As a result, cross-browser testing is an essential aspect of automated testing. The problem is, adapting a test script to work on a different browser always requires significant effort. It can even require completely rewriting the script. When you multiply this by the large number of browser/platform combinations, this reflects significant extra effort.
Test scripts interact use selectors to locate elements in the UI. They then perform actions like clicking buttons, toggling switches or entering text. Unfortunately, these selectors are very brittle. Even simple CSS changes can alter every selector on the page. And where there are multiple possible elements, Selenium will just select the first one. The result is, every change to your UI will break most of your tests. These then have to be fixed, meaning test maintenance can absorb half a test engineer’s time.
When a test script fails, the test engineer has to work out what triggered the issue. They then have to check whether this was a real failure or just a problem with the test script. The issue comes when you have tests that only fail many steps after there was an issue. For instance, if the wrong product was added to a shopping cart. The test may proceed perfectly until it comes to checking out the cart. Tracking down such failures takes ages, even for an experienced test engineer.
How can AI testing solve the problem?
AI testing leverages artificial intelligence to solve challenges like the ones listed above. Let’s look at each problem in turn and see how AI testing might help.
1Test Creation. The problem with test scripts is that they have to be written by skilled test engineers. The engineer takes a detailed test plan and has to painstakingly translate it into a working script. This means finding the best selector for each element that needs to be interacted with and determining just what elements need to be verified at each step. But nowadays computers can be taught how to understand complex natural language. NLP (natural language processing) has advanced leaps and bounds in the last 5 years. Now it is advanced enough to interpret test plans and even user stories. If you couple this with other forms of AI, you can do away with test scripts completely.
2Cross-browser testing. The main reason cross-browser test scripts are challenging is that each browser renders the UI slightly differently and each browser needs a different version of Selenium WebDriver. This means each selector has to be checked and updated for every browser. AI testing allows you to use machine learning to avoid this. ML can build a model of what the UI is really doing, so instead of using a single selector for each element, you can create a fingerprint. This means the element can now be located in every browser.
3Test maintenance. In theory, AI testing can eliminate test maintenance by combining machine learning with NLP and computer vision. Test maintenance is usually required following one of 3 things: a UI layout change, a CSS update or a change in the text on a page element. For instance, the login button may move from the top left to the top right, be renamed “sign in” and recolored. AI testing is able to cope with this. Firstly, the machine learning model knows that the button is still calling the login API. Secondly, NLP understands that “sign in” and “login” are synonymous. Thirdly, the system is able to recognize that the button is still next to the “register” button and knows this makes it likely to be the same button.
4Failure analysis. When you move to AI testing, you are able to leverage a whole range of AI approaches. Computer vision is particularly powerful for analyzing test failures. A manually created test script never checks every single element on the screen. To do so would be impractical. But computer vision allows you to compare each screen with what was expected and identify any changes. When you add real intelligence, it can even learn to ignore things that are known to change, such as the date. AI testing also implies that your test system understands how your UI works. It does this by using machine learning to constantly improve its knowledge. As a result, when things change, it is able to recognize the change the same way a human tester does, so it knows not to flag this as a failure.
How Functionize AI testing helps
Functionize uses artificial intelligence to simplify the whole process of automated testing, from writing new tests to analyzing the test results. AI testing like this can transform your QA process and dramatically reduce your time to market.
Sign Up Today
Functionize is designed with all levels of testers in mind. We simplify test creation while providing rich features to tune tests for your most complex scenarios. Running in our test cloud gives you elastic scale to handle continuous testing in your CI/CD pipeline or large regression suites. Meanwhile the deep machine learning engine is modeling your applications and dynamically updating your tests as your UI changes; saving you the aggravation of constant maintenance.