The problems with legacy mobile app testing

The legacy approach to mobile app testing has a number of major issues.
Let us look at these in detail:

Large Device Inventory
Large device inventories

The traditional approach involves testing your app across multiple platforms. There is a wide range of mobile OSes and platforms (both phone and tablet). So, this can easily result in your needing to maintain an (expensive) inventory of hundreds of mobile devices.

Slow Manual Test Maintenance
Clunky test automation

Appium-style testing tools for mobile app testing is very simplistic. It shares all the problems of Selenium (slow and complex to create tests, hard to select elements, needs constant maintenance, etc.) But it also has its own issues such as the overuse of Xpath selectors hitting performance.

Slow test performance
Slow to perform tests

Manual testing is slow, even with the most efficient test engineers. This makes it unproductive and has a direct impact on your release cycle. As you need to repeat the same test across a dozen or more devices, you start to get mistakes creeping in.

Hidden DOM selectors
Slow emulation and simulation

While Appium offers the chance to automate your mobile testing, it still requires mobile hardware to run. The alternative is to use device emulators, such as those included in Apple’s Xcode. However, such emulators (and the related simulators) run slowly and are inefficient.

The power of intelligent mobile testing

The power of intelligent mobile testing

Functionize leverages a combination of techniques including natural language processing, computer vision, and machine learning. The result is our intelligent test agent (ITA). This can take test plans written in plain English and convert them into fully-functional tests. These tests can then run on any platform (including mobile) without the need for modification. Because it is based on machine learning, the ITA learns how your application UI is meant to work. It can then cope with most UI changes and updates without the need for you to intervene. It is based on two core technologies.

Adaptive Language Processing (ALP™)

ALP™ is our natural language processing engine that takes test plans and converts them into tests. Test plans are written in plain English, so anyone on your team can contribute (unlike Appium, which requires skilled developers). These plans can be unstructured text (like the user stories your product team uses). Or it can take structured test plans, such as those produced by test management systems. In both cases, it compares the test plan with your UI and works out what is meant to happen in the test.

Adaptive Language Processing

Adaptive Event Analysis (AEA™)

AEA™ is the brains at the heart of our intelligent test agent. It combines multiple forms of artificial intelligence to learn how your UI really works. When you pass in a set of test plans via ALP™, these act as instructions to teach AEA™ about your application. It uses these to build a complex model of your entire application. This model takes account of hundreds of data points for every element within the UI. 50 test plans can be modeled in just a day, and once complete, each test can be run on any platform without modification.

Adaptive Event Analysis

Visual testing

Analyzing the results of Functionize tests couldn’t be easier. For every test step, AEA™ captures before, during, and after screenshots, highlighting any unexpected result on the screen. This approach has two big advantages. First, anyone can verify that the test is doing what it is meant to (something you can’t check in Appium). Second, anyone can instantly see any test failure and can then drill in to see more details (including the history of previous runs for that test).

Visual testing

Self-healing tests

All Apps and UIs go through multiple changes in their life. Often, these changes may be simple visual or style changes. Other times, they may involve moving elements on the screen or rearranging menu items. These sorts of changes cause real problems for any script-based mobile app testing system because they change the selectors for the UI elements. This results in legacy test scripts undergoing constant maintenance. Our tests are different. Elements are selected using machine learning and a complex set of descriptors that act like a fingerprint. Even when things change, the element can still be identified. This means Functionize tests are self-healing.

Self-healing tests

Root Cause Analysis and 1-click Updates

Some changes are more fundamental and affect the functionality or introduce ambiguity to the test. Here, test failures may not show up until many steps later in the test. In these cases, our Root Cause Analysis engine will identify the most likely cause of the failure. It then uses the history of that test to come up with likely fixes which it tests. These are then ranked and presented to you to choose the best one. You can then select this with 1 click, and the change will be learned and propagated through all the tests.

Root Cause Analysis and 1-click Updates

Info Box

Nested virtualization

Virtualization is where you emulate a server within a host system. Services like Google Cloud and AWS are built on virtualization. Generally, when we refer to virtualization we actually mean paravirtualization (where you provide access to the underlying hardware via drivers within the host operating system). Virtualization used to be very inefficient, but CPU extensions like Intel VT-X significantly improved performance. However, until recently, it was hard to virtualize mobile operating systems and hardware, particularly within public clouds. It could be done, but it was inefficient, slow, and lacked performance.

Google changed all that when they launched their nested virtualization platform built on Google Compute Engine. Functionize is a key partner of GCE thanks to the Functionize test-cloud. We were instrumental in driving forward this innovative technology. The result is you can now do performant emulation of mobile devices within a virtual server. We use this to accurately emulate both Android and iOS devices, obviating the need to maintain an inventory of physical devices.

Sign Up Today

Functionize is designed with all levels of testers in mind. We simplify test creation while providing rich features to tune tests for your most complex scenarios. Running in our test cloud gives you elastic scale to handle continuous testing in your CI/CD pipeline or large regression suites. Meanwhile the deep machine learning engine is modeling your applications and dynamically updating your tests as your UI changes; saving you the aggravation of constant maintenance.

For more information, read our blog, follow us @functionize or email us to learn how you can get started today with Functionize Intelligent Testing.

Increase your productivity