Mobile app testing with Functionize’s intelligent test agent
Mobile testing is critical for many businesses. You may be developing a native app or just want to know that your web application or site works on mobile devices. Either way, you need to do extensive quality testing across a range of mobile platforms. The traditional method for doing this was to manually test on each platform. More recently, platforms like Appium have allowed automated testing of mobile apps similar to that done by Selenium. Here at Functionize, we believe these methods are obsolete. Our intelligent test agent allows you to test any application on any device with no need for scripting or cross-platform modifications.
The problems with legacy mobile app testing
The legacy approach to mobile app testing has a number of major issues.
Let us look at these in detail:
Large device inventories
The traditional approach involves testing your app across multiple platforms. There is a wide range of mobile OSes and platforms (both phone and tablet). So, this can easily result in your needing to maintain an (expensive) inventory of hundreds of mobile devices.
Clunky test automation
Appium-style testing tools for mobile app testing is very simplistic. It shares all the problems of Selenium (slow and complex to create tests, hard to select elements, needs constant maintenance, etc.) But it also has its own issues such as the overuse of Xpath selectors hitting performance.
Slow to perform tests
Manual testing is slow, even with the most efficient test engineers. This makes it unproductive and has a direct impact on your release cycle. As you need to repeat the same test across a dozen or more devices, you start to get mistakes creeping in.
Slow emulation and simulation
While Appium offers the chance to automate your mobile testing, it still requires mobile hardware to run. The alternative is to use device emulators, such as those included in Apple’s Xcode. However, such emulators (and the related simulators) run slowly and are inefficient.
The power of intelligent mobile testing
Functionize leverages a combination of techniques including natural language processing, computer vision, and machine learning. The result is our intelligent test agent (ITA). This can take test plans written in plain English and convert them into fully-functional tests. These tests can then run on any platform (including mobile) without the need for modification. Because it is based on machine learning, the ITA learns how your application UI is meant to work. It can then cope with most UI changes and updates without the need for you to intervene. It is based on two core technologies.
Natural Language Processing
NLP is our natural language processing engine that takes test plans and converts them into tests. Test plans are written in plain English, so anyone on your team can contribute (unlike Appium, which requires skilled developers). These plans can be unstructured text (like the user stories your product team uses). Or it can take structured test plans, such as those produced by test management systems. In both cases, it compares the test plan with your UI and works out what is meant to happen in the test.
ML Engine
ML Engine is the brains at the heart of our intelligent test agent. It combines multiple forms of artificial intelligence to learn how your UI really works. When you pass in a set of test plans via NLP, these act as instructions to teach ML Engine about your application. It uses these to build a complex model of your entire application. This model takes account of hundreds of data points for every element within the UI. 50 test plans can be modeled in just a day, and once complete, each test can be run on any platform without modification.
Visual testing
Analyzing the results of Functionize tests couldn’t be easier. For every test step, ML Engine captures before, during, and after screenshots, highlighting any unexpected result on the screen. This approach has two big advantages. First, anyone can verify that the test is doing what it is meant to (something you can’t check in Appium). Second, anyone can instantly see any test failure and can then drill in to see more details (including the history of previous runs for that test).
Self-healing tests
All Apps and UIs go through multiple changes in their life. Often, these changes may be simple visual or style changes. Other times, they may involve moving elements on the screen or rearranging menu items. These sorts of changes cause real problems for any script-based mobile app testing system because they change the selectors for the UI elements. This results in legacy test scripts undergoing constant maintenance. Our tests are different. Elements are selected using machine learning and a complex set of descriptors that act like a fingerprint. Even when things change, the element can still be identified. This means Functionize tests are self-healing.
Root Cause Analysis and 1-click Updates
Some changes are more fundamental and affect the functionality or introduce ambiguity to the test. Here, test failures may not show up until many steps later in the test. In these cases, our Root Cause Analysis engine will identify the most likely cause of the failure. It then uses the history of that test to come up with likely fixes which it tests. These are then ranked and presented to you to choose the best one. You can then select this with 1 click, and the change will be learned and propagated through all the tests.
Info Box
Nested virtualization
Virtualization is where you emulate a server within a host system. Services like Google Cloud and AWS are built on virtualization. Generally, when we refer to virtualization we actually mean paravirtualization (where you provide access to the underlying hardware via drivers within the host operating system). Virtualization used to be very inefficient, but CPU extensions like Intel VT-X significantly improved performance. However, until recently, it was hard to virtualize mobile operating systems and hardware, particularly within public clouds. It could be done, but it was inefficient, slow, and lacked performance.
Google changed all that when they launched their nested virtualization platform built on Google Compute Engine. Functionize is a key partner of GCE thanks to the Functionize test-cloud. We were instrumental in driving forward this innovative technology. The result is you can now do performant emulation of mobile devices within a virtual server. We use this to accurately emulate both Android and iOS devices, obviating the need to maintain an inventory of physical devices.
Download Datasheet
AI-Powered Testing
Functionize is the industry’s most advanced enterprise AI-powered testing. We help teams break through testing barriers and enable organizations to release faster.
Learn how Functionize intelligent testing platform can help you:
- Create AI-powered tests using Architect or plain English via natural language processing
- Reduce test maintenance time by 85% with self-healing tests
- Scale test execution and run cross-browser tests in parallel