Functionize Testing ML Engine - how it works

How Big Data & ML Drive the Functionize Engine

Trained on the 8+ years of testing data from globally deployed applications, Functionize has the most advanced, accurate, and extensible machine learning models of any QA testing solution.
testGPT - autonomouse test generation

testGPT:
Auto-generated tests

Model generates required tests with data from user journeys & previous tests

ML model is 99.9% accurate

99.95% accuracy in
ML element tracking

Highest element selection accuracy in the industry coupled with tests that make self-healing decisions in sub-seconds

Dynamic learning - model adapts to changes in your application

Dynamic
learning

Functionize learns from every execution & adapts to changes in your application

Computer Vision

Computer
vision

Visual AI combined with machine learning that creates a deeper understanding of test intent

Functionize ML-engine

AI Features

Autonomous test creation with testGPT
testGPT

Fully autonomous test creation based on real user journeys

Visual testing, verifications and baselinening
Visual Testing

Visual verifications for full page testing or element-based testing

Anomaly detection - focus on real bugs
Anomaly Detection

Spot anomalies with AI to reduce fail positives

Sel healing or Auto maintenance
Self-Healing

Model learns on every execution and dynamically heals tests

Smart Screenshots
Smart screenshots

You can visually see all the elements as layers: baked into a screenshot

Visual testing feature - Quick select & Quick edit
Quick Select/Edit

Edit elements right in the screnshots

Modeling

Federated Training model is a combination of multiple machine learning models and approaches

Data Lake

Real user journeys
All application states
All elements on each page
Data from creation & execution
~2500 data points per page
~10M points per one journey
500,000 tests
100Gb+ dataset

How it works?

Our ML engine sees the application as a state machine

Applications change the state on the user input in our case test input.

The whole user journey can be thought of as a Markov decision process: a series of inputs and outputs to the state machine.

Engine learns all states

Deep Learning Navigates the test machine to understand:

  • The interactivity of the app - is it ready or loading something;
  • It understands all the elements on the page, their types and properties (DOM)
  • It sees the app in a high dimensional space, like timing and network traffic and have a human-like logic:
  • Do I need to scroll to the element?
  • Which part of the button should I click? etc.
ML engine sees the application as a massive state machine graph (Makrov decition process)

Evaluating the outcome

Functionize collects hundreds of attributes per element

What is the probability that I was successful in that action?

Vastly reduce visual-based test failures

It is able to dynamically adapt over time, self-heal and improve.

Computer Vision Validation

ML engine gathers the data

Each step is represented as a massive high-dimensional matrix with rich data:

  • Locations on the page
  • Structural position
  • Scrolling data
  • Timing data
  • Pre, post states
  • Relationship to code
  • CSS properties
  • Visual styles
  • Context & relation to other elements
  • Network metrics
  • Screenshots

We collect this data for every element on the page!

Test Data types being collected: location, code, DOM, CSS properties, context, nesting, timing, scrolling, pre and post states, screenshots.

Engine also learns on test runs & test edits

ML engine gathers the data on each test run which forms a time series data set, which is leveraged to adapt to changes and perform auto maintenance or self healing.

Every time a tester edits the test - our engine learns with reinforcement learning making self-healing even more robust.

Functionize collects unpresedented ammount of test data to build large data sets and and train super presize application models

Each test generates tens of millions of data points

An average web application has nearly 3500 elements on each page. We collect around 200 attributes per element that results in 700K data points per one step.

So for a 100 step test, we collect an average of over 70 million attributes and that's only the data from the initial test creation.

Every test run and edit adds even more data.

What is the next logical step?

Millions of data points in a vast Data Sea

Why cloud platform is essential

The resulting training model can be very heavy: tens of millions of data points per execution.

That Data lake requires a cloud to store and process all of this massive data including screenshots for each and every run of your test.

With CI/CD you want to test and co-locate it with data to get all the benefits of self-healing, have better latency etc.

Cloud platform is essential for testing as it stors and process all massive datasets at one place
Sign up for Demo

Autonomous Test Creation with testGPT

Create realistic tests based on real user behavior and test data

New predictive model

New state-of-the-art predictive model

based on real user journeys

User Behavior based learning

User journeys
drives representation

of your application usage

Large GPT predictive model

Large GPT
Deep Learning Model

generates relevant test cases

Our Engine can generate automatic tests - It gets the real user journeys, streams them and fine-tunes a large transformer base model to the given target application.

It learns all the specific correlations, types of data inputs etc. to predict the likely next steps in a given probability distribution range.

The results are meaningful tests - just like the text from popular GPT models.

Get the Superpower of AI Testing today with our Collaborative AI platform

Deep learning continues to get smarter with time, but companies still need a professional tester with Domain Knowledge to operate, that gets you to the next level of testing performance!

Functionize is the industry’s most advanced enterprise AI-powered testing. We help teams break through testing barriers and enable organizations to release faster.

Learn how Functionize intelligent testing platform can help you:

  • Create AI-powered tests using Architect or plain English via natural language processing
  • Reduce test maintenance time by 85% with self-healing tests
  • Scale test execution and run cross-browser tests in parallel

Why Millions of Data Points?

True Automation is impossible without building a test definition model with deep learning and massive data sets.

Automated UI test scripts are notoriously expensive to maintain. They constantly break because each step is made up of brittle parts: brittle elements that are rigid to change and brittle actions that neglect how the application actually behaves.

Automation is supposed to save time. Yet these broken tests require manual fixes and slow down the speed of releases.

We fix exactly that.

We don't use rigid selectors

Instead, we capture multi-dimensional state of the entire page on every test run.

  • We do not use a single selector like the ID or XPATH which would break.

  • In fact, we don't use backup selectors which would slow down the execution and we certainly don't ask you to weight the likelihood of different selectors, which would be an annoyingly manual effort.

  • On test creation: Architect captures the entire state of the web page at every single step in the test. We collect hundreds of attributes per element, not just for the target element in the test step.

We never use rigid selectors

The Benefits of AI Powered Testing

01
Unique fingerprints for elements

Unique fingerprints for elements

Functionize builds a detailed fingerprint for every element in your UI. This allows it to identify elements even if they are moved, restyled, and renamed.

02
Deep knowledge and understanding of your UI

Deep knowledge of your UI

Functionize uses these digital fingerprints to create a complex composite model of your UI. This allows it to understand what each element actually does, not just what it is.

03
Self-healing tests

Self-healing tests

Test maintenance is a real time-sink for QA engineers. Functionize learns from every execution and dynamically heals tests. This is because it knows how your UI is really working.

04
Root Cause Analysis

Root Cause Analysis

One of the most advanced features of our Engine is its ability to track back to the actual cause of a test failure. This is especially critical in complex systems where data changes and external factors can trigger test failures.

05
Finding solutions

Finding solutions

Identifying a failure is only half of the problem. Functionize also uses its knowledge of your previous test runs and test procedures to resolve errors. It presents suggestions to fix common failures so you can resolve issues in 1-click.

Computer Vision

Computer Vision Validation

More Than a Pixel by Pixel Comparison. Functionize will visually learn your application layout and identify any test failures or anomalies leveraging our ML Engine.

Smart screenshots: ML can understand application changes

Smart screenshots: Combining computer vision with machine learning allows for a deeper understanding of application changes.

Functionize leverages true color, no compression screenshots

Functionize leverages true color, no compression screenshots, analyzed by our ML Engine, to quickly mitigate visual issues, supporting development velocity.

Computer Vision Validation

Smart Element Detection

Eliminate visual test failures quickly & easily. Measure visual completion of each page versus only the DOM to better understand when users can fully interact with your product.

Functionize collects hundreds of attributes per element

Functionize collects thousands of attributes per element. Their unique combination creates a digital fingerprint, which is kept up to date with every execution.

All of this data enables 99.9% element selection accuracy. This makes xpath/element ID selector-based test failures a thing of the past.

Vastly reduce visual-based test failures

Vastly reduce visual-based test failures, regardless of the complexity or density of the page.

Accurate Element Detection with ML

Reduce Maintenance

Eliminate hundreds of hours debugging/maintaining with one-click update and self-healing tests. Functionize’s Root Cause Analysis engine supports teams to free up resources and focus on mission-critical tasks.

self healing tests

Tests self-heal to take the burden of maintenance off the user and onto the AI.

Root Cause Analysis

Remedy the root cause of any failed test with a one-click update across one of many tests.

ML Engine

Functionize ensures visual changes to your site do not break your test suites.

Reduce maintenance with Root Cause Analysis