Functionize Testing ML Engine - how it works

How Big Data & ML Drive the Functionize Gen AI Engine

Trained on the 10 years of testing data from globally deployed applications, Functionize has the most advanced, accurate, and extensible machine learning models of any QA testing solution.

testGPT:
Generative AI tools

Gen AI agents can generate realistic tests, resolve issues & perform maintenance and more

99.95% accuracy in
ML element tracking

Highest element selection accuracy in the industry coupled with tests that make self-healing decisions in sub-seconds

Dynamic
learning

Functionize learns from every execution & adapts to changes in your application

Computer
vision

Visual AI combined with machine learning that creates a deeper understanding of test intent

Functionize ML-engine

AI Features

Generative Test Creation

Fully autonomous test creation based on real user journeys

Visual Testing

Visual verifications for full page testing or element-based testing

Anomaly Detection

Spot anomalies with Gen AI agent to reduce fail positives

Self-Healing

Model learns on every execution and dynamically heals tests

Smart screenshots

You can visually see all the elements as layers: baked into a screenshot

Quick Select/Edit

Edit elements right in the screnshots

Modeling

Federated Training model is a combination of multiple machine learning models and approaches

Data Lake

Real user journeys

All application states
All elements on each page

Data from creation & execution

~2500 data points per page

~10M points per one journey

500,000 tests
100Gb+ dataset

Our Journey to Generative AI

Our ML engine sees the application as a state machine

Applications change the state on the user input in our case test input.

The whole user journey can be thought of as a Markov decision process: a series of inputs and outputs to the state machine.

Engine learns all states

Deep Learning Navigates the test machine to understand:

The interactivity of the app - is it ready or loading something;
It understands all the elements on the page, their types and properties (DOM)
It sees the app in a high dimensional space, like timing and network traffic and have a human-like logic:

Do I need to scroll to the element?
Which part of the button should I click? etc.

ML engine sees the application as a massive state machine graph (Makrov decition process)

Evaluating the outcome

What is the probability that I was successful in that action?

It is able to dynamically adapt over time, self-heal and improve.

ML engine gathers the data

Each step is represented as a massive high-dimensional matrix with rich data:

Locations on the page
Structural position
Scrolling data
Timing data
Pre, post states
Relationship to code
CSS properties
Visual styles
Context & relation to other elements
Network metrics
Screenshots

We collect this data for every element on the page!

Test Data types being collected: location, code, DOM, CSS properties, context, nesting, timing, scrolling, pre and post states, screenshots.

Engine also learns on test runs & test edits

ML engine gathers the data on each test run which forms a time series data set, which is leveraged to adapt to changes and perform auto maintenance or self healing.

Every time a tester edits the test - our engine learns with reinforcement learning making self-healing even more robust.

Data Collection

Functionize collects unpresedented ammount of test data to build large data sets and and train super presize application models

Each test generates tens of millions of data points

An average web application has nearly 3500 elements on each page. We collect around 200 attributes per element that results in 700K data points per one step.

So for a 100 step test, we collect an average of over 70 million attributes and that's only the data from the initial test creation.

Every test run and edit adds even more data.

What is the next logical step?

Millions of data points in a vast Data Sea

Why cloud platform is essential

The resulting training model can be very heavy: tens of millions of data points per execution.

That Data lake requires a cloud to store and process all of this massive data including screenshots for each and every run of your test.

With CI/CD you want to test and co-locate it with data to get all the benefits of self-healing, have better latency etc.

Test Cloud

Cloud platform is essential for testing as it stors and process all massive datasets at one place

Journey to Full Autonomy

Autonomous Testing with Digital Workers

Advance towards testing autonomy with AI Agents, blending efficiency with cutting-edge technology.

Introducing test agents that make decisions and perform testing actions on your behalf:

Test Generation

Crate tests from prompts based on real user journeys.

Diagnostics & Maintenance

Auto-debug your tests with automated root-cause analysis and quick updates.

Test Documentation

Test Plan Generation for your case management tool.

Digital Workers Represents the Next Frontier in Achieving Fully Autonomous Testing

Real AI Test Automation.
Proven AI Value & Benefit.

test agents

Most sophisticated ML architecture to achieve testing at scale

AI Models and
Cloud Power

Unleash the power of fully autonomous testing. Harness agent technology, large-scale AI models, and cloud-scale to propel your testing into a new era of autonomy.

Powered by
Agent Technology

testAGENTS performs important autonomous testing actions based on application "awareness”, translating to massive productivity gains and cost reductions.

Interactions Through
Language

Command with clarity. All user interactions with testGPT are streamlined through English prompts, the foundational language of AI.

testAGENTS Make the Impossible, Possible

faster

Accelerate software
delivery by slashing
test cycle times

98%

coverage

Elevate release
confidence with the
heightened app visibility

70%

lower cost

Free up resources
to focus on revenue-driving initiatives

Ready to
Take Off?

Schedule a demo to see how Functionize is redefining the testing landscape.

Empower your team to achievesuperior testing results in record time.

Why Millions of Data Points?

True Automation is impossible without building a test definition model with deep learning and massive data sets.

Automated UI test scripts are notoriously expensive to maintain. They constantly break because each step is made up of brittle parts: brittle elements that are rigid to change and brittle actions that neglect how the application actually behaves.

Automation is supposed to save time. Yet these broken tests require manual fixes and slow down the speed of releases.

We fix exactly that.

We don't use rigid selectors

Instead, we capture multi-dimensional state of the entire page on every test run.

We do not use a single selector like the ID or XPATH which would break.
In fact, we don't use backup selectors which would slow down the execution and we certainly don't ask you to weight the likelihood of different selectors, which would be an annoyingly manual effort.
On test creation: Architect captures the entire state of the web page at every single step in the test. We collect hundreds of attributes per element, not just for the target element in the test step.