The problem with traditional testing
1Cheaper. Renting cloud infrastructure from an IaaS provider is almost always cheaper than relying on your own servers. Moreover, when you have no tests to run you won’t get charged for servers that are lying idle.
2Scalable. Cloud is nearly infinitely scalable. The only limitations on scaling your tests are likely to come from the underlying test automation software (and from the resource limitations of the VMs you are using).
3Reliable. Virtual servers always offer some level of reliability. That could be cold migration (where you power off, start a new instance and recover using the backup). Or it could be hot migration of running servers. The better providers even offer forms of Disaster Recovery. This is in marked contrast to what happens if your own server fails.
4Realistic. This is one of the more subtle benefits of using cloud testing. Generally, you will be able to run your tests from multiple locations and, importantly, your tests are coming from an external network.
5Effective. With cloud-based testing, you get access to almost infinite reliable storage. This means you are able to store all your historical test results. In turn, this enables you to fully analyze any test failures or issues. It also allows you to observe trends in your testing and is useful when you are dealing with regressions.
The problems with dedicated test infrastructure
Dedicated test infrastructure has several problems that make it both a drain on your resources and a source of inefficiency.
Expensive to maintain
Running your own infrastructure is expensive and inefficient. Servers use a lot of energy, especially if they have to be cooled. Most test teams can’t afford new servers, so “make do and mend”. But repairing and replacing server components is never cheap. Moreover, every time there is an issue it disrupts your testing significantly.
Hard to scale up
Creating a setup that can test at scale is hard. Even using Selenium Grid you will struggle to be able to run more than a few dozen tests in parallel. If you look at a typical project you might have 100 tests, each needing to be run across 10 browser/OS combinations. So, just a simple smoke test might require 1,000 test runs.
Requires sysadmin support
Running your own infrastructure implies the need for sysadmins. Servers need to be set up. Software and OSes need to be installed. Any security patches must be installed since this infrastructure needs external connectivity (and hence poses a security risk). All browsers and libraries must be kept up to date to ensure accurate testing. All these tasks take significant time and resources.
Unrealistic
One of the biggest issues with running your own infrastructure is the lack of realism. All your test traffic will originate from a single location. Chances are, you don’t have an especially fast network connection, so you may suffer from self-congestion with test traffic. Above all, the chances are you will be unable to fully test any system that sits behind a load balancer.
What is needed to create a test cloud?
Setting up your own testing cloud is perfectly feasible. But moving your test infrastructure to the cloud does require a certain amount of effort. So, you may find that without the support of a competent dev-ops engineer, it can be quite a painful experience.
1Provider. Needless to say, there are thousands of providers to choose from. Not only the usual suspects (Google Cloud, AWS and Azure), but also vast numbers of smaller providers often offering very competitive prices and good customer service.
2VM specification. Decide how powerful the VMs need to be. Most providers allow you to specify the number of (virtual) processors, the amount of RAM and the storage per-VM.
3Storage. Do you need additional storage that can be accessed by all VMs? And do you also need it to be accessible externally?
4Scale. How many VMs will you (usually) need? Remember, many providers will allow you to scale up on demand if you need additional VMs.
5Networking. How would you like to network them? Will you need your own virtual switch? Do you need to pay for external IP addresses for each VM?
6Data I/O. How much data will you need to transfer in and out? Think about the data needed for testing your app, as well as the data needed to access test results.
7Location. Where physically would you like your VMs? Many operators allow you to pay a premium to gain some control over things like rack-locality of your VMs.
8Software. You need to decide exactly what software you need and set things up properly. This is nearly as hard as doing it for your own infrastructure. However, many cloud providers will provide images with much of the software already installed.
The Functionize Test Cloud
The better alternative to all that is to use the Functionize Test Cloud. This is based on Google Cloud and so it gives you access to test locations around the world. We ensure that all images are completely up-to-date and you are able to test with pretty much every possible browser/OS/device combination you need.
There are several compelling reasons to use the Functionize Test Cloud. Space doesn’t let us list them all, so let’s look at a few of the important ones.
Convenience
First and foremost has to be the convenience. Our system is designed to be intuitive and quick to learn, and you won’t have to waste time setting up any infrastructure before starting your testing.
Simplicity
Our Natural Language Processing engine takes test cases written in plain English and converts them to tests you can run on our Test Cloud. And running tests is equally easy with integrations with all the best-known CI engines.
Reliability
Our service is highly available and dependable because we partner with Google, The last thing you need is for your test infrastructure to suddenly get flaky just before a release!
Scale
We offer load testing at impressive scales. You write a single test case and it can run across every browser/OS combination in parallel. Moreover, we allow you to test with up to 100,000 simultaneous connections from multiple locations globally. This means you will be sure your servers can handle the load. Even better, our performance metrics are based on realistic user sessions using the tests you have defined.
