Software development in the year 2018 is mostly repetitive tasks, and the bulk of end-to-end testing tasks is now scripted. Combining myriad consoles and scripting languages to achieve CI/T/CD was until recently state-of-the-art automation testing. Ultimately, this craving to automate everything motivates the creation of end-to-end testing tools. In reality, there is no single tool which satisfies the craving. But why automate a task when it really needs intelligence? If a redundant process can be scripted then we can use machine learning code to truly automate it. To reach this peak, we must first understand the reigning development regime and the tools we currently grapple with. Before we can make an intelligent change we must map this imaginary realm in which developers script the operation of myriad other scripts.
Automation testing of today is largely a developer’s task of writing programs to test other programs. This endeavor spawned hundreds of tools like Chai, Mocha, and Protractor. Today, an industry begging for a singular end-to-end solution to the piecemeal jumble of testing tools latches onto an amalgam like Cypress. But these supposed end-to-end testing tools really just mash all the Cucumbers and Gherkins into a distribution. They’re still the same tools and you still have to learn three scripting languages. It is not a homogenous tool, and it’s definitely not the solution. Functionize illustrates that intelligent software can completely replace both the redundant process and especially the scripting of test cases. In order to fully understand how, let’s explore today’s model of end-to-end testing by looking at the most popular tools and methods. First, here are the quintessential concepts.
When a developer adds or updates a module the new file must be copied to a server accessible to end users. This was once as simple as using an FTP client to send an HTML file to a server. The page went live instantly. But now we live in the Amazon era, wherein millions of dollars are packed into a single button click, careers are built and demolished by mouse hover events, and this previously simple task is now a “pipeline” of associated tasks in which many agents intervene to ensure everything functions precisely as intended. Enterprises are now burdened to staff QA with engineers. One recurring task is to assign the correct permissions to the new file and to make it executable. As we dissect this pipeline, which is now described variously as continuous integration, continuous testing, and continuous deployment, we see layer upon layer of increasing complexity. A hodgepodge of goofy names like Jenkins and Groovy are no less mystifying to spectators than the engineering practices they implement.
Jenkins scripts contain embedded Groovy code to automate the continuous integration of new modules. Bash scripts attached to code releases and called “hooks” in the Git repository lingo set permissions for the new file and do other file management operations. When it comes to testing the new module, the setup may be scripted with Cucumber, which is described as a Behavior Driven Development cycle (usually BDD). Cucumber’s own scripting language is called Gherkin, and is supposedly comprehensible to “business facing” team members. Decipherable may be a more appropriate word, but the prospect is spurious. Most of the testing frameworks include event recorders which create scripts as testers enter assertions. Coded UI is one such framework. Its Test Builder writes scripts compatible with .Net framework. Testers can then use the Visual Studio Enterprise edition to modify test cases. And the list of buzzwords continues throughout continuous deployment. We will have to delve into Docker and even get into Git if we are going to sort out this devolution before the road bends toward the abyss.
The Rise of Repos
All tools ultimately begin and end with the extraordinary rise of the versioning repository which has become standard fare in software integration, testing, and delivery. GitHub enables the Agile team to collaborate in new versions of code, perform rollbacks, and track changes to their apps. The core of this functionality is a set of event-driven actions attached to each commit. Events are scripted to react and automate every code commit. These events are called “hooks” in the Git realm. On each commit, Git checks the hooks directory to find and execute any attached scripts. Scripts run before, during, and after to correctly establish file permissions and deploy code, for example. This is a brilliant method of standardizing the deployment to ensure consistent compliance upon every event. Many end-to-end testing tools build on this repository strategy for sharing code and supporting documentation and data.
All Your Cukes In One Basket?
End-to-end testing tools have the purpose of testing the user experience and accuracy of an application, especially focusing on UI elements. During e2e testing data integrity must be verified, components and dependencies confirmed, and all discovered issues reported along the way. Although testing is straightforward in concept, tools which purportedly automate the process deliver limited success at a high overhead cost. The main source of this overhead is that fact that each testing tool is by nature limited to testing one target technology, but there are many technologies to test. The discussion begins with frameworks.
Which leads smart customers to this new assertion:
Bare Metal Versus Nonmetal
Wrapping it all up:
The next step in our end-to-end testing tool journey is to choose a wrapper for Selenium. What does this mean? Generally it means that we need another tool, like protractor or nightwatch Protractor is indeed a wrapper, because it actually contains Selenium Webdriver! And Protractor is also billed as an end-to-end test framework especially for AngularJS applications. Finally, we will need some more libraries like Sinon and TestDouble, because Mocha really does not include these doubles. And that brings us to the exciting conclusion of 101 testing tools (that is the number of tools, not the MOOC title). But there is still one monster tool which we must not overlook.
Instead of moving toward intelligence in automation, testing now requires an overhead of scripting by QA Engineers with advanced degrees. Look at Jenkins, for example, a common skill requirement of Fortune 500 QA engineering team candidates. Jenkins is an automation server, which runs on Apache Tomcat servlets, and which enables developers to script a lot of continuous integration. This current trend toward engineering level testers is not viable because customer experience is ultimately defined by customers who cannot be likewise required to have engineering skills to operate the application under test. Now we are in position to bring intelligence to replace scripting. Functionize brings a unique intelligence to replace the scripting that once encumbered engineering talent, allowing them to optimize their focus on product development.