6 Open Source Frameworks for test automation

Software development is composed of several phases. From its initial planning and the development until its official release. Each stage has its process, timing and costs. Precisely, testing and error resolution is the stage that usually takes longer. Therefore, if you are thinking about automation as a solution, you are in the right path. Besides, if you are thinking about creating your own automation framework, better you think twice. In most of the cases, it is quite advisable to consider one or several of the available open-source options.

Working and Developing Teams don’t need to invest time in complex mechanisms or exhaustive processes to test their software. Most of the existing software already shares a set of best practices, common tools and libraries. It is simply not necessary to re-invent the wheel, an available framework will help to make the code of automation testing re-usable, maintainable/tenable and stable.

And it is not an easy task. It is not just testing the user interface (UI), it is necessary to have tools to test the APIs as well. These API/Integration tests, when omitting the user interface, become faster and reliable than GUI tests. Accordingly, it is important to rely on tools that can perform a set of tests in several layers (layered testing) of our application.

It is clear that “building our own testing framework” can be an obvious trap, unless when having considerable time and resources. That is the main reason why it is best to check first some available options based on open source and evaluate their capabilities.

6 Open-source frameworks for test automation

1. Serenity – open source

It is a Java based framework which integrates perfectly developing tools such as Cucumber and JBehave. It is designed to facilitate writing automated acceptance and regression tests. It allows abstracting most of the repetitive code that is needed some times to start using tools such as Selenium WebDriver.

On the other side, Serenity also offers many integrated functionalities, such as managing WebDriver instances, managing the states between steps, making screenshots and executing tests in parallel and all this without writing a single line of code.

Therefore, Serenity creates highly detailed reports. Out of the box, it creates live documentation that can be used not only to review the results of the test with Selenium BDD, but also to document the requirements.

Recently, Serenity has recently added integration with REST Assured, one of the most popular Java open-source libraries for API testing. It means that it allows us to check and validate easily all the details of requests and responses in test reports without adding additional code.

Lastly, SerintyJS, a version including support to JavaScript developed to provide support to many Front-End developers are also responsible for testing their projects.

2. Cypress

Cypress is a framework of automation testing for developers focused on making Testing Driven Development (TDD) a simple and pleasant experience.

It has an architecture different from the other testing frameworks; for example, Selenium WebDriver is executed remotely out of the web browser, Cypress is executed inside the web browser offering more consistency.

3. Robot Framework

Robot Framework is based on Python, but it can also use Jython(Java) or IronPython (.NET). It uses a keyword-driven automation approach for testing, which makes the tests legible and easy to create. It includes many libraries for testing and other tools to add new functionalities to generate a test bench which is perfectly adapted to our application.

Selenium WebDriver may be by far the most extended testing library currently in use, but Robot Framework can also test other applications different from websites, such as FTP, MongoDB, Android and Appium.

4. TestProject.io

Here comes one of my favourite options. Are you looking for an automation tool which is totally free and has the support of a robust community?

TestProject is the first automation testing platform based on the cloud and based on the world’s cloud. TestProject makes it easier for testers to make their job fast and collaborate using open-source frameworks such as Selenium and Appium.

TestProject has been created, thanks to its open-source approach, to allow test automation experts and organisations to profit from the two best open-source tools for test automation: Selenium and Appium. TestProject is shaping the future of testing software by encouraging a collaborative community which can assemble as individuals or as teams.

N.B. In the moment of writing this article, Java and C# implementations are available. Currently, compatibility with Python, JavaScript and other programming languages is under development.

5. Galen Framework

If your efforts in automation are mostly focused on the user experience design (UX) or on design tests, Galen Framework could be the perfect complement for your needs.

It has specifically been created for the UX tests and Galen has special syntax and rules which can be used to describe and verify the design of the application.

It also allows specifying the size of the browser’s window to later execute the test against our application to verify the design specifications. The Galen tests also generate detailed HTML reports containing screenshots, and the tool includes tools for image comparison with a very characteristic heatmap.

6. WebDriverIO

Still looking for an automation framework which uses Selenium and which allows programming in JavaScript? WebdriverIO offers a complete framework to test our web apps. Its main advantage, besides being very light, is that it does not require implementation from scratch.

Using a “click and run” philosophy, it comes with all the necessary tools to create sets of scalable and sustainable tests and contains additional characteristics which cannot be found in other frameworks.

Which one to choose?

Now that all the options have been presented, it is time to pick up one. There is no clear winner, in fact. The present article has not the intention to create a ranking of automation test frameworks but to enumerate some available open-source options.

Perhaps the best advice is that before writing the first line of code to create your own framework, ensure that there is no library or framework you can use. Stop wasting your time reinventing the wheel, have a look at all these automation frameworks first.

Follow all Miguel Parada‘s articles (english version)

Follow all Miguel Parada‘s articles (spanish version)

Translation by Santiago Moreira