Test automation with Zephyr and Selenium

We have recently undertaken a project for a customer who had a web application that was built externally, but which was tested in-house. There were a series of existing manual tests which were to be automated. The goal was to automate these and create a maintainable, automated testing pack using Selenium and Python. The web application is under active development, so testing would need to evolve over time. Automating the testing would improve efficiency, and allow the customer to QA the outsourced development in a more timely fashion.

Zephyr Squad is a very popular Jira App, that supports manual testing. It supports the importing of externally generated test results, so that users can get combined reports of both manual and automated tests.

We did an internal proof of concept to see how we could integrate the Selenium automated tests, with Zephyr Squad, thus combining both manual and automated test results. The same results can be achieved with any tool that generates test results, or a testing tool which can import results from test automations.

By integrating the output of Selenium and Python with Zephyr, we get one overall view of test results, whether automated or manual. The same results can be achieved with any tool that generates test results, or a tool which can import results.

Selenium and Python - automating the UI test pack

SeleniumBase which is an open-source Python framework built on top of Selenium WebDriver, a popular automation tool for web browsers. It provides a higher-level interface and a range of additional functionalities that facilitate the creation, execution, and maintenance of automated tests. Whether you're a seasoned QA engineer or a developer new to web testing, SeleniumBase offers a user-friendly and powerful Python-oriented toolkit.

We structured a set of test cases to drive the Web UI, which ensures that the UI behaves as expected. This is useful in a number of scenarios, such as integration testing, regression testing and user interface testing. Traditionally, much of this had been manual, but automating this is a key enabler for rapid development and the use of automated pipelines. In fact, the ability to rapidly re-test quickly enables faster development, and helps “shift left”, promoting test driven development, so it has a real tangible benefit.

With this testpack, it would be possible to check UI functionality for the website and that site security have not been affected by any of the ongoing development updates performed, allowing rapid QA of outsourced development.

Python Squad

Zephyr Squad is a very popular manual testing tool that is closely integrated with Jira. It allows test cases to be created within Jira, using a special issue type to represent them, which can contain test steps and other useful information. It also has the concept of “Test Cycles”, which is used to encapsulate the results from executing a set of test cases. We can have multiple cycles - e.g. we may need to run several test cycles when doing regression testing - so it may be that the first cycle has a number of failures, but the second has fewer, once bug fixes have been done.

Zephyr also has the ability to bring in results from from automated frameworks, so that automated test cases and test cycle results can be seen alongside the manual test execution results. If a set of automated test cases are re-executed, we can either refresh the test cycle results, or we can keep the new set. Test cycles can be linked to a Jira Release (FixVersion), so it is possible to report on the test results against Releases of software.

Manual testing still has its place. Sometimes it may not be economically viable to automate everything, or it may be that we are in a transition phase from manual to automated testing.

Integrating the two approaches allows us to get combined QA results from both automated and manual testing - the best of both worlds!

Integrating SeleniumBase with Zephyr Squad

To get the single pane of glass with automated and manual test results, we wanted to import the results file into Zephyr. It is possible to do this direct via the UI, or automate it using the API or ZBot agent. We opted to use the ZBot which can be left running to check directories for new output files. The ZBot integrates with Zephyr and enhances its capabilities by providing a range of automation features that streamline the testing process, improve efficiency, and enable seamless integration with other testing tools and frameworks.

Checking the BDQ website!

We obviously can’t use customer tests in a public blogpost, so we did a simple proof against our own website. We have created a basic test which can navigate around the site and check the validation applied to our contact form.

• Start from the beginning: a couple of Python test cases
  We wrote a couple of simple test cases - one of these would “fail” first time through, and we would “fix” it for the second execution. You can see the results below in Python, and the results flowing through into Zephyr.

• First pass: one pass and one fail
  We can see the results in the test harness, and inside Zephyr, we can see that we have a new test cycle with a pass and a fail. The tests inside Python have been created as test cases within Zephyr.

Python Tests with a failure

The Python test failure can be seen in the Zephyr Test cycle

• Second run: After fixing the problem
  

  We re-run the test automation. Both tests have now passed. Inside Zephyr, we can see that this has created an additional new test cycle with two passes. The previous results are still there for reference. It is possible to simply “re-use” the same test cycle to avoid an excessive build up of results, if you are running tests very frequently.

Python Tests after fix

Both tests are now showing as fixed inside the Zephyr Test cycle

Summary

If you are using Zephyr Squad and Jira Software, and want to introduce automation, it can be done! It is possible to take the output from a wide variety of test tools, at different stages of development. We’ve given an example of one of the thornier problems, that of automating UI tests. This can be very useful for integration testing, or doing final in-house QA against an application that has been outsourced for development.

If you have questions regarding anything in this blogpost or questions in general, please get in touch. Let's talk about what you need.