BDQ Blog - News, information, opinions & reviews | BDQ

Test Automation - How it works with Zephyr

Written by Chris Bland | 26 Aug 2020

This is a short blog post about Zephyr for Jira and the various options for integrating it with test automation. This was correct at the time of writing, but Zephyr are very active with their product development, so expect more features to be on their way!

What is Zephyr for Jira and what has it got to do with test automation?

Zephyr for Jira is a very popular test automation tool that is fully integrated with Jira. It is available both on Server and on Cloud. It is very effective for managing manual testing and has all the usual facilities that one might expect. However, it also exposes interfaces and has plugins that allow it to show results from automated builds and other test tools. 

You can therefore show both results from manual testing, and automated testing, all within the same product!

Core Zephyr for Jira concepts

Zephyr for Jira uses the following. There is more, but this is relevant to the automation discussion.

Object Description

Test Cases

These are specialised type of Jira issues, which contain test steps, summaries, descriptions and so forth. A Test Case will be executed within the context of a Test Cycle. Passes, fails etc will appear within Test Cycles.

Releases/FixVersions

This is a Jira concept, which Zephyr for Jira uses. The basic idea is that work is grouped into Releases. V1.0, V2.0 and so forth. Stories would generally be in one Release, but an Epic may span more than one Release (depending on how you are structuring your Agile projects). It is important to note this, as Zephyr Test Cycles, are associated with Releases.

Therefore you can have the concept of Smoke Tests, Functional Tests, re tests or any other group of tests, all associated with a given Release. E.g. "We need to run the Regression Cycle before releaseing V3.0.

Test Cycles

These are associated with a Release, and contain a number of Test Cases.

A Test Case can be in any number of Test Cycles, and its result may be different depending on the Test Cycle. E.g. it may fail in Cycle 1, but after being fixed, it may pass in Cycle 2.

The Execution result of a Test Case exist in the context of a Test Cycle.

 

The automation interfaces allow us to push results into these constructs. Here we have an example of a Test Cycle for a version 3.0.0 release.

Integration options

These interfaces mainly use REST and JSON - currently within an API called the ZAPI (Zephyr API). Zephyr do provide all the documentation for this.

You must push data into the structures above.

For example:

  • A set of Junit tests would each correspond to a test case
  • Each execution of those tests would correspond to a Test Cycle, with the results in that Test Cycle.

As things stand, the integration options are as follows:

  • Jenkins/Bamboo plugins. These are plugins which you can use within your CI/CD tool. 
    • These will generate Test Cases and Test cycles.
    • Junit or TestNG files should be generated during your build job, and passed to the plugin
    • The plugin will push the results back to Zephyr for Jira
  • ATOM - the user uploads files directly into Zephyr For Jira, creating test results
  • ATOM Zbot agent installed on remote server:
    • The Zbot can retrieve files from a given location.
    • It can be triggered using the ATOM API, remotely (e.g. during a build, or manually)
  • ZAPI - direct REST API access - full control
    • You can add data directly via the REST API
    • It is your responsibility to create the required structures
    • This is the most complex route
    • The API documents are here: https://getzephyr.docs.apiary.io/#

An ATOM task looks like the screen below, and is accessible from the Jira Server. 

The Jenkins plugin is similar:

In both cases, you can see that you have to choose a Verions and a Cycle (or allow a new Cycle to be created). Test Cycles are where the automation frameworks post their results.

Summary

The easiest way of pushing back results is to use the plugin - drop them into Bamboo or Jenkins, configure accordingly, and you will start to get results from your CI/CD processes appearing within Zephyr for Jira.

The most control, and most complex route, involves using the full ZAPI REST api. You have to do everything yourself manually, and the ZBot and ATOM framework is in the middle - this allows you to pull results, and trigger via the REST API, or manually.

If you would like more advice, or assistance with implementing test automation, please get in touch.

Useful links:

Test Management for global IT implementations with Jira & Zephyr webinar recording
Test Automation and Management solution page
Zephyr product page