Xray is a complete Test Management tool for Jira. It is a full-featured app that does not require any other software in order to run. Xray supports the entire testing life cycle: test planning, test design, test execution and test reporting.
On the other side, Cucumber is a framework for behavior-driven development (BDD), where the test specification is done in a natural language, while the implementation of each specification’s phrase (i.e. step) is done in code (e.g. Java, Ruby or some other language).
In this article we give an explanation of the steps about how you can create Cucumber test Automation with the help of Xray and how you can import the automated test results back in BDD JIRA.
Create Cucumber Test with Xray
One of the ways that a Senior PHP Programmer creates an automated Cucumber using Xray is to navigate to the Jira user story that you are planning to test.
By clicking on Create new Test button, the form for creating a test case is opened. In order to create Cucumber Xray test, under Test Details tab, for Test Type choose Cucumber and from Cucumber Type drop-down choose Scenario. (The option Scenario Outline is used when you want to create instructions that contain variables).
The next step is to populate Cucumber Scenario with test steps in the Xray Gherkin language.
Under the Link issues tab you can see that the test is automatically linked to the user story from where the test was created.
By clicking on “Create” button the Cucumber test will be created.
Test Execution and Importing Test Results in JIRA
Xray export Cucumber tests is enabled by Xray as .feature file that further can be used in your automated tests. In order to export .feature file, from the created test choose More -> Export to Cucumber.
Further, you can import the exported .feature file in your Xray Cucumber automation test project, for example implemented in Eclipse.
After implementing the step definition file, using for example Selenium Web Driver, for the given feature file, and executing Cucumber test in Eclipse, Cucumber.json file with results will be generated under the target folder in your Eclipse project.
Using Xray, the test results from the generated json file with the test results can be imported back in JIRA, and associated with the corresponding test, by manually importing the json file or by using API that is provided by Xray.
Importing Cucumber automated test results in JIRA through UI
In order to import the generated result from the automated tests, in the Test Execution* associated with the Cucumber test, click on More -> Import Execution Results and import the generated json file in JIRA. (* Test Execution is custom JIRA issue type introduced by Xray, that can be described as an assignable, “schedulable” task to execute one or more tests for a given version/revision along with its results. For more information on different Xray custom JIRA issue types, please refer to https://confluence.xpand-addons.com/display/XRAY/Terms+and+Concepts)
Once the execution results are imported in JIRA, Test status is updated. In our case the test has failed, so correspondingly its status is updated in the associated Test Execution.
Execution details, including the error that was thrown during automated test execution, can be accessed as depicted on Figure. 10
Importing Cucumber automated test results in JIRA through API
Xray provides specific REST endpoints for different test execution results depending on the testing framework that is used for test automation. In the table below are listed some of the REST endpoints, while the full list can be seen on: https://confluence.xpand-addons.com/display/XRAY/Import+Execution+Results+-+REST
In a real-world scenario, importing automated tests results in JIRA through Xray provided APIs will be executed in conjunction with CI/CD tools, like Jenkins, however in this article, for demonstration purposes we will use JMeter to perform POST operation on the REST endpoint for Cucumber test result output format: /rest/raven/1.0/import/execution/cucumber
After invoking REST API, through JMeter, the corresponding Test Execution associated with the Test is updated, showing Test as Pass:
Conclusion
Xray plug-in for JIRA provides ideal solution for test management on all projects where JIRA is used for tracking requirements and bugs. Besides it supports management of test cases, test planning, test execution and requirements coverage with its built-in support of Cucumber and other test automation frameworks enabling support whether is needed manual or automate testing.
We have presented to you the Jira Xray Cucumber solution, so now is your turn to try. And don’t worry, if you have any issues, you can always use an Xray Jira tutorial and Cucumber Jira instructions or feel free to contact us!
