Introduction

Most of the time, regression testing reports should be sent to an ALM to keep a history of what has been happening and see how balanced the SUT is.

The following article describes how to add the necessary lines of code in a Jenkinsfile to push custom information to JIRA-X-RAY (ALM) from Cucumber framework report. It barely depends on the framework but on the report (json file) Cucumber provides once the tests are executed.

Be aware to have previously installed and configured the following (not covered in this article):

X-RAY Jenkins Plugin

  • X-RAY configured as required in your Jenkins and verified successfully.

X-RAY Jenkins Plugin Configuration

Scope

In our specific case, we have previously defined all the tests and test plans in X-RAY. That's the reason, we only want to create a new test execution and link it to whatsoever is necessary.

Custom message

There are a wide list of X-RAY endpoints to keep any test, test plan and/or test execution up to date. Just because we use Cucumber and want to make some arrangements to the title, description and link to other JIRA issues, we have choosen cucumber/multipart.

The main difference between cucumber and cucumber/multipart is the data you may send to the server to customize the new test execution info.

After checking different options, we finally wrote this small snippet to send data to the server:

    post {
        always {
        ...
            cucumber fileIncludePattern: 'target/test-report-${BROWSER}.json', sortingMethod: 'ALPHABETICAL', reportTitle: '${BROWSER}'
            ...
            ...    
            step([
                $class: 'XrayImportBuilder',
                endpointName: '/cucumber/multipart',
                inputInfoSwitcher: 'fileContent',
                importInfo: """{"fields":{
                                "project":{"key":"${clientData['JIRA_PROJECT']}"},
                                "summary":"${clientData['XRAY_SUMMARY']} environment ${ENVIRONMENT} and date ${TEST_DATETIME}",
                                "issuetype":{"name":"${clientData['XRAY_SUMMARY']}",
                                            "id":"${pipelineData.xray.TEST_EXECUTION_ID}"},
                                            "components":[{"name":"QA"}],
                                            "description":"${clientData['XRAY_DESCRIPTION']}",
                                            "labels":["qa","cucumber","${clientData['XRAY_MAIN_TAG']}","selenium","regression","${BROWSER}"]},
                                            "xrayFields":{"testPlanKey":"${TEST_PLAN_KEY}","environments":["${ENVIRONMENT}"]
                                }
                            }""",
                importFilePath : 'target/test-report-${BROWSER}.json',
                projectKey: "${PROJECT_KEY}",
                serverInstance: "${pipelineData.xray.SERVER_INSTANCE}"
            ])
        }
    }

When running the Jenkinsfile stage covering the above code, the Jenkins console displays the following message:

X-RAY Jenkins Execution Info

By Checking X-RAY, a new test execution has been created and linked to whatever tests executed and also to the appropiate test plan.

NOTE: Sorry, some data in the following images are shown in Spanish:

X-RAY Report 1

X-RAY Report 2

X-RAY Report 3

Conclusion

For X-RAY, there is a Jenkins plugin to import Cucumber reports. It provides some ways to customize data for any of the following JIRA types:

  1. test,
  2. test plan and
  3. test execution.

You may find more information in the plugin documentation page here.