Skip to main content

Jenkins

Integrate Testiny with Jenkins and submit automated test results from your CI/CD deployments to Testiny.

You can use any test automation framework with Testiny, you'll just have to configure your tests to generate a result file in one of the supported file formats (JUnit XML, TestNG, Playwright JSON, NUnit and more). The test results are then submitted with the Testiny CLI (available as npm package or standalone binary). With Testiny, you can track your tests over time, identify frequently failing tests, efficiently debug and fix CI failures or flaky tests, and have acccess to all test results in a single place.

General Workflow​

  1. Install the Testiny CLI. Typically, you would install the CLI with NPM, making it easy to install and to keep it up-to-date.
  2. Execute your automated tests within your Jenkins CI pipeline and generate a test result report in one of the supported formats.
  3. Use the Testiny CLI to upload the results to Testiny.
info

You don't need to use Node.js for your project or automated tests, but you can conveniently install the Testiny CLI as a npm package via Node.js. This way you can easily upgrade the CLI or always use the latest version. If you do not want to use Node.js, you can also use the Testiny CLI standalone binaries to upload test results.

Basic Example​

This example shows how to upload your automated test results to Testiny from your Jenkins CI pipeline. In this example, we show how to upload test results from Playwright, but you can use any other automation framework.

In this example, we install the Testiny CLI via npm, run the automated test and finally, upload the results with the CLI. The CLI automatically adds some environment variables from the Jenkins CI environment to the test run. See the list below for all used environment variables.

You'll also need an API key to authenticate with Testiny. Learn how to create an API key in Testiny.

caution

Please note, that the Testiny API key is not stored in the code, but as credentials in Jenkins. Define an environment variable TESTINY_API_KEY and it will be automatically used by the CLI.

Jenkinsfile
pipeline {
    agent {
        docker { image 'node' }
    }
    environment {
        TESTINY_API_KEY = credentials('TESTINY_API_KEY')
    }
    stages {
        // ...
        stage('Test') {
            steps {
                // Install project dependencies
                // ...
                // Run your automated tests & generate a report file
                // ...

                // Install Testiny CLI & upload results
                // Note, that the API key is stored as a credential in Jenkins.
                sh 'npm install --no-save @testiny/cli@latest'

                sh '''
                  npx @testiny/cli automation --project 1 --source "e2e-tests" --playwright results/results.json
                '''
            }
        }
        // ...
    }
    // ...
}

For projects with multiple test suites — such as frontend, backend, or e2e-tests — you can report each suite as a separate test run within the same GitHub Actions pipeline. Simply execute the CLI command npx -y @testiny/cli automation (or testiny-importer automation) for each test suite. Make sure you use different source names (otherwise, Testiny would group together the results from the different test suites).

More CLI Options​

The CLI also offers option to add additional fields with --field-values or to specify a custom run title with --run-title-pattern.
If you're using Testiny Server, add the URL to your instance to the CLI command with the --app option.

Learn more about the CLI in our CLI Usage Guide.

Environment Variables​

Following environment variables are automatically collected by the Testiny CLI for Jenkins CI:

ci_build = BUILD_NUMBER
ci_job = JOB_NAME
ci_repository = GIT_URL
ci_runner_id = EXECUTOR_NUMBER
ci_build_url = BUILD_URL
vc_branch = GIT_BRANCH
vc_commit = GIT_COMMIT

Default Run Fields = [ci_build]

Default runTitlePattern = "%{ci_repository} - %{ci_build} - %{vc_branch} - %{vc_commit}"

More resources​