Skip to main content

CircleCI

Integrate Testiny with CircleCI and submit automated test results from your CI/CD deployments to Testiny.
You can use any test automation framework with Testiny. Configure your tests to generate a result file in one of the supported file formats (JUnit XML, TestNG, Playwright JSON, NUnit and more) and submit them with the Testiny CLI. With Testiny, you can track your tests over time, identify frequently failing tests, efficiently debug and fix CI failures or flaky tests, and have acccess to all test results in a single place.

General Workflow​

  1. Install the Testiny CLI or use its standalone binary file.
  2. Execute your automated tests within your CircleCI pipeline and generate a test result report in one of the supported formats.
  3. Use the Testiny CLI to upload the results to Testiny.
info

You don't need to use Node.js for your project or automated tests, but you can conveniently install the Testiny CLI as a npm package via Node.js. This way you can easily upgrade the CLI or always use the latest version. If you do not want to use Node.js, you can also use the Testiny CLI standalone binaries to upload test results.

caution

Use contexts in CircleCI to store your Testiny API Key. Define an environment variable TESTINY_API_KEY and it will be automatically used by the CLI.

Default Environment Variables​

Following environment variables are automatically collected by the Testiny CLI for CircleCI:

ci_workflow_id = CIRCLE_WORKFLOW_ID
ci_build = CIRCLE_BUILD_NUM
ci_job = CIRCLE_JOB
ci_job_id = CIRCLE_WORKFLOW_JOB_ID
ci_repository = CIRCLE_PROJECT_REPONAME
ci_runner_id = CIRCLE_NODE_INDEX
ci_build_url = CIRCLE_BUILD_URL
vc_branch = CIRCLE_BRANCH
vc_commit = CIRCLE_SHA1
vc_tag = CIRCLE_TAG

Default Config Options​

Following options are used by default by the Testiny CLI for CircleCI:

Default Run Fields = [ci_workflow_id]

Default runTitlePattern = "%{ci_job} - %{ci_workflow_id} - %{vc_branch} - %{vc_commit}"

The default options are equivalent to adding the following options to the automation command:

--run-fields ci_workflow_id --run-title-pattern "%{ci_job} - %{ci_workflow_id} - %{vc_branch} - %{vc_commit}"

More CLI Options​

The CLI also offers option to add additional fields with --field-values or to specify a custom run title with --run-title-pattern.
If you're using Testiny Server, add the URL to your instance to the CLI command with the --app option.

Learn more about the CLI in our CLI Usage Guide.

More resources​