Hi we are setting up the test automation reporter form cypress to testiny. we used the mochawesome but we could not figure out how to set up that we could post the result to the same run. Currently for each run of cypress tests it always creates a new test run.
is there some setting for this?
Hi,
Yes, you can configure how you want to group results into automated test runs.
The API/CLI groups together automated results based on the specified source and the specified Run Fields (--run-fields). You also need to specify the --incomplete flag to report results to the same run.
In some known CI environments, the CLI sets some default Run Fields. For example, if executed in GitLab CI/CD, the results are by default grouped by the environment variable CI_PIPELINE_ID.
If you just want to try it locally, you can specify a value by which to group the results:
# report results to a run
npx @testiny/cli automation [...] --source unit-backend --incomplete --field-values my_run_key=run1 --run-fields my_run_key
# more results to same run
npx @testiny/cli automation [...] --source unit-backend --incomplete --field-values my_run_key=run1 --run-fields my_run_key
To mark the run as completed, execute the following command:
# complete run after jobs have finished:
npx @testiny/cli automation [...] --source unit-backend --complete-runs --field-values my_run_key=run1 --run-fields my_run_key
More details on automated test runs in this docs section and on how to report to the same automated run from parallel/shared runs in a CI environment.
Please let me know if anything remains unclear or if you have any other questions.
Best regards,
Hanna