Working with Test Plans

In Testiny, a test plan is a meta-layer between test cases and test runs that stores additional "generic information" for the later creation of test runs. Test plans in Testiny provide an interface where you can assemble your test cases. All test runs created from the plan have a common ancestor and allow for a detailed execution analysis over all runs.

In the following sections, we will showcase the user interface, explain how to create test plans, how to add test cases to plans and how to create a run from plans.

Test Plan List

When navigating to test plans, e.g. by clicking the item Test plans in the sidebar, you get to a screen showing a list of all existing test plans. If you have created a new project, it will be empty and you can create a new test plan. The screenshot below shows the list of test plans in the Demo Project:

You can create as many test plans as you need, there is no limit. Via the button, you can duplicate, edit, or delete test plans.

To open a test plan, just click the respective plan in the list and you will get navigated to the overview of the test plan, as shown in the example below.

The screenshot above highlights the following elements:

1. Go back to the list of test plans
2. Tabs (Overview and Test cases)
3. Actions area (Create test run, edit, duplicate, delete...)
4. Copy link to test plan to clipboard

Test Plan — Overview

This view gives you an excellent overview of the status of your test plan and the results of the tests assigned to that plan. At one glimpse, you can see the number of open test runs, how far these runs have progressed and most importantly, you can identify problematic test cases at a glance.

In the first part of the overview, as shown in the example below, you can see in the 'Test Plan Details' who created the plan, when it was created and how many test cases are currently included. Next to it, you see a link to open test run(s) that are linked to this test plan and the 'Test Case Coverage', i.e. the number of test cases that are in this test plan out of all test cases.

Most Frequently Failed Test Cases

This chart puts the focus on the most interesting test cases, which are in (most cases) failed or blocked ones. The chart is sorted by test cases that had a result other than "successful". When a test fails regularly, it is likely to be at the very left side of the chart. Tests that only succeeded or have not been executed are on the very right-hand side.
With the "hide passed test cases" slider in the upper right corner, you can hide passed tests in the chart, leaving only those tests that failed or were blocked at least once.

Test Runs Created from this Plan

This part shows a list of all test runs created from (or were linked with) this plan, their results, when they were created and when they were closed.
As the focus is often on open test runs, you can filter for open runs by selecting the checkbox 'Show only open test runs' in the upper right corner.

Test Plan — Test Cases

The 'Test cases' view within the test plan can be seen as the cockpit, where you can assemble, change or update which tests are part of your test plan. As mentioned above, the test plan can consist of statically added tests and dynamically added tests using a condition or by folder.

This view contains the following elements as highlighted in the screenshot above:

1. 'Add test cases' buttons
2. List of queries
3. List of included test cases

The test cases list (3) shows exactly which tests are part of the test plan when they had been last executed, their last result etc. and — as with every table in Testiny — you can configure the columns of the table as you like by clicking the icon.
Moreover, the "Origin" column tells you exactly which query or queries add which test to the plan. By deleting a query, the test also gets removed from the test cases list, unless it is also added by another query.

The queries list (2) shows all queries, which belong to the test plan. Static queries show a list of their manually selected tests. Dynamic queries show their conditions or the folder(s) they add to the plan. You can create as many queries as you need. If you have a large number of queries, we recommend giving each query a proper title for easier distinction.

Creating Test Plans

To create a test plan, navigate to the test plans list and click the 'Create' button in the upper left corner to open the creation dialog. Enter a meaningful title for the test plan (e.g. bugfix release plan, etc.). You can also add an optional description. Click 'Create' to confirm and create the test plan.

A new screen appears with the option to add test cases either by condition, by folder or from a static list.

Adding Test Cases to Your Plan

There are two ways of adding tests to the test plan — statically or dynamically.

Quickly summarized, using a "static" query, individual tests are part of the plan and they are independent of any changes in the test case (e.g. the priority is adjusted). Using a dynamic query ('by condition' or 'by folder'), tests are part of the test plan if they meet a condition (e.g. priority is 'high') or are in a folder. If values are changed (e.g. priority is set to 'low') or the test is moved to a different folder, it won’t be part of the test plan anymore.

For further information, please have a look at the Statically added Test Cases or Dynamically added Test Cases sections in the topic "General Concepts" of test plans.

Add from List (static)

This opens the list of test cases and you can select individual test cases or entire folders, which will select all tests in that folder.
The selected test cases are added to the test plan "by ID". Wherever you move the tests or if you change the attributes of the tests, they belong to the test plan until the test case is deleted or removed from the plan.

Select all desired test cases, change the default title of the query and proceed with "Save" to add the selected test cases to the test plan.

Add by Condition (dynamic)

This opens a view where you can specify conditions for test cases to become part of the test plan, e.g. setting "priority equals high" will add all tests with high priority to the test plan. Test cases that meet the specified condition are shown in the list below the conditions. If no condition is provided, all tests are shown.

Using "by condition" is a powerful way to assemble your test plans, as new test cases that meet the condition automatically become part of the test plan.
You can also adjust/extend the columns shown in the test cases list in the panel to show the fields being used for the conditions.

Specify any desired conditions, change the default title of the query and proceed with "Save" to add the condition set to the test plan. Whenever a test case (newly created tests, or modified ones) meets the condition, the test will become part of the test plan.

Add by Folder (dynamic)

This opens a view where you can select parent or subfolders to add tests dynamically to the test plan. All tests in a selected folder are part of the test plan. Selecting a parent folder automatically selects all subfolders.

Select all desired folders, change the default title of the query and proceed with "Save" to add the folders to the test plan. Whenever a new test is created in one of the selected folders or an existing test is moved to one of these folders, the test becomes part of the test plan. Conversely, if a test is removed from the folders, it’s also dynamically removed from the test plan.

Creating a Test Run out of your Test Plan

To create a test run from the test plan, use the "Create test run" button in the upper right corner. All test cases currently listed in the test plan will be added to the test run and you can start executing the tests once the run got created.

The newly created run will then appear in the Overview tab under Test Runs Created from this Plan and also get listed in the Most Frequently Failed Test Cases chart. This gives you an overview of which test runs from this test plan are currently in progress, their current state and which test cases have the highest failure rate.

Furthermore, the test plan is also linked in the Details tab in the test run.