Add Test Runs to Test Suite
A test run is an executable version of a certain test case. The idea is that a test case can be executed (as a test run) in multiple areas, such as different Releases, Sprints, and more.
To create test runs within a particular test suite, perform the following steps:
- Select the Test Execution module.
- In the left panel, for the applicable release, expand the folder to display the test cycles and test suites associated with the release. In the example shown here, the test suite is called New Test Suite.
- In the example below, we will add test runs to the New Test Suite. You can add test runs by clicking on the Add Test Runs icon, or by selecting the Add button in the test suite grid.
- From the Add Test Runs window, click on the Test Cases tab where you can select the particular test case you wish you add as a test run. Then, click the Add button.
qTest provides a variety of different ways to add test runs, whether it's based on linked requirements, test cases, previous executions, linked defects, or based on a more complex query. See the sections below for examples of adding test runs based on these various criteria.
Add Test Runs from a Folder of Test Cases
Instead of selecting the specific test case one by one, qTest Manager allows you to add test runs from a folder of test cases. To do so, follow the steps below.
- From the Add Test Runs window, click on the Test Cases section. Select a test case folder from the list to add all of the folder's tests into your test suite. Then, click the Add button.
The Test Suite page is redisplayed. The test runs that have been added are displayed in the Test Run grid of the Test Suite page, as shown here.
For information on creating test runs from the Test Design module, see the article Create Test Runs with Parameters.
Add Test Runs and Retain Test Design Structure
When managing test runs in the Test Execution module, it can be beneficial to create a nested hierarchy to help structure execution activities and reporting. This can be accomplished by using test cycles and test suites. A great way to save time when creating this type of structure is to mirror the test design hierarchy.
In the Test Cases section of the Add Test Runs window, you can check 'Retain test design structure' to mirror the folder structure from Test Design. This will enable you to create cycle(s), test suite(s), and test run(s) in the same hierarchical structure used within Test Design.
- Test Design folders that contain subfolders will be created as test cycles in the test execution tree. Otherwise, the folders are created as test suites.
- If 'retain test design structure' is grayed-out and cannot be selected, it is because the test suite was selected before clicking "Add Test Runs."
- To use this feature, make sure to select either the root, release or test cycle before clicking "Add Test Runs."
Add Test Runs Using Test Case from Another Project
If you have many test cases that need to be executed in different projects, you can add test runs using a test case that is from another project. Before you can do this, however, you will need to share test cases from one project to another. For details, see the article Sharing Test Cases to Another Project.
To add a test run using a test case from another project, perform the following steps.
- From the Add Test Runs window, click the Test Cases option. In the Projects drop-down list, select the project from which you will borrow a test case. When you select the project from the list, the folder structure for test cases comes from the project where the test case is stored.
- From the tree view, select the test case you want to add as a test run. Then, click the Add button.
- The Test Suite page is redisplayed, which shows the test runs that were added from test cases in another project.
HINT: You can view the test case properties by clicking directly on a test run. The test run level with also contains your run properties and test execution history. To access the view below, make sure you are on the test run level.
Add Test Runs via Linked Requirements
Another way to add tests runs is based on requirements. If you select a requirement, any test case linked to the requirement will be added as a test run. This is useful if testers are responsible for new feature testing. Perhaps you are held accountable to test two new stories being developed, you can easily add all the test cases linked to those specific stories to a respective test suite.
- From the Add Test Runs window, select the Requirements tab at the top of the pop-up window and select the requirements folder.
- Click the Add button.
For example, there are two requirements in the “Medium” folder. Each requirement was linked to two test cases. By selecting the “Medium” folder and Add, four test cases will be added as test runs.
Add Test Runs Using Configurations
Test Run Configurations can be a powerful way to dictate how a test must be performed based on one or more variables, such as Browser + OS. Adding Test Runs with Configurations allows you to manage a single test case in the Test Design module and create multiple test runs in the Test Execution module with each test run representing a different configuration. For example, you may want to test a test case in Chrome, Firefox, Safari, and IE.
Before using this feature, perform the necessary setup in the article entitled Configuration Field Settings. Then, perform the following steps.
- From the Add Test Runs window, select the Test Cases option. Then, find the test case for which you want to create test runs. In the following example, we will select the ‘Become a partner page’ test case and create multiple test runs from the test case using configuration labels.
- Select the ‘Use configurations to add multiple Test Runs’ checkbox. This option is used to pick which individual configuration for which to create test runs.
IMPORTANT: You must have previously selected the Configuration field check box from within Field Settings for this project to make this field active. If you did not do so, the ‘Use configurations to add multiple Test Runs’ checkbox is not displayed.
- Do one of the following. a.) Click the Select configuration labels radio button. You must have previously created labels for this project from the Field Settings option. If you have not created labels for this project, this radio button is not displayed. b.) Click the ‘Select individual configurations radio button.
- Click the arrow to the right of the Select configuration labels radio button. Select the desired label from the drop-down list. In this example, we will select the ‘Windows browsers’ label. If there were five configurations for the selected label, five test runs will get added (one for each 'Windows browser').
- Click the Add button. For the selected test suite, a test run for every configuration within the selected label is created. Each test run is a different executable instance of the same test case.
Add Previously Executed Test Runs
qTest Manager can be used to add previously executed test runs to the current release. To add a previously executed test run, perform the following steps.
- From the Add Test Runs window, select the Test Execution tab at the top of the pop-up window.
- Select one previously executed test run you want to add to the current suite. Alternatively, select an entire test suite, test cycle, or release to add test runs in bulk. In the example shown below, all of the previously executed test runs in test cycle 2.1 are selected.
- Click the Add button.
Another way to copy previously-executed test runs in bulk to the current release is described in the article Copy or Move Objects in Test Execution.
- Right-click the desired test run, test suite, test cycle, or release in the left panel of Test Execution. Then, select the Copy option from the right-click drop-down menu.
- Right click the current release in the left panel and select the Paste option from the drop-down menu. The previously-executed test runs are pasted into the current release.
- The copied runs will be completely new and unexecuted. The status and history will not be carried over.
Add Test Runs Based on Linked Defects
Test runs can also be added by selecting defects associated with previously-executed test runs. This is particularly useful when re-testing bugs after they have been fixed by Development. To add test runs in this manner, perform the following steps.
- From the Add Test Runs window, select the Defects tab at the top of the pop-up window. A list of defects found in previous test executions is displayed.
- Refer to the Status column and select the defects that are ready to be retested. To select a defect, click the checkbox to the left of the defect or click anywhere on the line for the defect. In the example shown here, DF-3, which has a status of “Closed” has been selected, since "Closed" signifies the defect is ready to be retested. The defect status may be different based on your testing process.
- Click the Add button. Any test run associated with the selected defect is added to the test runs for the current release.
HINT: By adding test runs based on linked defect, you will create a separate test run with a fresh execution history. Otherwise, if you wish to add to the execution history of the existing test run, you can drag/drop that test run into the new Test Suite. See the article Copy or Move Objects in Test Execution.
Add Test Runs with Data Query
In the "Add Test Runs" window, you can switch between the tree view to the data query View. The data query view allows you to build complex queries to isolate objects based on certain criteria. You are able to query Requirements, Test Cases, and previous Test Runs within the respective tabs. An example would be querying for test cases with a certain tag, such as "Sanity", that you can then add to a test suite.
The following clauses are added by default to your new queries to make sure the found objects are valid objects for test run creation.
- For Requirements: Has Association with Test Cases = YES
- For Defects: Has Association with Test Runs = YES
- The data query view cannot be used to search for external defects (e.g., JIRA defects). Instead, you can use the tree view in the "Add Test Runs" window and select the Defects tab. From there you can search for defects by their ID, Summary or Status.
- Custom queries from the data query view of the "Add Test Runs" window cannot be seen in the normal data query and vice versa, and they cannot be shared across the team.
See the following articles.