Test run configurations are used to manage the different environments in which a test is executed. For example, you may have to test functionality on two operating systems, such as on a Windows computer and a Mac computer.
The test run configurations feature lets you save time to use a single test case in test design and to easily create separate test runs for each combination so each test run has its own execution history.
Two levels of setup are required in order to use this feature - one on the site level and one on the project level.
Create Configurations in - Site Administration
- Position the arrow cursor over your username located in the top right corner of the Home page and Select the Administration option.
- Select the Configuration Field Settings button. The Site Administration - Configuration Field Settings page is displayed. This page contains three areas: Variables, Configuration Sets, and Configurations.
Define Variables for Test Run Configurations
The following steps are used to define the variables (such as browsers) and the values associated with each variable (such as Chrome, Firefox, Safari, Internet Explorer).
- Choose the variables you will use. qTest Manager comes with pre-defined variables shown in the Variables area of the page. These variables are provided as sample data to provide an example. For the applicable variable, click the Gear icon on the right side of the page.
The Edit Variable window is displayed.
- Click the Add Value button. A new line is created in the Allowed Values area of the window.
- Move the cursor to the Value field on the new line. Click the new line, remove the words “new value”, and type the value, such as Chrome.
- Repeat steps 2-3 as needed. Click the OK button to save your changes on this screen.
Create Configuration Sets
Configuration sets are used to define the combination of variables to test. (The variables were defined above in the Variables section of this procedure.)
- Click the Add Sets button located in the Configuration Sets area of the page.
The Add Configuration Set screen is displayed. This screen is used to determine which variables you will test in a given combination.
- In the Configuration Set Name field, enter the name of the configuration set. The name should allow you to easily identify this configuration set, such as Hosted Location Plus Plan.
- Click the Add Variable button. Then, click the Variable field to display a down arrow to the right of the field. Click the down arrow to display a drop-down menu.
- Select the applicable variable associated with this configuration set, such as Hosted Location.
- Click the Value field to display a down arrow to the right of the field. Clicking the field also displays “All” in the Value field. Click the down arrow to display a drop-down menu. If you wish to remove some value(s) for the variables, de-select the value(s) associated with the variable selected in the previous step.
- Click the Add Variable button and select another variable and associated values for this configuration set. Repeat this step as needed. When finished, click the OK button.
qTest Manager generates all possible combinations based on the selected variables and lists them in the Configurations area of the page. The list of configurations saves the user from creating each combination one at a time and also ensures standardization of configurations across projects.
Optional: Hide Configurations
Based on the list of configurations that were automatically generated from configuration sets, you can choose to omit certain configurations from being tested. For example, if you are testing the OS + Browser configuration set, the configuration for “Windows OS + Safari Browser” would be omitted.
- To remove a configuration, navigate to the Configuration area of the page, move your cursor to the Active field and deselect the applicable checkboxes for the items for which you do not want a test run. Then, click the Save button. In the example shown below, two configurations have been deselected.
HINT: Now that you have set up the configurations at the site level, you can test them at the project level. In other words, for the configurations available to use on the site level, you will now determine which ones are applicable to a particular project.
Enable Test Run Configurations in Project Field Settings
- Move your cursor to the Gear icon and select the Field Settings option from the drop-down menu.
The Release page is displayed.
- Select the Test Run option from the left panel. The Test Run page is displayed.
- Click the Configuration field listed in the Display Name column. The Edit System Field screen is displayed. The configurations you set up at the site level are displayed at the bottom of the screen. By default, all configurations are selected, as shown in the Active column.
- Deselect the Active checkbox for any configuration(s) you wish to hide from the list of available configurations in Test Execution.
Optional: Assign Labels to Test Run Configurations
You can group multiple configurations using a label. This feature is used to select, in bulk, which configurations you want to test and allows you to quickly generate multiple test runs for all configurations in the label rather than selecting one configuration at a time.
- Select the desired configurations you want to assign to the label by clicking the checkbox located to the left of the Set column.
- Click the Assign Labels button. The Assign Labels pop-up is displayed.
- Enter the label name in the New Label field and click the OK button. The Test Run fields page is displayed. Verify that the checkbox in the Active column is selected for the Configuration display name (shown in line #14 in the screenshot below).
- Click the Save button.
See the article Add Test Run Using Configurations