Menu

Building Your qTest Manager Repository: Test Design and Test Execution

Overview

qTest Manager provides considerable flexibility in how you organize your Test Cases.  As a user, you can change Test Case structure both from project to project and from release to release. 

Note: Keep in mind that each qTest Project is its own separate entity, meaning each Project is fully customizable. For example, you can have a Project that integrated with Jira, and a Project that is not. No matter the integration status, each Project can be tailored to meet the needs of your team.

Organization

As a rule of thumb, try to organize Test Design so that it is easy to link Test Cases to Requirements for traceability, and to reuse test cases in the future. When it comes to organizing Test Design, the goal should be to have users able to add Test Cases for Execution quickly and easily. To accomplish this, try to keep the number of Modules and Sub-Modules (folders in qTest) to a manageable number. The more folders and sub-folders you have, the less efficient your Testers will be when adding Tests to execute. 

Build your Test Execution structure so that it is easy to coordinate activities between different teams and look back at test log history. Ideally, test runs should be structured using the various hierarchies. In qTest the Test Execution hierarchy is:

Releases > Cycles > Suites > Test Runs:

Screenshot_1.png

Ensuring you use the hierarchy properly not only allows for an easier experience for your testers but also plays a large role in reporting in Insights.

Waterfall Methodology

Projects that follow a waterfall methodology have longer, less frequent releases with multiple testing phases. For example, you might have a round of unit testing, integration testing, system testing, and user acceptance testing. Users can consider these examples for this type of development methodology.

Jira Release Integration for Fix Versions

If you are practicing Waterfall or a Hybrid methodology, you are most likely using Fix Versions in Jira. The Jira Release Integration allows you to bring in your Fix Versions to the Test Plan as qTest Release objects. Please note that these Release objects automatically mirror those in Test Execution, allowing you a starting point for your folder structure. This also provides you with the Release objects for your Test Execution hierarchy.

Note: Everything updated in Jira reflects in qTest in real-time

The Release Integration is configured in Project Admin>Integration Settings>Release Integration.

Release_Integration.png

The Integration allows you to pull in Unreleased or Released Fix Versions.

By checking the Release Scope box, any issues being brought in as Requirements tied to that Release object will appear in the Release Scope in the Test Plan.
Release_Scope.png

By Testing Stage

In Test Design, organize test cases into folders by testing stage and then by product feature to make it easy to maintain tests specific to each stage that may need to repeat in future releases.  In Test Execution, use test cycles to differentiate between the testing stages that need to occur for a specific release.  This approach can be useful when different teams are responsible for executing each testing phase.

                  Test Design                                                 Test Execution
  By_Testing_Stage__Test_Design_.png   By_Testing_Stage__Test_Executio_.png

By Feature Set

Another approach is to organize Test Design folders by product feature. Within each product, use subfolders to organize test cases for different testing phases.  In Test Execution, you can separate by feature set and then by testing phase.  This approach is particularly useful if different teams are responsible for creating and maintaining test cases in Test Design based on the product feature or if it is helpful to drill-down your test execution metrics by testing phase.

                  Test Design                                              Test Execution
By_Feature_Set__Test_Design_.png  By_Feature_Set__Test_Execution_.png

Agile Methodology

qTest Manager is ideal for Agile methodologies because it is flexible in organizing releases and sprints. In an Agile or Scrum environment, you have more frequent releases, each consisting of multiple sprints.  Each sprint schedule may contain several user stories targeted for development.  

Jira Release Integration for Sprints

qTest allows you to pull in your Sprints from Jira into the Test Plan. This provides your Testers with essential data in the Test Plan section. This also provides you with the Release objects for your Test Execution hierarchy, as all Release objects mirror those in Test Execution.

Note: Everything updated in Jira will be reflected in qTest in real-time

The Release Integration is configured in Project Admin>Integration Settings>Release Integration.

Release_Integration2.png

The Integration allows you to pull in Active, Future or Completed Sprints.

Enter_Data_Retrieval-_Sprint.png

By checking the Release Scope box, any issues being brought in as Requirements tied to that Release object will appear in the Release Scope in the Test Plan.

Scope.png

By Sprint

Another approach is to organize Test Execution by Sprint.  You can use the Release Object [the highest level object created (imported from Jira) in the Test Plan] for each sprint. This approach can be helpful for customers who have very long releases, with many sprints within the release.  It can also be beneficial if there are many user stories that full testing within each sprint.

                   Test Design                                    Test Execution
By_Sprint__Test_Design_.png  By_Sprint__TE_.png

Complex System with Multiple Products

To track your testing for complex systems with multiple products, each having their own development and testing teams, we suggest considering the following options:

Define a Separate Project for Each Product 

This approach is useful if different teams track progress for their own products and each team requires its own metrics. This may also make sense if your external ALM, like JIRA, has separate projects for each product. The one to one approach is the most streamlined. You choice simply depends on how many qTest Projects your team is comfortable managing.

Define one project for multiple products 

We recommend using this option when testing products with the same release schedule.  For example, the Mobile and Web products may be managed by different teams, but it is easy to track the activities of the two groups in one project if both teams are on the same release schedule. In Test Execution, you can use the cycle and suite objects to separate one product from another product. You can also use this approach if a single project management group oversees testing across these products.

                Test Design                                             Test Execution
Define_One...Multiple__TD_.png  Define_one...Multiple__TE_.png

Testing Across Multiple Browsers

When testing across multiple browsers, such as Internet Explorer and Chrome, you may want to track the differences between the browsers, but make sure you can trace back and compare how the tested browser varies from release to release. Two options are described below.

Separate Test Cases for Each Browser 

One option is to organize your test cases by browser type, which means having multiple test cases for a single test. This approach is useful if the steps to execute the test vary more between browsers (e.g. the steps to execute one test case on Internet Explorer are very different from the steps to execute another test case on Chrome).

                   Test Design
Multiple_Browswers__TD_.png

Test run configurations for execution 

For greatest efficiency, we highly encourage using the Test Run Configurations feature. This feature allows you to use a single test case (browser agnostic) and create multiple test runs, one test run for each browser. This approach is useful when the differences in test cases per browser are minimal. In Test Design, you only need to create and maintain a single test, while in Test Execution multiple test runs can be used to ensure proper testing has been done. 

For more details, see Adding Test Runs with Configurations.

Checkout.png

Customer Specific / Customized Releases

Another factor to consider when organizing your test case repository is whether you have customer-based or customized releases. Consider the balance of reusing common test cases for efficiency versus tracking functional differences. You can use two approaches:

One Project for the Product, for Multiple Customers 

The first approach is to define a project per product and use separate releases per customer. This approach is useful if there are very few releases per customer and the test cases will vary minimally across the different customers.

One Project per Customer 

Another approach is to use a separate project per customer. This option is especially useful if you have different release schedules for each customer, or if the application under test is innately different for each customer (e.g., different features).

Powered by Zendesk