Structuring qTest Projects Integrated with Rally
This article focuses on best practices for integrating your qTest Projects with Rally and how to establish an efficient process and workflow between the two tools. When thinking about how to best integrate your Rally projects with qTest Manager, it is important to understand which development milestones best drive your testing efforts: Releases or Sprints.
For detailed instructions on how to connect your project with your Rally instance please refer to the setup steps described in the following article: Integration With Rally.
First, it is important to fully understand the major objects that you and your team(s) will be working with. Take a look at the two definitions below:
- A Release is a set of features (Backlog Items of Stories) deployed together as a single update to your application being developed. Assigning features to Releases will determine which new features will be available to your customers when the release is deployed. To ensure teams release new versions on a regular basis, releases are often scheduled on a fixed date.
- Sprints define a period of time in which development teams implement and deliver a prescribed amount of product functionality. A sprint is comprised of a collection of features you ideally will complete in a fixed period of time, but you may not necessarily do so by the release date. Features and issues may change sprints during a release (or even change releases), and, once a team reaches a release date, any incomplete features may move out of their active Sprints into the product backlog to be reassigned.
Also, it is important to understand the basic motivation behind integrating qTest with Rally. Rally, unlike other tools such as Jira, has native Test Management functionality. Inside Rally, you will find Test Cases and Test Runs and the ability to track and modify them. However, qTest offers much more Test Management functionality. For example:
- In Rally, there are no dedicated Test Steps attached to a Test Case. In order to track Test Steps, you would have to manually create them and then dump them all into a basic description field, whereas in qTest there are dedicated fields and functions to track Test Steps for a more complete approach to Test Case Management.
So, the qTest integration is the right step to bolster and simplify your team's overall Test Management abilities by taking the place of Rally's more limited Test Management offerings. All the while, Rally will still be used for Development Backlog, Requirement, and Defect management.
Example Rally Structure
In order to make the best use of your Rally and qTest integration, you must organize and structure your Rally environment in an efficient manner.
To begin with, let's take a look at the basic hierarchical structure of a Rally environment:
- Workspace (This is the highest level container available)
- Child Projects
A workspace is made up of Projects, and those Projects can have child projects. How you make use of this structure is going to heavily weigh upon how you organize and structure your development efforts in Rally. Whichever way you structure your Rally environment will then influence how you should integrate Rally with qTest.
In this article, we will be referring to a standard example where all development efforts are structured as a Project under one Workspace in Rally. The Development Project is broken up into various initiatives housed within Child Projects.
Each Release is divided into Sprints. This release has three Sprints as seen below in the 'Iterations' section of the screen:
Sprints also called an iteration, are comprised of items from the Backlog. As teams begin their Sprint/Iteration scheduling and planning, the team could:
- group them by Portfolio,
- assign work to specific teams,
- further break down Backlog Items into specific tasks,
- and estimate their effort to complete them.
Any items not assigned to a Sprint remains in the Backlog until they can be committed to a Sprint effort.
Configuring the Integration in qTest
After you successfully set up the connection with your Rally instance, you can begin the integration configuration by setting up the defect integration.
For detailed instructions on how to configure defects with Rally, please refer to the setup steps described in the following article: Configuring Defects with Rally
In this example, we are configuring the integration to allow our qTest users to log defects both at the root project level, "Sample Project," and at the Child Project level, "Sample Child Project."
We recommend you add the overall project and each child-project as a defect type so that your testing team can select the appropriate option when logging an issue at the time of test failure. But, if you want to limit users to only log defects at the root project folder or solely into a specific release this can be configured here.
Next, we will configure the Requirement integration.
For detailed instructions on how to configure Requirements with Rally, please refer to the setup steps described in the following article: Configuring Requirement Integration with Rally
In this example, we are only adding requirement types for our Release, "Sample Project" and not for the entire Workspace, "Workspace 1."
The reason for this is because we only want our testing team to work on items that are committed to a release. When and only when Backlog Items are assigned to a specific release will they be imported in qTest automatically.
Organizing your qTest Project
Create your Releases in Test Plan
You will initially want to create a Release folder in Test Plan for each Release in your Rally project. In this example, we have one Release and we will title it "Sample Release 1." For each new Release that is created in Rally your qTest team will need to create a Release folder in the Test Plan tab to ensure that your qTest project is aligned to the same release cadence as your development team in Rally.
View Imported Rally Requirements
Upon initial import, Rally objects will be treated as Requirements in qTest Manager. By default, these integrated requirements will be loaded into a common Module folder named 'Imported from Rally' as shown below:
Re-organizing your Imported Rally Requirements
You will first want to re-organize your initial imported requirements. When integrating with active Rally projects, this folder could contain hundreds of requirements, but, ideally, this integration and initial import should be configured at the very beginning of each new Rally project or Release. Therefore, the process is more concentrated on managing the integration than an initial organization effort.
When organizing the initial import, we suggest bucketing your requirements at a single property level so the management efforts are mitigated. In this example, we are going to organize our Requirements by their specific Sprint value. Backlog Items are less likely to change Sprint values as often as they may change status or team, so if you organize requirements by more than one level e.g. (by Sprint, Team and Status), your qTest users will be spending too much time keeping your requirement folders up to date.
We are going to create a folder outside of the 'Imported from Rally' folder for each of our Sprints. This way you can think of the 'Imported from Rally' folder as the container for any new Backlog Item that comes into scope for you release that then needs to be placed in the appropriate Sprint folder. You want to create your Sprint folders outside of the 'Imported from Rally' folder and not nested within as sub-folders. Create your Sprint folders as seen below:
Next, you need to drag/drop your imported Rally requirements into the appropriate Sprint folders:
Maintaining your Imported Rally Requirements
Now that you have updated your existing Backlog Items into their appropriate Sprint folders, your qTest users will need to manage existing issues and organize any new Backlog Items that come into scope. This should require minimal effort if maintained properly.
The image below shows a new backlog item being created in Rally.
When your development team creates new stories in the Rally backlog and saves, they will automatically appear in qTest in the 'Imported from Rally folder', as seen below:
This story will remain in the Backlog in Rally until it has been assigned to a Sprint. In the same vein, the Requirement should remain in the 'Imported from Rally' folder in qTest until it has been assigned to a Sprint and at that point, you should drag/drop it into the appropriate Sprint folder in qTest.
Your qTest users will need to periodically check the Requirements in the backlog ('Imported from Rally' folder) and move them to the appropriate Sprint folder once they have been assigned a Sprint value.
You do not have to Re-Retrieve each Requirement to get the updated data of Rally objects using the 'Retrieve Data' button on the requirement. Doing this would work, but the integration should pull new objects from Rally into qTest every minute or two.
HINT: Currently, all Rally fields cannot be used to Search or Query in qTest Manager, but you can Search for the name of a Rally Requirements by using double quotes.
Clicking on the search results will take you to that object in qTest Manager.
Optional: Maintaining your Release Scope in Test Plan
In the Test Plan, you may want to take the time to populate and maintain the Release Scope for each Release object.
For detailed instructions on how to add Requirements to your Release please refer to the setup steps in the following article: Add Requirements to your Release
By adding Backlog Items to their appropriate Release folder in Test Plan you are ensuring you have full end-to-end traceability between objects in qTest. Several Reports in qTest Insights are driven off the Release Scope and you can also plan your Test Runs in the Test Execution Tab of qTest Manager using this Release traceability.
One limitation may be the amount of time this may take to manage if you have large Releases. You can still generate the same reports in qTest Insights that are driven from this traceability but they may need to be manually created and not provided out-of-the-box.
The other limitation is when planning Test Runs your qTest users will not be able to add Tests based on the Release Traceability. They will not have the option to add Test Runs based on their linked Requirements in a specific Release (any requirement that has linked test cases that are part of the Release Scope will be available for selection).
This is not a critical piece of functionality, but simply something to consider when deciding if it’s worth taking the time and effort to manage the Release Scope in Test Plan.
However, if all Backlog Items in a Sprint are tied to a single Release this can easily and quickly be managed by selecting the entire Sprint folder when adding Requirements as seen in the screenshot above.
Linking Test Cases to Rally Requirements
Imported Rally objects can be linked to qTest Manager Test Cases. The linking with qTest Manager Test Cases will be generated in the corresponding Rally object's Links section.
For detailed instructions on how to link Test Cases and Requirements, please refer to the setup steps described in the following article: Linking Test Cases and Requirements
When you link or create associated Test Cases to an Imported Rally object, qTest Manager also generates a link under the corresponding Rally object. To quickly navigate to the Backlog Item in Rally click on the ID hyperlink on the Requirement in qTest.
Submitting Defects to Rally
You can submit defects to Rally from qTest Manager during TestPad Execution or from the Test Log.
For detailed instructions on how to submit Defects to Rally, please refer to the setup steps described in the following article: Submitting Defects to Rally
qTest users can submit a defect during test execution using the Quick Run or Test Pad window. Optionally they could use the Test Log in Execution History to submit a defect after executing a test run as seen below:
Test Pad window:
Test Log in Execution History:
The Rally Defect Submission page will display.
Specific fields could be auto-populated with qTest information (e.g. the Test Steps in the Description field) depending on your integration configuration.
After clicking save and submitting the Defect into Rally, your qTest users will be returned back to qTest Manager.
qTest users should be able to quickly reference the linked Rally Requirements in their active qTest Manager window.
Once the BackLog item has been assigned to the Defect it should be visible as a link in the Rally Defects tab.
Now your qTest users can return to the Defect window on the Test Pad or Test Execution tab of qTest Manager to close it and return to testing.
Review Rally Defect After Submission
Review Rally Defects in Manager
In qTest Test Execution, you can find Rally defect information in many areas:
To view Rally defects for a single Test Run, select the Defect icon from the Test Run grid.
For an aggregate list, select the Defect Summary tab for a roll-up of all Rally defects related to associated Test Runs. This Defect Summary tab can be customized so that you can display additional Rally Defect fields as columns.