Menu

Traceability and Coverage within your Agile Testing Environment Using Jira

Overview

Traceability is a fundamental component of what qTest provides your Agile environment. By centralizing all of your Test Cases (including manual, BDD, various automated, and even exploratory) within qTest Manager, you are able to provide uniform traceability and coverage across your Requirements. qTest is the central repository of all test assets, regardless of how or where they are executed. This can be seen at a high level in the below diagram. 

1.png

Requirements and Test Cases

Within qTest, traceability follows both a linear and web-like path. In the linear direction, each Release (Jira Sprint or Fixed Version) is defined by any number of Requirements (Jira Epics, User Stories etc). It is important to remember that this traceability is actually established within Jira and integrated over into qTest automatically. These Requirements are then associated with any number of Test Cases. Keep in mind that this mapping is flexible and:

  • One Requirement might be mapped to multiple Test Cases
  • Multiple Requirements might be mapped to the same Test Case

The association between Requirements and Test Case objects can be facilitated in multiple ways. From both the Requirement and Test Case Object, a user has the ability to add the respective objects to one another. In the below screenshot, on the left-hand side, you will see the Requirement Object in qTest, and, on the right-hand side, you will see the Test Case Object in qTest. Notice the Resources area towards the bottom section of the screen and the +Add icon to link existing objects:

 2.png

In addition to mapping existing objects within qTest, a user has the ability to create a Test Case object directly from the Requirement object itself. This Test Case will then be automatically associated with the Requirement. This prevents a user from needing multiple screens open, and, rather, to create a new Test Case directly on the Requirement Object. In the below screenshot, notice the bottom section of the Requirements Module:

3.png

Test Cases and Test Runs

Test Case Objects are in turn associated with and executed as Test Run Objects in the Test Execution Module. A Test Run is automatically associated with the Requirement and Release that the Test Case is subsequently linked to. Each Test Case (whether automated or manual) can have multiple Test Runs. For example, you might execute the same parameterized Test Case multiple times – one time for Chrome, one time for Firefox, and one time for Internet Explorer. In this instance, you would have one Test Case object, and three Test Run objects. Another example might be executing the same Test Case across multiple Release, Test Cycles, or Test Suites. Regardless, the traceability between Test Casse and Test Runs is automatically established.

Defects

The final area of traceability to discuss is Defects. During the execution of any type of Test Run, when a Defect is raised, it will automatically be linked back to the original Requirement and Release object the Test Case was associated with. qTest completes the circle of linear traceability, bridging multiple object types in Jira.­­­ In the below screenshot, you will see the Defect Summary of a specific Test Run from qTest:          

4.png

Exporting Traceability Reports

Traceability's linear fashion across qTest and Jira can be visualized from any object, at any time. Within qTest, a user has the ability to export any number of traceability matrixes. These include:

  • Requirements Detail report
  • Requirements Traceability matrix
  • Requirements and Test Cases Report
  • Requirements and Test Execution Reports (with and without cell merging)

Furthermore, similar reports can be exported from the Test Case Objects, including:

  • Test Case Detail Report
  • Test Case Traceability Matrix
  • Test Case and Requirements Report

Lastly, similar reports can be exported from the Test Run Objects, including:

  • Test Run Detail Report
  • Test Execution and Defects Reports (with and without cell merging)
  • CI Tool Integration Report

Traceability in qTest Insights

All traceability and coverage is available within the qTest Insights reporting and analytical module. qTest Insights can be leveraged to visualize the big picture of traceability and coverage across multiple projects or releases, as well as granularly and specifically to individual features or Test Cycles/Suites.

The Coverage Analysis is a pre-built report as seen below:

 5.png

The heat map on the left-hand side of the screen is showing the coverage of each requirement. Each rectangle represents a specific Requirement. The size of the rectangle is dictated by the respective total number of associated Test Runs. The color indicates potential risk per requirement. The risk is calculated by the tallying the failures, defects, and unexecuted Test Runs that can be traced to a specific requirement.

Coverage Analysis: Tabular View

Under the Coverage Analysis, a detailed breakdown of Coverage can be found in a tabular view. This table provides even more dynamic detail regarding the specific coverage of both Automated and Manual Test Cases.

6.png

On top of these pre-built reports, qTest Insights includes over 60 native metrics to select from in order to build out different dashboard views for different users and roles. Many of these metrics and KPI’s are rooted in the established traceability between Jira, qTest, and heterogeneous Automation environments. For more information on qTest Insights Dashboards, refer to this article

Powered by Zendesk