Menu

Tests Planned vs Tests Executed Report

Overview

The Tests Planned vs Tests Executed Report allows teams to optimally track their testing progress. This report tracks the actual number of tests executed vs the number of tests planned over an allotted amount of time.

Prerequisites

Users will need the following before being able to access and configure Tests Planned vs Tests Executed Reports:

  • qTest Insights access
  • Insights viewer permission (for viewing only)
  • Insights editor permission (for viewing and configuring)

Tests Planned vs Tests Executed Report Video

Hint: Watch the video in full screen for the best picture quality. 

Configure Your Report

Before being able to access and make the best use of the report, users will need to properly configure and customize their report. To do so, follow these steps:

  1. Access qTest Insights via the qTest app navigator. 
  2. In qTest Insights, select the Settings tab. From the drop-down menu, select Tests Planned vs Executed.
    Access_settings.png
  3. The "Tests Planned vs Executed" page displays. Select the Add icon. The "Planned vs Executed Report Settings" dialog displays. 
  4. Configure your report's basic settings:
    • Name: Define the name of your report
    • Project: Select the project you wish to build a report for
      IMPORTANT: In order for this report to function properly, the project you choose must use the "Test Run Status" site-field. For more information on Site Fields, refer to this article.
    • Artifact: Select the artifact within which you want to track your progress
    • Artifact value: Select the value from within your Artifact that you want to track
    • Date range type: Define just how granular you want your report to be as it pertains to reporting frequency.
      For example, if you want to track progress week by week, select the week option.
    • Date Range: Select the range for which you want your report to track progress. Once done, select the Add Dates icon.
      In this example
      , I have configured my report so that I will track weekly Tests planned vs executed between November 30th and December 30th. 
      IMPORTANT: These dates are not associated with the planned start and end dates set in qTest Manager for Test Runs.
    • Enter the number of planned executed test runs and the number of planned tests runs to be passed for each duration. To do so, qTest Insights gives you two options:
      • Same Settings Population Type: To quickly and easily set the number of planned vs executed Test Runs for each duration, you can choose the Same Settings option. This will result in each duration having the same amount of planned Test Runs vs Executed Test runs. After selecting your Planned Executions and Planned Status of Passed, select Apply to All.
        Populate_same_type.png
      • Individual Settings Population Type: If you are planning for each duration of time not necessarily having the same amount of planned executions vs planned statuses of passed, you can choose the Individual Settings option. If so, define your number of Planned Executions and Planned statuses of passed individually for each duration.
        Indidivual_settings.png

  1. When you are finished configuring your Report Settings, select Save.
  2. Select the Test Run Status(es) that you want to represent passed Tests from the drop-down menu. These statuses are pulled from field templates associated with your already-selected selected Projects and artifacts. Select Save.
    Statuses.png

Access and Navigate Your Report

After the configuration is finished, users can view the report. Follow these steps:

  1. Select the Report Gallery tab.
  2. In the Report Gallery, select the Tests Planned vs Executed tag. 
  3. The "Tests Planned vs Executed" page displays. Use the Select Report drop-down menu to select your already-configured report.
    Select_your_report.png

Note: The Global Filter is not available on this report due to your already configuring the report's Projects and Artifacts.

Users' Tests Planned vs Tests Executed reports now display. The "Test Runs Planned vs Executed" Line graph simply displays the relationship between planned executions and passed executions. The "Test Runs Planned vs Executed Details" grid gives users more in-depth detail associated with the planned executions vs passes. 

Grid.png

Note: The color blocks next to the Achieved Execution and Achieved Status of Passed sections in the grid are threshold color indicators. These colors and their threshold rules are system pre-defined and serve as a quick visual indicator of success. The system pre-defined rules and their associated colors are as follows:

  • Green will be assigned if the following is true:
    • If the actual number of Tests executed, when compared to the expected number, is greater than 90% 
    • If the actual number of Tests passed, when compared to the expected number, is greater than 90%
  • Yellow will be assigned if the following is true:
    • If the actual number of executed, when compared to the expected number, is greater than 80% but less than 90%
    • If the actual number of passed, when compared to the expected number, is greater than 80% but less than 90%
  • Red will be assigned if the following is true:
    • If the actual number of executed Tests, when compared to the expected number, is less than 80% 
    • If the actual number of passed, when compared to the expected number, is less than 80%

IMPORTANT: These threshold rules and their associated colors are not editable at this time.

Powered by Zendesk