7.3 Test Execution Reports

You can use the following reports to do statistics for Test Executions. They are grouped under category Test Execution Reports on the left menu.

»Test Execution Analysis

»Test Execution Analysis (Snapshot View)

»Execution Status With Defects

»Test Execution Summary

»Test Run Summary by Assignees

»Testing Velocity

Test Execution Analysis

This report provides the detailed statistics of Test Executions over different time ranges. It also allows users to filter by various and values so that you can track the overall performance of your QAs, the Test Execution progress or the environment being tested. . I.e. you can choose your desired-to-view reported data from the most recent 7 days, from the last 30 days, from the very beginning of your projects or in a very specific time frame.

Follow these steps to view this report:

  1. Select Test Execution Analysis Report from the menu on the left hand side.
  2. Select the filters in Report Settings section.
  3. Click on Show button to view the report.

There are four parts in the report:

A. Report Settings

This area allows you to specifically define your needs of reporting by using the combination of four criteria: Date range, Field, Value and Test Execution Type.

Date Range: depending on your selection, the report graph will display data as follow:

  • Last 7 days: Show data for the last 7 days, graph will show defects per day.
  • Last 30 days: Show data for the last 30 days, graph will show Defects per week.
  • Project to date: Show data from the very beginning of your project to the current point of time, graph will show the corresponding defects time frame depending on how long your project has gone.
  • From…To: You can select a specific time frame to show data. Data unit will be showed as the rules below:
    • If the selected duration is less than seven days, the report will display daily statistics.
    • If the selected duration is more than seven days and less than or equal to one month, the report will display weekly statistics.
    • If the selected duration is more than one month and less than or equal to five months, the report will display monthly statistics.
    • If the selected duration is more than five months and less than or equal to 12 months, the report will display quarterly statistics.
    • If the duration is more than twelve months, the report will display annual statistics.

Field: allows you to select one of a list of qTest Test Run’s fields for data filtering, including Status, Execution Type, Assignee and Environment.

Value: once a field has been selected, you can select its corresponding values from the drop-down list to finish specifying your filter criteria in the Report Settings area.

Test Execution Type: you can select either Manual or Automation or both of them.

B. Latest Information

The Latest Information area shows the current quantity and percentage of tests, grouped by the fields selected from the Report Settings area. This highlighted information gives you a quick view of all test executions in the system.

Test_Execution_3.png

C. Chart and Legends area

After specifying your data filter criteria, you can click on Show button to view your test execution data reported in form of a Column chart. Below the chart, there are legends displayed based on the fields and values that previously selected in the Report Settings area.

Test_Execution_4.png

D. Data Grid

In addition, this report provides a data grid which allows you to examine the reported data up to the date level. This table is automatically refreshed to reflect the state of the Column charts (I.e. data grid will display weekly statistics if the Column chart displays weekly statistics).

The statistics figures of a specific date displayed on the graph is the snapshot of Test Executions’ states at the end of that date.

On the row that displays the dates being reported, you can click on each date cell to view details of the tests of the selected date.

TX_Analysis_1.png

TX_Analysis_2.png

You can export the grid to Excel file for further reporting purpose. It can exported to different formats such as XLS, PDF, CVS. Click on Download button to download the report.

Test Execution Analysis (Snapshot View)

They report the activities recorded in a day, a week, a month, a quarter or a year instead of continuously aggregated numbers. You can choose your desired-to-view reported data from the most recent 7 days, from the last 30 days, from the very beginning of your projects or in a very specific time frame. It helps answer some questions such as: which test runs have been executed and failed within each day in the last 7 days,...

Follow these steps to view this report:

  1. Select Test Execution Analysis (Snapshot View) Report from the menu on the left hand side.
  2. Select the filters in Report Settings section.
  3. Click on Show button to view the report.

There are four parts in the report:

A. Report Settings

This area allows you to specifically define your needs of reporting by using the combination of four criteria: Date range, Field, Value, and Test Execution Type.

Date Range: depending on your selection, the report graph will display data as follow:

  • Last 7 days: Show data for the last 7 days, graph will show defects per day.
  • Last 30 days: Show data for the last 30 days, graph will show Defects per week.
  • Project to date: Show data from the very beginning of your project to the current point of time, graph will show the corresponding defects time frame depending on how long your project has gone.
  • From…To: You can select a specific time frame to show data. Data unit will be showed as the rules below:
    • If the selected duration is less than seven days, the report will display daily statistics.
    • If the selected duration is more than seven days and less than or equal to one month, the report will display weekly statistics.
    • If the selected duration is more than one month and less than or equal to five months, the report will display monthly statistics.
    • If the selected duration is more than five months and less than or equal to 12 months, the report will display quarterly statistics.
    • If the duration is more than twelve months, the report will display annual statistics.

Field: allows you to select one of a list of qTest Test Run’s fields for data filtering, including Status, Execution Type, Assignee and Environment.

Value: once a field has been selected, you can select its corresponding values.

Test Execution Type: you can select either Manual or Automation or both of them.

B. Latest Information

The Latest Information area shows the current quantity and percentage of tests, grouped by the fields selected from the Report Settings area.

C. Chart and Legends area

After specifying your data filter criteria, you can click on Show button to view your test execution data reported in form of a Column chart. Below the chart, there are legends displayed based on the fields and values that previously selected in the Report Settings area.

D. Data Grid

In addition, this report provides a data grid which allows you to examine the reported data up to the date level. This table is automatically refreshed to reflect the state of the Column charts (I.e. data grid will display weekly statistics if the Column chart displays weekly statistics).

The statistics figures of a specific date displayed on the graph is the snapshot of Test Executions’ states at the end of that date.

On the row that displays the dates being reported, you can click on each date cell to view details of the tests of the selected date.

TX_Analysis_1.png

TX_Analysis_2.png

You can export the grid to Excel file for further reporting purpose. It can exported to different formats such as XLS, PDF, CVS. Click on Download button to download the report.

Execution Status With Defects

With this report, you can get a quick overview of the execution status of a specific Release with the total test cases executed and their associated defects.

Follow these steps to view this report:

  1. Select Execution Status With Defects Report from the menu on the left hand side.
  2. Select a specific Release and then select up to ten Modules in this Release. Select Test Execution Type.
  3. Click on Show button to view the report.

The report is displayed in three parts:

A. Latest Information

It shows the number of Test Cases executed and the number of Defects associated with the Test Execution.

Test_Execution_8.png

B. Chart and Legends area

It is a bar chart that shows the number of Defects and Test Cases in each selected Module. There is line that indicates the average number of Defects.

Test_Execution_9.png

C. Data Grid

It contained the detailed data of the Test Cases executed and their associated Defects.

 Test_Execution_10.png

You can export the grid to Excel file for further reporting purpose. It can exported to different formats such as XLS, PDF, CVS. Click on Download button to download the report.

Test Execution Summary

This Report provides the Test Execution details at the report run time. The report can be filtered by Release/Test Cycle/Test Suite.

Follow these steps to view this report:

  1. Select Test Execution Summary Report from the menu on the left hand side.
  2. Select a specific Release/Test Cycle/Test Suite. Select Test Execution Type.
  3. Click on Show button to view the report.

There are four parts in the report:

A. Report Settings

You can select a specific Release/Test Cycle/Test Suite to filter the report. Only one field and one value can be selected at a time.

You can also select one or both Test Execution Type.

B. Latest Information

It shows the current number of Test Runs and the detailed number of Test Runs being grouped by the latest execution results in the selected Release/Test Cycle/Test Suite.

C. Chart and Legends area

After specifying your data filter criteria, you can click on Show button to view your defect data reported in form of a Pie Chart. The chart shows the portions of Test Runs grouped by their latest execution results.

D. Data grid

In addition to the Chart, the report provides a detailed table of each Test Execution that meets your selected filter.

Test_Execution_15.png

You can export the grid to Excel file for further reporting purpose. It can exported to different formats such as XLS, PDF, CVS. Click on Download button to download the report.

Test Run Summary by Assignees

This Report provides the summary of Test Runs by Assignees. This Report can be filtered by Release, Test Cycles, Test Suite and Assignee.

  1. Go to qTest Reports and select Test Run Summary with Assignees
  2. Select specific Release/Test Cycle/Test Suite, Assignee or All and select Test Execution Type to run Report
  3. Click Show button to generate Report

Test Run Summary by Assignees include 4 parts:

A. Report Settings

This section allows you to select Release, Test Cycle, Test Suite, Assignee(s), and Test Execution Type to run Report. Corresponding values will display upon a specific filter.

      

button-round-warning-icon.pngHINT:

  • You can select either one Filter and Value or ALL to show Reports.
  • You can select one, multiple or all Assignee to show Report.

B. Chart with Latest Information

This section shows latest information of all Test Runs within selected criteria. Data are displayed in both statistics and pie chart.

C. Data Grid

Data Grid contains detailed Report of Test Run Summary, ordered by Releases, Test Cycles, Test Suites and Assignees.

D. Test Run Summary by Assignees Table

This table summarizes and groups Test Runs by Assignees and Test Run's statuses.

Testing Velocity

This Report provides the Rollup of time per Test Execution to a Test Suite/Cycle/Build/Release and the different between planned and actual time.

  1. Go to qTest Reports and select Testing Velocity
  2. Filter Report by Release, Build, Test Cycle, Test Suite, Test Run Type or ALL
  3. Select Time Unit to display Execution Time in Report
  4. Click Show button to generate Report

The Report displays two parts:

A. Summary with Latest Information

This section shows the latest information of execution time in 4 terms:

  • Total Planned Time: sum of planned execution time of all Test Run Logs in the selected Report Settings
  • Total Actual Time: sum of actual execution time of all Test Run Logs in the selected Report Settings
  • Planned Time: sum of planned execution time of all latest Test Logs of Test Runs in the selected Report Settings
  • Actual Time: sum of actual execution time of all latest Test Logs of Test Runs in the selected Report Settings

B. Data Grid

Data Grid contains detailed Report of Test Execution Time displayed in Summary section.

Reports can be emailed to internal or/and external users in different formats such as XLS, CSV, or PDF. Click Mail and enter Recipients' email(s) to send Report.