qTest Insights 1.2 On-Demand Release Notes - September 27th, 2016

Overview

qTest Insights is a complete reinvention of qTest reporting. Insights gives the team not only useful pre-built metrics but also a self-service intelligence tool to consolidate, manage and analyze all your test metrics. 

Features

1. Analysis Reports

Provide several pre-built cross-project reports in 3 different perspectives: Quality, Coverage, and Velocity. Quality is the default view of Analysis reports.

User can also save these pre-built Analysis Reports for later managing and scheduling.

1.1 Quality Analysis

Provide 3 charts and a flat grid view about Quality Analysis.

  • Chart "Latest Test Run Results": The results from the latest runs of each test case
  • Chart "Test Results by Day": A breakdown of the test run results by day
  • Chart "Defect Status": Show the distribution of defects by status. 

NOTE: The chart "Defect Status" with the most data will be shown. For example, if according to the current filters there are more (Linked) defects than (qTest) then the (Linked) version of the chart should be shown.

  • Analysis Grid with Tabs for multiple views of the data broken down by Project, Release, Test Cycle, Configuration.

1.2 Coverage Analysis

Provide 3 charts and a flat grid view about Coverage Analysis.

  • Chart "Test Runs Per Requirement": Number of Test Runs per Requirements.
  • Heat map "Requirements Test Run Coverage"
  • Chart "Requirements Run Results": The results from the latest run of each test case.
  • Analysis Grid shows the test cases for each requirement along with the testing results and number of defects.

1.3 Velocity Analysis

Provide 3 charts and a flat grid view about Coverage Analysis.

  • Chart "Test Planned vs. Executed": cumulative results by day for the # of test planned & executed & execution results
  • Chart "New Requirements and Test Cases": # of requirements and TCs (manual & auto) added by day
  • Chart "Defects Opened and Closed": # of defects opened and closed by day.

NOTE: The chart "Defects Opened and Closed" with the most data will be shown. For example, if according to the current filters there are more (Linked) defects than (qTest) then the (Linked) version of the chart should be shown.

  • Analysis Grid shows activities and results broken down by Testers

1.4 Functionalities on Analysis Grid

  • Sort by Column header
  • Click on the data in the grid to drill-down to details
  • Provide couples of estimation or forecast data like Forecasted Defects Remaining, Estimated Test Time Remaining

Example of Analysis Grid of Coverage Analysis Reports:

2. Dashboard 

2.1 Shared and Personal Dashboard

  • Shared dashboard is only be customized by user with Editor privilege. All team members with view access to Insights can see it.
  • Personal dashboard is user private space, they can set it up whatever they want.

2.2 Organize Dashboard

  • Add/remove/rename tab
  • Change layout, Drag & drop panels, resize panels
  • Add panel (system panel, custom panel), rename panel, remove panel

3. Global Filters & Refresh

3.1 Global Filters

  • Global Filters: User can multi-select projects/ modules/ releases/ people to filter the charts and analysis grid in Dashboard and Analysis Reports
  • These filters are global to the application, but they are personal to each user as they are saved into user cookies. That way, user can have their own current settings on the Shared or the Personal Dashboard. 

3.2 Refresh

  • Ability to automatically refresh data at user-specified interval (every 30, 60, 90 or 120 seconds)
  • Refresh Now: intermediately reload reports with latest data

4. Explore Data - Build Custom Reports

  • This feature allows user to build their own reports based on many different views: Requirements, Test Cases, Test Runs, Defects (qTest), Defects (Linked), Combined.
  • This feature is only available to users with Editor privilege. 

4.1 Provide multiple analysis grids

  • Requirements: Provide analysis view on Requirements data. 
  • Test Cases: Provide analysis view on Test Cases data.
  • Test Runs: Provide analysis view on Test Runs data.
  • Test Run Logs: Provide analysis view for each test run log result.
  • Defects (qTest): Provide analysis view on qTest Defects data.
  • Defects (Linked): Provide analysis view on Defects data from external tracker which are linked to qTest objects.
  • Combined: This view combines test run data with test case and requirement and defect data. It's especially useful to build traceability reports for example.

4.2 Functionalities on Analysis grid

  • Select which column you want to display on the grid
  • Ability to download grid data to several formats like Excel, CSV, PDF
  • Click on column name, you will have ability to sort, filter, group, aggregate, add to chart, etc.
  • Drag & drop column around the grid
  • Resize column width
  • Click linked data on the grid to drilldown to other dataset 
  • (Linked) data also supports going to the external system if you click on the Defect ID or Summary
  • Click thru to qTest for object names and IDs shown in the report: user can click to go back to qTest tab and view details on those objects 

Example of Analysis grid under Combined Data view:

4.3 Provide ability to build your own reports based on analysis grids mentioned above

  • Select fields to build your chart or crosstab
  • Ability to turn the chart into different chart type like Bar, Line, Curved Line, Pie, Heatmap, Gauge, Scatter Plot
  • Ability to create new column from a formula
  • Save and share their own reports to the Dashboard (add custom panel)
  • Save their own reports for later managing and scheduling

Example of How to create Crosstab:

5. Manage and schedule report

  • Rename your saved reports 
  • Show in Menu: Share your reports to everyone 
  • Send Now : Immediately send a scheduled report to your configured emails

  • Set Schedule to email the reports to recipients 

6. User Access to Insights

  • Insights is only available for user with qTest's Elite package
  • The administrator will need to grant Insights privileges to individual Admin Profile users.
  • Insights Privileges: Insights Viewer and Insights Editor 

  • Read-only Access to Insights Shared Dashboard:  the administrator has ability to generate a secure URL that can be given to others for read-only access to the Shared Dashboard.

Known Issues and Limitations

  • Do not have support for custom fields; custom values in default fields are supported, but support for custom fields will come in a future release.
  • Custom charts added to dashboards will not work with the filters available on the dashboard, they will just use whatever filter is built-in to the custom chart.
  • Printing from the app is not directly supported, to print the user will need to use the browser capabilities or download the data to a file format such as PDF.
  • All users can see all projects; the data in Insights is read-only, but any user with viewer or editor permissions will be able to see reports for all projects within the organization.
  • If the Quality Analysis report is saved and then scheduled, the scheduled report will only contain the charts and the default tab (the Project tab view), the other tabs are not included in the scheduled report.
  • If a user generates the read-only Shared Dashboard link and tries to use it, but the Shared Dashboard has not yet been created, user will get an error message when trying to access it.
  • Insights is determining that a test case is Automation by looking at the test case type field. However this field can be manually changed or customized so it might not be set to "Automation" and yet the test case still can be an automation test case.