- Getting Started with Test Suite
- Studio
- Orchestrator
- Testing robots
- Test Manager
Test Suite User Guide
Reports
Understand the progress, detect problems at a glance, and review KPIs through the reports.
The following report types are available:
- Dashboard reports that provide daily and weekly breakdown of test results, including passed/failed ratio, test execution list, and KPIs, such as the total coverage, or open defects.
- Reporting service through ODBC or native SQL Server connections, so you can use the reporting tool of your choice to create custom reports and additional dashboards.
You can examine the reports of each project individually. To view dashboard reports, open Test Manager and select a project from the list to view the Dashboard page.
|
Section Number |
Report |
Description |
|---|---|---|
|
1 |
Results |
Bar chart displaying the results of your test runs. To switch between daily and weekly views, click the focus indicator, at the upper-right side of the dashboard and choose an option from the dropdown. The weekly report goes back 14 weeks. |
|
2 |
Current Day/Week Indicator |
Day/Week indicator of the results shown in the bar chart. This section is not captured in the above figure. |
|
3 |
KPIs |
Open Defects is available only for connected projects. For example, defect reporting from the Jira project connected to Test Manager. The number of defects defined with the highest priority in Jira are shown in the Critical section. You can click the hyperlink to open the Jira filter. If you unlink a connection, Open Defect will not be shown anymore. Total Coverage The percentage of requirements that have at least one test case out of the total number of requirements. Automation Rate The percentage of test cases without automation/manual steps defined, as well as the automated, and manual test cases, respectively. |
|
4 |
Latest Results |
Latest test executions that contributed to your test results displayed in the bar chart. |
For example, you have a high number of defects that need to be dealt, alongside an equally high number of manual test cases.
You can glance over the dashboard to check the number of defects, where you can click on the Critical section within the Open Defects report, to open the filter in Jira. The Automation Rate report indicates the degree of automation of your test project. Using this information, you can start reducing the number of manual test cases, so that automated test cases come ahead, and allocate more testers to deal with the high number of defects.
You can examine the reports of each project individually. To view dashboard reports, open Test Manager and select a project from the list to view the Dashboard page.
|
Section Number |
Report |
Description |
|---|---|---|
|
1 |
Results |
Bar chart displaying the results of your test runs. To switch between daily and weekly views, click the focus indicator, at the upper-right side of the dashboard and choose an option from the dropdown. The weekly report goes back 14 weeks. |
|
2 |
Current Day/Week Indicator |
Day/Week indicator of the results shown in the bar chart. This section is not captured in the above figure. |
|
3 |
KPIs |
Open Defects is available only for connected projects. For example, defect reporting from the Jira project connected to Test Manager. The number of defects defined with the highest priority in Jira are shown in the Critical section. You can click the hyperlink to open the Jira filter. If you unlink a connection, Open Defect will not be shown anymore. Total Coverage The percentage of requirements that have at least one test case out of the total number of requirements. Automation Rate The percentage of test cases without automation/manual steps defined, as well as the automated, and manual test cases, respectively. |
|
4 |
Latest Results |
Latest test executions that contributed to your test results displayed in the bar chart. |
For example, you have a high number of defects that need to be dealt, alongside an equally high number of manual test cases.
You can glance over the dashboard to check the number of defects, where you can click on the Critical section within the Open Defects report, to open the filter in Jira. The Automation Rate report indicates the degree of automation of your test project. Using this information, you can start reducing the number of manual test cases, so that automated test cases come ahead, and allocate more testers to deal with the high number of defects.
If you enable an Insights integration in your Test Manager tenant, you can obtain analytics of all your testing projects within the tenant where you enabled the integration. Insights retrieves data from Test Manager, based on a specific data model, and offers two pre-defined dashboards:
- Test Manager Default(link) – This dashboard provides an overview of all your testing projects within the same tenant.
- Test Execution Overview(link) – This dashboard provides an overview of all your test executions within the same tenant.
The following tables represent the Test Manager - specific data model, used by Insights to generate analytics. For each test object, the tables show the dimensions:
| Name | Type | Description |
|---|---|---|
ProjectId | guid | The ID of the project |
Type | Application or RPA | The type of testing project, based on what you are testing: Application or RPA. |
CreationDate | date/time | The date when you created the project. |
| Name | Type | Description |
|---|---|---|
TotalCount | integer | The |
| Name | Type | Description |
|---|---|---|
ProjectId | guid | The ID of the requirement. |
CreationDate | date | The date when the requirement was created. |
| Name | Type | Description |
|---|---|---|
TotalCount | integer | The total number of whatever test object type you select. |
CoveredByAutomationTotal | integer | The number of selected test objects type covered by automated test case. |
Coverage | integer | The total coverage?? |
