Executing automated tests

link

• At the tenant level: Folders.View and Users.View.
• At the folder level, where the test automation package was published: TestSets.Create and TestSetExecutions.Create.

• Test execution for test sets linked from Orchestrator includes only the test cases published to the Orchestrator Tenant Process Feed in the latest package version. To execute new test cases, publish a new package version in Orchestrator, and link the test set to Test Manager.
• The following information is applicable only if you are using the legacy execution context in Test Manager: If your Test Manager test set includes test cases from multiple Studio projects, then the project that has the highest number of test cases is considered for execution. As a result, only the test cases from this project will be executed. Additionally, Test Manager will display a warning stating that multiple packages have been selected, and only some of them will be executed. You can execute test sets including test cases from multiple Studio projects, only if you have a test set linked from Orchestrator.
• If your test set includes duplicate test cases in multiple Studio projects, the sorting is done by the project name in descending order. Only the test cases from the project that is placed first in the list will be executed.

1. Open your project in Test Manager and then navigate to Test Sets.

2. Select a test set and select the vertical ellipsis.

3. Select Execute Automated.

   For more information on test results, see Analyzing Test Results.

   Note:

   • Test cases that cannot be executed are skipped during execution through Orchestrator.
   • Test cases that are not executed generate test case logs with no result.

1. Locate the configmap file in your Orchestrator Automation Suite installation folder.
2. Open the file, and add the TestAutomation.TestSetExecutionTimeoutMinutes setting.
   assignment

   <add key="TestAutomation.TestSetExecutionTimeoutMinutes" value="40" />
   <add key="TestAutomation.TestSetExecutionTimeoutMinutes" value="40" />
   

3. The value of the TestAutomation.TestSetExecutionTimeoutMinutes setting is the required timeout duration. The value, expressed in minutes, represents the duration after which your test executions are marked as Cancelled.

Prerequisites​

1. Ensure that your Test Manager role has the following permissions:
   • TestSets.Edit: To enable live streaming and remote controlling,
   • View live stream of automated executions: To watch the test execution.
   • Remotely control live stream of automated executions: To control the test execution remotely.
     Note:

     By default, the Test Manager project role has the View live stream of automated executions permission. But to remotely control the live stream, a Test Manager Administrator must manually assign this permission to users via a custom role.

2. Enable the live stream or the remote control capability at the test set level, for the desired test set:
   1. Open the test set or test execution.
      • For test sets: Go to the Configuration tab.
      • For test executions: Go to the Settings tab,
   2. Select the Enable Live Streaming toggle to enable the capture of a live stream of a test execution.
   3. Alternatively, if you want to be able to remotely control the running test execution, under Allow Remote Control select Enable.
3. Install a VNC viewer on your machine.

Viewing the live stream​

1. Trigger the execution of a test set.
2. In the Execution section, open the running test execution.
3. Go to the Results tab.
4. In the Video column, select Live.

Remote controlling a test execution​

1. In the live stream window, select Take remote control.
2. Use your local mouse and keyboard to control the robot session.

Prerequisites​

1. Enable the proper permissions.
   1. Orchestrator permissions - Enable the folder-level Execution Media permission, with the following options:
      • View - Lets the account see job execution recordings.
      • Create - Lets the account/role that executes the job also create the recording of the job execution.
      • Delete - Lets the account delete video recordings from Orchestrator.
   2. Test Manager permissions
      • TestExecution.View
2. Get familiar with the types of supported recordings and associated robots.
   • Options: Record and store failed executions (for 14 days), Record all executions (for 7 days)
   • The options are available for Robot version 2023.2 or later.

Enabling recording of test executions​

At test set level​

1. Navigate to Test Sets.
2. Select a test set and then select the Configuration tab.
3. Toggle on Record automated execution.
4. Under Recording type, select one of the options: Record and store failed executions (for 14 days) or Record all executions (for 7 days).

At test execution level​

1. Navigate to Execution.
2. Open an execution for which you want to have recording enabled.
3. In the Settings tab, toggle on Record automated execution.
4. Under Recording type, select one of the options: Record and store failed executions (for 14 days) or Record all executions (for 7 days).
   Tip:

   We recommend that you enable Record all executions for a brief amount of time so as to observe potentially odd behavior during job execution. You can then switch to only storing failed executions in order to efficiently use bandwidth and storage.

Viewing recording of test executions​

1. Navigate to Execution.
2. Trigger the execution of a test set.
3. Select the Recording tab.
4. Open a test case log and select the Recording tab to view the recording.
5. To download the recording, from the upper right-corner of the recording screen, select Download.