CI/CD integrations user guide

Last updated May 5, 2025

Testing multiple packages

Prerequisites:

Create multiple packages or test sets.
Run the CLI exe (for Windows) or dll (for Linux) file.
Add the required Orchestrator API access application scopes.

To test multiple packages, take the following steps:

Run the test task, using uipcli test. You can notice a list of the available sub-tasks.

Specify the required parameters and run the parallel sub-task.

The following table describes the required parameters:

Parameter	Description
`--testsConfigurationFilePath`	Required. Path to file with test projects configurations. This configuration outlines the list of projects that you want to package and then execute in parallel, with certain execution options. Each project can be configured with the following options: `ProjectPath`: The path to the project on the agent running the task. This can be a relative or absolute path. `AttachRobotLogs`: If set to `true`, the robot logs will be included in the test report file. `ContinueOnError`: Determines if the pipeline task should fail in the event of associated test failures. `RetryCount`: The number of retry attempts per test project. `Timeout`: Timeout per test project in seconds (default is 7200). `ParametersFilePath`: The path to a JSON file that contains input parameters for the test. This is similar to the parameters file used for the UiPathTest task. Test package metadata: `ReleaseNotes` `RepositoryUrl` `RepositoryCommit` `RepositoryBranch` `RepositoryType` `ProjectUrl` This configuration can also be formatted in JSON or CSV. JSON schema example: [ { "ProjectPath": "path/to/project", "AttachRobotLogs": true, "ContinueOnError": false, "RetryCount": 3, "Timeout": 7200, "ParametersFilePath": "path/to/parameters.json", "ReleaseNotes": "release notes for the test project", "RepositoryUrl": "http://repositoryurl.com", "RepositoryCommit": "commit hash", "RepositoryBranch": "branch name", "RepositoryType": "git", "ProjectUrl": "http://projecturl.com" }, { "ProjectPath": "path/to/project2" }, { "ProjectPath": "path/to/project3", "ParametersFilePath": "path/to/parameters3.json", } ][ { "ProjectPath": "path/to/project", "AttachRobotLogs": true, "ContinueOnError": false, "RetryCount": 3, "Timeout": 7200, "ParametersFilePath": "path/to/parameters.json", "ReleaseNotes": "release notes for the test project", "RepositoryUrl": "http://repositoryurl.com", "RepositoryCommit": "commit hash", "RepositoryBranch": "branch name", "RepositoryType": "git", "ProjectUrl": "http://projecturl.com" }, { "ProjectPath": "path/to/project2" }, { "ProjectPath": "path/to/project3", "ParametersFilePath": "path/to/parameters3.json", } ] CSV example: `ProjectPath,AttachRobotLogs,ContinueOnError,RetryCount,Timeout,ParametersFilePath,ReleaseNotes,RepositoryUrl,RepositoryCommit,RepositoryBranch,RepositoryType,ProjectUrl path/to/project,true,false,3,7200,path/to/parameters.json,release notes for the test project,http://repositoryurl.com,commit hash,branch name,git,http://projecturl.com path/to/project2,,,,,,,,,,, path/to/project3,,,,,path/to/parameters3.json,,,,,,`ProjectPath,AttachRobotLogs,ContinueOnError,RetryCount,Timeout,ParametersFilePath,ReleaseNotes,RepositoryUrl,RepositoryCommit,RepositoryBranch,RepositoryType,ProjectUrl path/to/project,true,false,3,7200,path/to/parameters.json,release notes for the test project,http://repositoryurl.com,commit hash,branch name,git,http://projecturl.com path/to/project2,,,,,,,,,,, path/to/project3,,,,,path/to/parameters3.json,,,,,,
`--projectsRootDirectoryPath`	Path to the root directory of projects to be tested. Required when `projectPaths` from the `testsConfigurationFile` are relative.
`--executionArtifactsDirectoryPath`	Directory path where to publish test execution artifacts.
`--cliDirectoryPath`	Path to the CLI directory or `uipcli.dll`. Required if executing CLI through the `dotnet` executable, optional when executing `uipcli.exe`.
`--out`	Required. Type of the result file <junit\|uipath>.
`--disableBuiltInNugetFeeds`	Disable built in nuget feeds used at test projects packing.
`-u, --username`	Required. The Orchestrator username used for authentication. Must be used together with the password.
`-p, --password`	Required. The Orchestrator password used for authentication. Must be used together with the username.
`-t, --token`	Required. The Orchestrator refresh token used for authentication. Must be used together with the account name and client id.
`-a, --accountName`	Required. The Orchestrator organization name. Must be used together with the refresh token and client id.
`-A, --accountForApp`	The Orchestrator organization name. Must be used together with id, secret and scope(s) for external application.
`-I, --applicationId`	Required. The external application id. Must be used together with organization name, secret and scope(s) for external application.
`-S, --applicationSecret`	Required. The external application secret. Must be used together with organization name, id and scope(s) for external application.
`--applicationScope`	Required. The space-separated list of application scopes. Must be used together with organization name, id and secret for external application.
`-o, --organizationUnit`	The Orchestrator folder name.
`-l, --language`	Language for the translated logs.
`--traceLevel`	Specifies what types of log messages to output One of the following values: None, Critical, Error, Warning, Information, Verbose. (default is Information).
`--identityUrl`	URL of your identity server. This is only required for PaaS deployments.
`Orchestrator URL`(pos. 0)	Required. The URL of the Orchestrator instance.
`Orchestrator Tenant`(pos. 1)	The tenant of the Orchestrator instance.

Usage:

./uipcli.exe test parallel <orchestrator_url> <orchestrator_tenant> [--testsConfigurationFilePath <tests_config_file_path>] [--projectsRootDirectoryPath <test_projects_root_directory_path>] [--executionArtifactsDirectoryPath <execution_artifacts_directory_path>] [--cliDirectoryPath <uipcli_dll_path>] [-u <orchestrator_user> -p <orchestrator_pass>] [-t <auth_token> -a <account_name>] [-A <organization_name> -I <application_id> -S <application_secret> --applicationScope <application_scope>] [-o <folder_name>] [-l <language>] [--out <junit|uipath>] [--disableBuiltInNugetFeeds] [--traceLevel <None|Critical|Error|Warning|Information|Verbose>] [--identityUrl <identityUrl>]./uipcli.exe test parallel <orchestrator_url> <orchestrator_tenant> [--testsConfigurationFilePath <tests_config_file_path>] [--projectsRootDirectoryPath <test_projects_root_directory_path>] [--executionArtifactsDirectoryPath <execution_artifacts_directory_path>] [--cliDirectoryPath <uipcli_dll_path>] [-u <orchestrator_user> -p <orchestrator_pass>] [-t <auth_token> -a <account_name>] [-A <organization_name> -I <application_id> -S <application_secret> --applicationScope <application_scope>] [-o <folder_name>] [-l <language>] [--out <junit|uipath>] [--disableBuiltInNugetFeeds] [--traceLevel <None|Critical|Error|Warning|Information|Verbose>] [--identityUrl <identityUrl>]

Examples:

./uipcli.exe test parallel "https://orch-23-10-paas.cloudapp.azure.com/" "TenantName" -u "orchestratorUsername" -p "********" -o "FolderName" --traceLevel Information --testsConfigurationFilePath "C:\userprofile\pipeline-repo-folder\test-configuration.json" --projectsRootDirectoryPath "C:\userprofile\test-projects-repo-folder" --executionArtifactsDirectoryPath "C:\userprofile\pipeline-execution-folder" --cliDirectoryPath "C:\userprofile\cli-download-path\UiPath.CLI.Windows\tools\uipcli.dll" --out "junit"
./uipcli.exe test parallel "https://orch-23-10-paas.cloudapp.azure.com/" "TenantName" -I "76000552-3e4f-4590-9317-cdb420001f1d" -S "********" --applicationScope "OR.Folders OR.BackgroundTasks OR.TestSets OR.TestSetExecutions OR.TestSetSchedules OR.Settings.Read OR.Robots.Read OR.Machines.Read OR.Execution OR.Assets OR.Users.Read OR.Jobs OR.Monitoring" --identityUrl "https://orch-23-10-paas.cloudapp.azure.com/identity" -o "FolderName" --traceLevel Information -P "C:\userprofile\AutomationProjectWithTestCases\project.json" --testsConfigurationFilePath "C:\userprofile\pipeline-repo-folder\test-configuration.json" --projectsRootDirectoryPath "C:\userprofile\test-projects-repo-folder" --executionArtifactsDirectoryPath "C:\userprofile\pipeline-execution-folder" --cliDirectoryPath "C:\userprofile\cli-download-path\UiPath.CLI.Windows\tools\uipcli.dll" --out "junit" "C:\userprofile\AutomationProjectWithTestCases\project.json"
./uipcli.exe test parallel "https://automation-suite.base.url.com/" "TenantName" -A "organizationName" -I "3f6239b9-e0e8-465e-a429-d9ffd1d9e57a" -S "********" --applicationScope "OR.Folders OR.BackgroundTasks OR.TestSets OR.TestSetExecutions OR.TestSetSchedules OR.Settings.Read OR.Robots.Read OR.Machines.Read OR.Execution OR.Assets OR.Users.Read OR.Jobs OR.Monitoring" -o "FolderName" --traceLevel Information --testsConfigurationFilePath "C:\userprofile\pipeline-repo-folder\test-configuration.json" --projectsRootDirectoryPath "C:\userprofile\test-projects-repo-folder" --executionArtifactsDirectoryPath "C:\userprofile\pipeline-execution-folder" --cliDirectoryPath "C:\userprofile\cli-download-path\UiPath.CLI.Windows\tools\uipcli.dll" --out "junit"
./uipcli.exe test parallel "https://cloud.uipath.com/" "TenantName" -A "organizationName" -I "becc663c-8f1e-409a-a75f-c00330d80bc8" -S "********" --applicationScope "OR.Folders OR.BackgroundTasks OR.TestSets OR.TestSetExecutions OR.TestSetSchedules OR.Settings.Read OR.Robots.Read OR.Machines.Read OR.Execution OR.Assets OR.Users.Read OR.Jobs OR.Monitoring" -o "FolderName" --traceLevel Information --testsConfigurationFilePath "C:\userprofile\pipeline-repo-folder\test-configuration.json" --projectsRootDirectoryPath "C:\userprofile\test-projects-repo-folder" --executionArtifactsDirectoryPath "C:\userprofile\pipeline-execution-folder" --cliDirectoryPath "C:\userprofile\cli-download-path\UiPath.CLI.Windows\tools\uipcli.dll" --out "junit"

Was this page helpful?

PREVIOUSTesting a package or running a test set

NEXTDeploying assets to Orchestrator

Support and Services

Get The Help You Need

UiPath Academy

Learning RPA - Automation Courses

UiPath Forum

UiPath Community Forum

Trust and Security

Cookies Policy