automation-suite

2024.10

true

Automation Suite on OpenShift Installation Guide

Last updated Jan 27, 2025

Performing Insights database maintenance

Overview

Insights Database works using two tables: dbo and read. Both tables store data from Orchestrator databases and pass data between each other. Over time, tables store a large amount of data that can impact database performance. You can free up space by removing data that is not relevant or outdated.

The estimated threshold for storing historical data is set to

200 million RobotLogs
100 million QueueItemEvents
100 million JobEvents
20 million QueueItems
20 million Job

Use the following database maintenance SQL scripts to create SQL Stored Procedures, then execute these SQL Stored Procedures to either delete all data from [dbo].Jobs, [dbo].JobEvents, [dbo].QueueItems, [dbo].QueueItemEvents, [dbo].RobotLogs tables, or just the [dbo].RobotLogs.

Note: Back up your database before executing the scripts. Before running the

PROCEDURE
                     [read].[Delete_Insights_Data_Read]

and

PROCEDURE
                     [read].[Delete_Process_Logs_Read]

scripts, the Insights module needs to be disabled.

Considerations

Scripts with the _DBO suffix are used to delete data from DBO schema tables,
Scripts with the _Read suffix are used to delete data from Read schema tables.
Large-sized batches might impact the speed of execution. Consider using the default batch size, which is set to 100000.

Prerequisites

Before running the PROCEDURE [read].[Delete_Insights_Data_Read] and PROCEDURE [read].[Delete_Process_Logs_Read] scripts, the Insights module needs to be disabled.

Log in to the VM where Orchestrator is installed.
Remove files that may conflict with the commands:
```
rm -f appsettings.jsonrm -f appsettings.json
```

Get the current settings:

Ignore the error message if any, it just implies there are no custom settings.

./bin/uipathctl config orchestrator get-config --app-settings > appsettings.json || echo '{}' > appsettings.json./bin/uipathctl config orchestrator get-config --app-settings > appsettings.json || echo '{}' > appsettings.json

Add the Insights.ModuleEnabled key and set it to False:

jq '.["Insights.ModuleEnabled"] = "false"' appsettings.json > temp.json && mv -f temp.json appsettings.jsonjq '.["Insights.ModuleEnabled"] = "false"' appsettings.json > temp.json && mv -f temp.json appsettings.json

Update the configuration:

./bin/uipathctl config orchestrator update-config --app-settings appsettings.json./bin/uipathctl config orchestrator update-config --app-settings appsettings.json

Validate the configuration:

./bin/uipathctl config orchestrator get-config --app-settings./bin/uipathctl config orchestrator get-config --app-settings

The expected result is as follows:

{
    "Insights.ModuleEnabled": "false"
}{
    "Insights.ModuleEnabled": "false"
}

The result can contain other settings unrelated to Insights.

Run the maintenance scripts.
After you run the maintenance script, re-enable Insights using the following steps.

Remove any conflicting files:

rm -f appsettings.jsonrm -f appsettings.json

Remove the Insights.ModuleEnabled key:

jq 'del(.["Insights.ModuleEnabled"])' appsettings.json > temp.json && mv -f temp.json appsettings.jsonjq 'del(.["Insights.ModuleEnabled"])' appsettings.json > temp.json && mv -f temp.json appsettings.json

Update the configuration:

./bin/uipathctl config orchestrator update-config --app-settings appsettings.json./bin/uipathctl config orchestrator update-config --app-settings appsettings.json

Confirm the configuration:
```
./bin/uipathctl config orchestrator get-config --app-settings./bin/uipathctl config orchestrator get-config --app-settings
```
The expected result is as follows:
```
{}{}
```
The result can contain other settings unrelated to Insights.

Resources

SQL Stored Procedures download location

SQL stored procedure 1: delete all data based on cut-off timestamp

Generate and save the following SQL Stored Procedures to delete all data types (Jobs, JobEvents, QueueItems, QueueItemEvents, RobotLogs) for a cut-off timestamp.

SQL Stored Procedures	Parameter	Description
Delete_Insights_Data_DBO	@CutoffTimeStamp `DATETIME` (Required) Delete all five types of data before a specified timestamp (e.g., `2021-01-01 01:00:05`). Do not provide a timestamp that deletes all data to an empty table. @BatchSize `INT = 100000` (Optional) Default is set to `100000`. Data is deleted batch by batch. For example, if you have 1 million RobotLogs and leave the default value, the delete command runs ten times to remove all data.	Delete data from `[dbo].[Jobs]`, `[dbo].[JobEvents]`, `[dbo].[QueueItems]`, `[dbo].[QueueItemEvents]`, `[dbo].[RobotLogs]` tables based on a cut-off timestamp.
Delete_Insights_Data_Read	N/A	Truncate `[read].Jobs`, `[read].JobEvents`,`[read].QueueItems`, `[read].QueueItemEvents`, `[read].RobotLogs` tables.

Note:

The Insights Dashboard widgets are available again after SQL Stored Procedure 1 has finished, and the data has been backfilled. Keep in mind that the backfill process may take hours to complete given that you have a large amount of data.

When you first run the Delete_Insights_Data_DBO SQL Script, it creates a QueueItems.IX_CreationTime nonclustered index with the purpose of making future executions faster.

SQL stored procedure 2: delete RobotLogs data based on ProcessNames and Tenant ID

Generate and save this SQL Stored Procedure to delete RobotsLogs based on ProcessNames and Tenant ID.

SQL Stored Procedures Parameter Description

SQL Stored Procedures	Parameter	Description
Delete_Process_Logs_DBO	@ProcessName `NVARCHAR(128)` (Required) The Orchestrator process name. @TenantId `INT` (Required) Tenant ID is required as two tenants might run processes with identical names. @BatchSize `INT = 100000` (Optional) Delete data in batches of 100000. See Considerations.	Delete data from `[dbo].[RobotLogs]` table by ProcessName and Tenant ID.
Delete_Process_Logs_Read	@ProcessName `NVARCHAR(128)` (Required) @TenantId `INT` (Required)	Truncate `[read].RobotLogs` table.

Delete_Process_Logs_DBO

@ProcessName NVARCHAR(128) (Required)

The Orchestrator process name.

@TenantId INT (Required)

Tenant ID is required as two tenants might run processes with identical names.

@BatchSize INT = 100000 (Optional)

Delete data in batches of 100000. See Considerations.

Delete data from [dbo].[RobotLogs] table by ProcessName and Tenant ID.

Delete_Process_Logs_Read

@ProcessName NVARCHAR(128) (Required)
@TenantId INT (Required)

Truncate [read].RobotLogs table.

Note: The Insights Dashboard widgets related to RobotLogs are available again after SQL Stored Procedure 2 has finished, and the data has been backfilled. Keep in mind that the backfill process may take hours to complete given that you have a large amount of data.

Stored Procedures (Database Engine)

On this page