Live streaming and remote control via RealVNC

Important:

This applies to Robot versions 2023.12+.

The live streaming and remote control features are available for:

Windows Robots versions 2023.12+
Automation Cloud Robots - Serverless
Automation Cloud Robots - VM
Linux robots

Note: This functionality cannot be used with Windows background jobs.

Prerequisite for Windows Robots 2023.12 and newer

Enable Signal R. Go through the SignalR tenant setting section for details.

Network requirements

The minimum network requirements for a live streaming and remote control session to work smoothly can vary significantly, due to a number of factors:

The robot's screen resolution
The changes driven by your automation - the more dynamic the changes in images and text, the higher the volume of traffic
The latency between your machine and the Automation Cloud region
The latency between your robot and the Automation Cloud region

Considering the above, we recommend a 3MB/s network bandwidth for a 1920x1080 screen resolution.

Security considerations

The live streaming connection between the client and the target machine is tunneled through TLS via the VNC proxy, meaning that the server is secured. Additionally, every time a live streaming session is started, a unique JWT Token is generated and used to authenticate the request. The only instance of unsecured traffic is on the robot machine for the localhost connection.

On the robot side, a unique password is generated for each job, and set for the VNC server. The server only runs during the live streaming session. When the user disconnects from the session, the VNC server closes as well.

Network flow

When you make a request from the Orchestrator interface to start live streaming, Orchestrator sends a command to the robot to initiate a new remote control session.
The robot starts a VNC server that listens on a dynamic port. The VNC server is started as loopback-only, which disables remote connections. The Orchestrator command also includes a password for VNC authentication, that is shared between the VNC server and the browser client.
The robot then starts the following:
1. another program that connects to the local VNC server on a dynamic port
2. a UiPath live streaming proxy
- The program forwards all TCP traffic to the live streaming proxy by converting it into the WebSocket Secure protocol (wss).
The Orchestrator VNC client connects to the same live streaming proxy whose job is to pair the two websockets: one from the VNC client in the browser, and the other from the robot. It then creates a tunnel between the two websockets and forwards any traffic from one websocket to the other.