UI Automation activities

Description​

Project compatibility​

Windows and Cross-platform configuration​

• Work in scope - Choose the browser tab rendering the web application to automate, when the activity is placed inside nested Use Application/Browser cards.
• Indicate target on screen - Indicate the UI element to use as target. After you indicate the target, click the hamburger menu next to the informative screenshot to access the following options:
  • Indicate target - Indicate the target again. If the Descriptor is defined in the Object Repository, you must unlink it from the Object Repository if you want to specify a Descriptor only for the current activity.
  • Edit target - Open the selection mode to configure the target.
  • Remove Informative Screenshot - Remove the screenshot of the target application or web page.
  • Change Informative Screenshot - Change the screenshot of the target application or web page.
  • Add to Object Repository - Add the current UI element to the Object Repository, from where it can be reused throughout your automation process. This option is only displayed when the object has not yet been added to the Object Repository.
  • Show in Object Repository - Open the Object Repository panel with the Descriptor selected. This option is only displayed when the object has been added to the Object Repository.
  • Unlink from Object Repository - Unlink the Descriptor from the Object Repository. This option is only displayed when the object has been added to the Object Repository.
• Wait for (appear/disappear) - Select which event to use for determining the state of the application or browser:
  • Element to appear - Wait for the element to appear in the user interface. If you select this option:
    • Add the activities to execute if the element appears in the Target appears block.
    • Add the activities to execute if the element does not appear in the Target does not appear block.
  • Element to disappear - Wait for the element to disappear from the user interface. If you select this option:
    • Add the activities to execute if the element disappears in the Target disappears block.
    • Add the activities to execute if the element does not disappear in the Target does not disappear block.
  • Element window not active - Wait for the element top-level window become inactive. If you select this option:
    • Add the activities to execute if the element top-level window become inactive in the Window not in freground block.
    • Add the activities to execute if the element top-level window remains active in the Window still active block.
• Timeout (seconds) - The amount of time to wait for the element to appear or disappear before executing one of the two activity blocks. The default value is 5 seconds.

• State - A true or false value indicating the detected state of the target.
• Input element - The UI element on which the activity is executed, stored in an UIElement object. This field supports only UIElement objects. This object can be obtained from the Output Element property field of other UI Automation activities.
• Output element - Outputs a target UI Element and stores it in a UIElement object, which can be further used to target the same element with other activities.

• Wait active - Indicates whether the activity should wait for the UI element's top-level window to be in te foreground.
• Healing Agent mode - Configures the behavior of the Healing Agent for this activity, based on Governance or Orchestrator settings at process or job level.
  • Disabled - Turns off the Healing Agent for this activity.
  • Inherit job settings - Applies the settings defined in Governance or in Orchestrator at the process or job level.
  • Recommendation only - Enables the Healing Agent to provide recommendations on how to fix UI automation issues on running jobs, but only if permitted by Governance or Orchestrator settings.
  • Same as App/Browser - Inherits the configuration from the parent Use Application/Browser activity.

• Expand target properties - Displays the target container and its anchor(s) properties.

• Collapse target properties - Hides the target container and its anchor(s) properties.

• Targeting methods - The selector types that you want to use for identifying the element. This property can be set to any combination, and at least one targeting method must be selected:

  • Strict selector - The strict selector generated for the target UI element.
  • Fuzzy selector - The fuzzy selector parameters.
  • Computer Vision - To use Computer Vision as a targeting method, you need to go to Project Settings > Computer Vision and make sure to set the correct server endpoint. You can leave this field as it is and use UiPath Cloud, which is filled-in by default, or you can enter another value either by selecting any of the other public endpoints specific to a server region or your own on-premises server. Another way of using Computer Vision as a targeting method is to set Use Local Server to True. This implies that the UiPath.ComputerVision.LocalServer package is installed in your project. When Local Server is enabled, the server value is over-written. However, the Local Server is not available in cross-platform projects.
    • CV Control type - Indicates the type of control identified using Computer Vision.
    • CV Text - Indicates the text identified using Computer Vision.
    • CV Text occurrence - Indicates a specific occurrence to be used, when multiple matches are found.
    • CV Text accuracy - Indicates the accuracy level for OCR text matching.
  • Semantic selector - Leverages large language models (LLMs) to generate functional, human-like descriptions of UI elements. These descriptions are highly reliable making them adaptable to UI changes and significantly improving automation resilience when traditional selectors fail.
  • Image (only available for Windows projects)
    • Image accuracy - Indicates the accuracy level for image matching.
    • Image occurrence - Indicates a specific occurrence to be used, when multiple matches are found.
  • Native text (only available for Windows projects)
    • Native text - The text to find to identify the UI element.
    • Native text occurrence - Indicates a specific occurrence to be used, when multiple matches are found.

• Window selector (Application instance) - Selector for the application window. Only applicable when window attach mode is set to application instance.

• Wait for page load - Before performing the action, wait for the application to becaome ready to accept input. The options are:

  • None - Does not wait for the target to be ready.
  • Interactive - Waits until only a part of the app is loaded.
  • Complete - Waits for the entire app to be loaded.

• Visibility check - When enabled, the activity also checks whether the UI element is visible or not. You can choose one of three options from the drop-down menu:

  • None - Does not check for visibility.
  • Interactive (for Fuzzy Selector) - Default option. Checks if the element is potentially visible, ignoring page scroll and obstructions by other apps, or the fact that the application is minimized. This check is useful when trying to ensure that you are not targeting invisible elements that exist in the DOM but are hidden.
  • Fully visible - Checks if the UI element is visible or not.

• Responsive websites - Enable responsive websites layout.

• Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.

Windows - Legacy configuration​

• Select Indicate in App/Browser to indicate the UI element to use as target.

• Indicate target on screen - Indicate the target again.
• Edit target - Open the selection mode to configure the target.
• Indicate in any app - When used inside a Use Application/Browser activity, you can select this option to check application states for windows or elements not associated with the current application instance without having to add a new application card.
• Remove informative screenshot - Remove the screenshot of the target application or web page.
• Add to Object Repository - Add the current UI element to a library project, from where it can be reused throughout your automation process.

• Wait for - Select which event to use for determining the state of the application or browser:
  • Element to appear - Wait for the element to appear in the user interface. If you select this option:
    • Add the activities to execute if the element appears in the Target appears block.
    • Add the activities to execute if the element does not appear in the Target does not appear block.
  • Element to disappear - Wait for the element to disappear from the user interface. If you select this option:
    • Add the activities to execute if the element disappears in the Target disappears block.
    • Add the activities to execute if the element does not disappear in the Target does not disappear block.
• Seconds - The amount of time to wait for the element to appear or disappear before executing one of the two activity blocks. You can choose a predetermined value from the drop-down menu. The default value is 5 seconds.
• Click Toggle branches in the body of the activity to configure the visibility of the Target appears and Target does not appear containers where you can add activities to execute, depending on the state. You can work with both containers, either one of them, or none. By default, both blocks are visible.

Properties panel​

• Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.

• Display name - The name displayed for the activity in the Designer panel. A display name is automatically generated when you indicate a target.

• • CV Control type - Indicates the type of control identified using Computer Vision.
  • CV Text - Indicates the text identified using Computer Vision.
  • Fuzzy selector - The parameters for the fuzzy selector.
  • Native text - The text that is used to identify the UI element.
  • Responsive websites - Enable responsive websites layout.
  • Strict selector - The strict selector generated for the UI element.
  • Targeting methods - The selector types that you want to use for identifying the element. This property can be set to any combination from the drop-down menu:
    • None
    • Strict selector
    • Fuzzy selector
    • Image
    • Native text
    • Computer Vision - To use Computer Vision as a targeting method, you need to go to Project Settings > Computer Vision and make sure to set the correct server endpoint. You can leave this field as it is and use UiPath Cloud, which is filled-in by default, or you can enter another value either by selecting any of the other public endpoints specific to a server region or your own on-premises server. Another way of using Computer Vision as a targeting method is to set Use Local Server to True. This implies that the UiPath.ComputerVision.LocalServer package is installed in your project. When Local Server is enabled, the server value is over-written. However, the Local Server is not available in cross-platform projects.
    • Semantic Selector - Leverages large language models (LLMs) to generate functional, human-like descriptions of UI elements. These descriptions are highly reliable making them adaptable to UI changes and significantly improving automation resilience when traditional selectors fail.
  • Visibility check - Checks whether the UI element is visible or not. You can choose one of three options from the drop-down menu:
    • None - Does not check for visibility.
    • Interactive (for Fuzzy Selector) - Default option. Checks if the element is potentially visible, ignoring page scroll and obstructions by other apps, or the fact that the application is minimized. This check is useful when trying to ensure that you are not targeting invisible elements that exist in the DOM but are hidden.
    • Fully visible
      • Checks if the UI element is visible or not.
  • Wait for page load - Before performing the action, wait for the application to become ready to accept input. You can choose one of three options from the drop-down menu:
    • None - Does not wait for the target to be ready.
    • Interactive - Waits until only a part of the app is loaded.
    • Complete - Waits for the entire app to be loaded.
  • Window selector (Application instance) - The selector that is used for the application window. Only applicable when the window attach mode is set to Application instance.

• Input element - The UI element on which the activity is executed, stored in an UIElement object. This field supports only UIElement objects. This object can be obtained from the Output Element property field of other UI Automation activities.

• Output element - Outputs a target UI Element and stores it in a UIElement object, which can be further used to target the same element with other activities.

• Private - If selected, the values of variables and arguments are no longer logged at Verbose level. This field supports only Boolean (True, False) values. The default value is False.

• Timeout (seconds) - See seconds in the Designer panel.

• Wait for (appear/disappear) - See Wait for in the Designer panel.

• Result - Indicates if the element exists or not. The field supports only Boolean variables. Optionally, you can save this value (True/False) for later use, or save it in a cell in a parent Excel file or the Project Notebook.