UI Automation activities

Description​

Project compatibility​

Windows and Cross-platform configuration​

• Work in scope - Choose the browser tab rendering the web application to automate, when the Take Screenshot activity is placed inside nested Use Application/Browser cards.

• Screenshot container - Select between Entire page and Element to either take a screenshot of the entire page or a screenshot of a single element. The default option is Entire page.

• Indicate target on screen - Indicate the UI element to use as target. This button is only available when the Element option is selected as the Screenshot container. After you indicate the target, click the hamburger menu next to the informative screenshot to access the following options:

  • Indicate target - Indicate the target again. If the Descriptor is defined in the Object Repository, you must discunlinkonnect it from the Object Repository if you want to specify a Descriptor only for the current activity.
  • Edit target - Open the selection mode to configure the target.
  • Remove Informative Screenshot - Remove the screenshot of the target application or web page.
  • Change Informative Screenshot - Change the screenshot of the target application or web page.
  • Add to Object Repository - Add the current UI element to a library project, from where it can be reused throughout your automation process. This option is only displayed when the object has not yet been added to the Object Repository.
  • Show in Object Repository - Open the Object Repository panel with the Descriptor selected. This option is only displayed when the object has been added to the Object Repository.
  • Unlink from Object Repository - Unlink the Descriptor from the Object Repository. This option is only displayed when the object has been added to the Object Repository.

• Output to (only available for Windows projects) - Use the radio buttons to choose the output type:

  • File - Saves the screenshot to file at a specified location. The default option.
  • Image - Saves the screenshot to the indicated image.
  • Clipboard - Saves the screenshot as an image to clipboard for later use.

• File name - The name of the file where the screenshot of the specified UI element is saved.

• Auto increment - Defines what to append to the filename in case of filename conflicts. Choose an option from the drop-down menu:

  • None (overwrite) - The filename will be exactly as specified, and in case another file with the same name already exists, it will be overwritten. This is the default option.
  • Index - If one or multiple files that match the pattern Filename (XX) already exists, a new file is created with name Filename (N+1), where N is the max index from the existing files.
  • YYYY.MM.DD at HH.MM.SS - If one file with specified name already exists, a new file is created with name Filename YYYY.MM.DD at HH.MM.SS. If a file with filename Filename YYYY.MM.DD at HH.MM.SS already exists, and index is added, similar to the Index case, resulting in a file called Filename YYYY.MM.DD at HH.MM.SS (1).

• Expand target properties/Collapse target properties button - Displays/hides the target container and its anchor(s) properties.
• Targeting methods - The selector types that you want to use for identifying the element. This property can be set to any combination from the drop-down menu, and at least one targeting method must be selected:
  • Strict selector
  • Fuzzy selector
  • Image (only available for Windows projects)
  • Native text (noonly available for Windows projects)
  • Computer Vision
  • Semantic selector
• Window selector (Application instance) - The selector that is used for the application window. Only applicable when the window attach mode is set to Application instance.
• Strict selector - The strict selector generated for the UI element. It is only displayed if Strict selecto targeting method has been selected.
• Fuzzy selector - The parameters for the fuzzy selector. It is only displayed if Fuzzy selecto targeting method has been selected.
• Image accuracy - Indicates the accuracy level for image matching. Default value is 0.8. It is only displayed if Image targeting method has been selected.
• Image occurence - Indicates a specific occurence to be used, when multiple matches are found. It is only displayed if Image targeting method has been selected.
• Native text - The text that is used to identify the UI element. It is only displayed if Native targeting method has been selected.
• Native text occurence - Indicates a specific occurence to be used, when multiple matches are found. It is only displayed if Native targeting method has been selected.
• Native text case-sensitive - Indicates weather text matching is case-sensitive. It is only displayed if Native targeting method has been selected.
• CV Control type - Indicates the type of control identified using Computer Vision. It is only displayed if Computer Vision targeting method has been selected.
• CV Text - Indicates the text identified using Computer Vision. It is only displayed if Computer Vision targeting method has been selected.
• CV Text accuracy - Indicates the accuracy level for OCR text matching. Default value is 0.7. It is only displayed if Computer Vision targeting method has been selected.
• CV Text occurence - Indicates a specific occurence to be used, when multiple matches are found. It is only displayed if Computer Vision targeting method has been selected.
• Semantic selector - A semantic description that defines the target. It is only displayed if Semantic targeting method has been selected.
• Wait for page load - Before performing the action, wait for the application to become ready to accept input. You can choose one of three options from the drop-down menu:
  • None - Does not wait for the target to be ready.
  • Interactive - Waits until only a part of the app is loaded.
  • Complete - Waits for the entire app to be loaded.
• Visibility check - Checks whether the UI element is visible or not. You can choose one of three options from the drop-down menu:
  • None - Does not check for visibility.
  • Interactive (for Fuzzy selector) - Default option. Checks if the element is potentially visible, ignoring page scroll and obstructions by other apps, or the fact that the application is minimized. This check is useful when trying to ensure that you are not targeting invisible elements that exist in the DOM but are hidden.
  • Fully visible - Checks if the UI element is visible or not.
• Responsive websites - Enable responsive websites layout.
• Target - Clipping region - Specifies a clipping region, which can be further configured. It can be used to establish smaller/larger region of the UI element found at runtime using the Target's Selector or provided directly via the Target's Input Element property.
  • Anchoring point - Describes the starting point of the cursor to which it offsets from Offset X and Offset Y properties are added. The following options are available: TopLeft, TopRight, BottomLeft, BottomRight, and Center (selected by default).
  • Offset X - Horizontal displacement of the region according to the option selected in the Anchoring point field. This field supports only Int32 variables.
  • Offset Y - Vertical displacement of the region according to the option selected in the Anchoring point field. This field supports only Int32 variables.
  • Offset width - Width of the offset region.
  • Offset height - Heightt of the offset region.

• Healing Agent mode - Configures the behavior of the Healing Agent for this activity, based on Governance or Orchestrator settings at process or job level.

  • Same as App/Browser - Inherits the configuration from the parent Use Application/Browser activity.
  • Inherit job settings - Applies the settings defined in Governance or in Orchestrator at the process or job level.
  • Disabled - Turns off the Healing Agent for this activity.
  • Recommendation only - Enables the Healing Agent to provide recommendations on how to fix UI automation issues on running jobs, but only if permitted by Governance or Orchestrator settings.

• Continue on error - Specifies if the automation should continue even when the activity throws an error. This field only supports Boolean values (True, False). The default value is False. As a result, if the field is blank and an error is thrown, the execution of the project stops. If the value is set to True, the execution of the project continues regardless of any error.

• Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
• Delay after - Delay (in seconds) between the time this activity is completed and the time the next activity begins any operations. The default value is 0.3 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
• Timeout - Specify the amount of time (in seconds) to wait for the activity to be executed before throwing an error. The default value is 30 seconds.

• Screenshot file - The screenshot saved as a .png file.
• Output element - Outputs a target UI Element and stores it in a UIElement object, which can be further used to target the same element with other activities.
• Input element - The UI element on which the activity is executed, stored in an UIElement object. This field supports only UIElement objects. This object can be obtained from the Output Element property field of other UI Automation activities.

Windows - Legacy configuration​

• Select Indicate in App/Browser to indicate the UI element to use as target.

• Indicate target on screen - Indicate the target again.
• Edit target - Open the selection mode to configure the target.
• Remove informative screenshot - Remove the screenshot of the target application or web page.

• Output to - Use the radio buttons to choose the output type:
  • File - Saves the screenshot to file at a specified location. The default option.
  • Image - Saves the screenshot to the indicated image.
  • Clipboard - Saves the screenshot as an image to clipboard for later use.
• File Name - The name of the file where the screenshot of the specified UI element is saved. Click Browse for File, then browse to the folder where to save the screenshot, and specify the file name and extension to use.
• Auto increment - Select what to append to the filename when saving the screenshot in case of filename conflicts. For example, if a file with the same name as the one you save already exists, choosing Index keeps the file name, creating a new file for each screenshot and adding an index number to each, consecutively.
  • None (overwrite) - If a file with the same name already exists, replace it.
  • Index - Add a number to the filename, for example screenshot (1).png.
  • YYYY.MM.DD at HH.MM.SS - Add the date and time when the screenshot is taken to the filename, for example screenshot 2020.06.22 at 10:22:32.

Properties panel​

• Continue on error - Specifies if the automation should continue even when the activity throws an error. This field only supports Boolean values (True, False). The default value is False. As a result, if the field is blank and an error is thrown, the execution of the project stops. If the value is set to True, the execution of the project continues regardless of any error.

• Delay after - Delay (in seconds) between the time this activity is completed and the time the next activity begins any operations. The default value is 0.3 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.

• Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.

• Display name - The name displayed for the activity in the Designer panel. A display name is automatically generated when you indicate a target.

• Timeout - Specify the amount of time (in seconds) to wait for the activity to be executed before throwing an error. The default value is 30 seconds.

• Auto increment - See Auto increment in the Designer panel.

• File name - See File name in the Designer panel.

• Target - The target element. Target element must be set. Before indicating on screen, this field is set to (null). Once the target is indicated, all properties regarding the element that was indicated are displayed.

  • CV Control type - Indicates the type of control identified using Computer Vision.
  • CV Text - Indicates the text identified using Computer Vision.
  • Fuzzy selector - The parameters for the fuzzy selector.
  • Native text - The text that is used to identify the UI element.
  • Responsive websites - Enable responsive websites layout.
  • Strict selector - The strict selector generated for the UI element.
  • Targeting methods - The selector types that you want to use for identifying the element. This property can be set to any combination from the drop-down menu:
    • None
    • Strict selector
    • Fuzzy selector
    • Image
    • Native text
    • Computer Vision - To use Computer Vision as a targeting method, you need to go to Project Settings > Computer Vision and make sure to set the correct server endpoint. You can leave this field as it is and use UiPath Cloud, which is filled-in by default, or you can enter another value either by selecting any of the other public endpoints specific to a server region or your own on-premises server. Another way of using Computer Vision as a targeting method is to set Use Local Server to True. This implies that the UiPath.ComputerVision.LocalServer package is installed in your project. When Local Server is enabled, the server value is over-written. However, the Local Server is not available in cross-platform projects.
    • Semantic Selector - Leverages large language models (LLMs) to generate functional, human-like descriptions of UI elements. These descriptions are highly reliable making them adaptable to UI changes and significantly improving automation resilience when traditional selectors fail.
  • Visibility check - Checks whether the UI element is visible or not. You can choose one of three options from the drop-down menu:
    • None - Does not check for visibility.
    • Interactive (for Fuzzy Selector) - Default option. Checks if the element is potentially visible, ignoring page scroll and obstructions by other apps, or the fact that the application is minimized. This check is useful when trying to ensure that you are not targeting invisible elements that exist in the DOM but are hidden.
    • Fully visible
      • Checks if the UI element is visible or not.
  • Wait for page load - Before performing the action, wait for the application to become ready to accept input. You can choose one of three options from the drop-down menu:
    • None - Does not wait for the target to be ready.
    • Interactive - Waits until only a part of the app is loaded.
    • Complete - Waits for the entire app to be loaded.
  • Window selector (Application instance) - The selector that is used for the application window. Only applicable when the window attach mode is set to Application instance.

• Input element - The UI element on which the activity is executed, stored in an UIElement object. This field supports only UIElement objects. This object can be obtained from the Output Element property field of other UI Automation activities.

• Output element - Outputs a target UI Element and stores it in a UIElement object, which can be further used to target the same element with other activities.

• Private - If selected, the values of variables and arguments are no longer logged at Verbose level. This field supports only Boolean (True, False) values. The default value is False.

• Saved file path - The full path to the file where to save the screenshot, including the appended suffix, if applicable. This also dictates where the Auto increment property saves the indexed screenshot files.

• Saved image - The screenshot saved as Image; used when Output is set to image.