- Overview
- UI Automation
- About the UI Automation activity package
- Applications and technologies automated with UI Automation
- Project compatibility
- UI-ANA-016 - Pull Open Browser URL
- UI-ANA-017 - ContinueOnError True
- UI-ANA-018 - List OCR/Image Activities
- UI-DBP-006 - Container Usage
- UI-DBP-013 - Excel Automation Misuse
- UI-DBP-030 - Forbidden Variables Usage In Selectors
- UI-DBP-031 - Activity verification
- UI-PRR-001 - Simulate Click
- UI-PRR-002 - Simulate Type
- UI-PRR-003 - Open Application Misuse
- UI-PRR-004 - Hardcoded Delays
- UI-REL-001 - Large Idx in Selectors
- UI-SEC-004 - Selector Email Data
- UI-SEC-010 - App/Url Restrictions
- UI-USG-011 - Non Allowed Attributes
- UX-SEC-010 - App/Url Restrictions
- UX-DBP-029 - Insecure Password Use
- UI-PST-001 - Audit Log Level in Project Settings
- UiPath Browser Migration Tool
- Clipping region
- Computer Vision Recorder
- Activities index
- Activate
- Anchor Base
- Attach Browser
- Attach Window
- Block User Input
- Callout
- Check
- Click
- Click Image
- Click Image Trigger
- Click OCR Text
- Click Text
- Click Trigger
- Close Application
- Close Tab
- Close Window
- Context Aware Anchor
- Copy Selected Text
- Element Attribute Change Trigger
- Element Exists
- Element Scope
- Element State Change Trigger
- Export UI Tree
- Extract Structured Data
- Find Children
- Find Element
- Find Image
- Find Image Matches
- Find OCR Text Position
- Find Relative Element
- Find Text Position
- Get Active Window
- Get Ancestor
- Get Attribute
- Get Event Info
- Get From Clipboard
- Get Full Text
- Get OCR Text
- Get Password
- Get Position
- Get Source Element
- Get Text
- Get Visible Text
- Go Back
- Go Forward
- Go Home
- Google Cloud Vision OCR
- Hide Window
- Highlight
- Hotkey Trigger
- Hover
- Hover Image
- Hover OCR Text
- Hover Text
- Image Exists
- Indicate On Screen
- Inject .NET Code
- Inject Js Script
- Invoke ActiveX Method
- Key Press Trigger
- Load Image
- Maximize Window
- Microsoft Azure Computer Vision OCR
- Microsoft OCR
- Microsoft Project Oxford Online OCR
- Minimize Window
- Monitor Events
- Mouse Trigger
- Move Window
- Navigate To
- OCR Text Exists
- On Element Appear
- On Element Vanish
- On Image Appear
- On Image Vanish
- Open Application
- Open Browser
- Refresh Browser
- Replay User Event
- Restore Window
- Save Image
- Select Item
- Select Multiple Items
- Send Hotkey
- Set Clipping Region
- Set Focus
- Set Text
- Set To Clipboard
- Set Web Attribute
- Show Window
- Start Process
- System Trigger
- Take Screenshot
- Tesseract OCR
- Text Exists
- Tooltip
- Type Into
- Type Secure Text
- Use Foreground
- Wait Attribute
- Wait Element Vanish
- Wait Image Vanish
- Application Event Trigger
- Block User Input
- Check/Uncheck
- Check App State
- Check Element
- Click
- Click Event Trigger
- Drag and Drop
- Extract Table Data
- Find Elements
- For Each UI Element
- Get Browser Data
- Get Clipboard
- Get Text
- Get URL
- Go to URL
- Highlight
- Hover
- Inject Js Script
- Keyboard Shortcuts
- Keypress Event Trigger
- Mouse Scroll
- Navigate Browser
- Select Item
- Set Browser Data
- Set Clipboard
- Set Runtime Browser
- Set Focus
- Set Text
- Take Screenshot
- Type Into
- Unblock User Input
- Use Application/Browser
- Window Operation
- Semantic activities
- ScreenPlay
- Perform browser search and retrieve results using UI Automation APIs
- Web Browsing
- Find Images
- Click Images
- Trigger and Monitor Events
- Create and Override Files
- HTML Pages: Extract and Manipulate Information
- Window Manipulation
- Automated List Selection
- Find and Manipulate Window Elements
- Manage Text Automation
- Load and Process Images
- Manage Mouse Activated Actions
- Automate Application Runtime
- Automated Run of a Local Application
- Browser Navigation
- Web Automation
- Trigger Scope Example
- Enable UI Automation support in DevExpress
- Computer Vision Local Server
- Mobile Automation
- Release notes
- About the mobile device automation architecture
- Project compatibility
- Get Log Types
- Get Logs
- Get Page Source
- Get Device Orientation
- Get Session Identifier
- Install App
- Manage Current App
- Manage Other App
- Open DeepLink
- Open URL
- Mobile Device Connection
- Directional Swipe
- Draw Pattern
- Positional Swipe
- Press Hardware Button
- Set Device Orientation
- Take Screenshot
- Take Screenshot Part
- Element Exists
- Execute Command
- Get Attribute
- Get Selected Item
- Get Text
- Set Selected Item
- Set Text
- Swipe
- Tap
- Type Text
- Terminal
- Release notes
- About the Terminal activity package
- Project compatibility
- Best practices
- Find Text
- Get Color at Position
- Get Cursor Position
- Get Field
- Get Field at Position
- Get Screen Area
- Get Text
- Get Text at Position
- Move Cursor
- Move Cursor to Text
- Send Control Key
- Send Keys
- Send Keys Secure
- Set Field
- Set Field at Position
- Terminal Session
- Wait Field Text
- Wait Screen Ready
- Wait Screen Text
- Wait Text at Position
UI Automation Activities
Semantic activities are specifically designed for working with forms, offering a direct and powerful method for data extraction and data input.
These activities allow you to utilize a flexible approach, while the intricate details are handled by our AI and advanced internal components, such as:
- AI-powered DOM Extraction Engine: It extracts all usable UI elements in the form.
- Semantic Matching Engine: Adds resilience to your automations by semantically matching text labels, ensuring stability against UI changes.
- Semantic Targeting and Execution Engine: Eliminates traditional selector-based challenges, enabling seamless targeting and execution without the usual complexity and resilience issues related to using selectors.
This section includes the following activities:
Currently, only English is supported. We will list the availability of other languages as soon as we extend our support to include more languages.
Each Semantic activity consumes 0.5 AI units per request.
Each request made to Extract Form Data, Fill Form, and Set Value consumes 0.5 AI units by default.
The Close Popup activity consumes 0.5 AI units per request only when the AI-Enhanced mode toggle button is enabled. In this case, each close action via AI consumes 0.5 AI units. When the toggle button is disabled, the activity does not consume any AI units.
The AI unit consumption can be monitored using the Tenant level AI units consumption dashboard in Insights. Until 2025.4, the Semantic activities metric is rendered as Semantic Activities (promo), and covers the number of AI units that you utilize, but are not charged for. You can monitor this dashboard to forecast your AI units consumption for Semantic activities after 2025.4.
Starting with version 2024.10.6 and until 2025.4, Semantic activities are available free of charge to both Community and Enterprise customers, including those without AI Units purchased. This promotional period aims to encourage adoption and build customer confidence in the feature's quality. Beginning with version 2025.4, the consumption of AI Units applies as described, ensuring seamless continuity and transparency for all users.
UiPath.UIAutomationNext.Activities.NClosePopup
Description
Dismisses all pop-ups that are on top of the application and block a target, using the configured close buttons.
This activity can be used both inside and outside a Use Application/Browser activity.
Project compatibility
Windows | Cross-platform
Configuration
- Popup close buttons - The labels of the buttons used to close the pop-up. In Studio Desktop, Close and Cancel button labels are filled in by default.
- AI-Enhanced mode - Leverages AI to close the pop-up if the toggle button is set to On. Each close action via AI consumes one AI unit. The default option is Off.
Additional options
Timings
- Popup Appear Timeout - The amount of time (in seconds) to wait for a pop-up to appear. The default value is 30 seconds.
Input
- Popup detected exception - The exception thrown when detecting a pop-up.
Output
- Popup handled - Indicate if a pop-up was handled or not.
UiPath.Semantic.Activities.NExtractFormDataGeneric
Description
Uses AI to seamlessly extract form data.
It can be easily integrated with Data Entities.
This activity must be added inside a Use Application/Browser activity.
Project compatibility
Windows | Cross-platform
Configuration
- Discover fields from screen - The DOM analysis which finds actionable elements with their associated labels.
- Form data - Where to save the extracted form data. The output is a structured data custom type.
Additional options
Timings
-
Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
-
Delay after - Delay (in seconds) between the time this activity is completed and the time the next activity begins any operations. The default value is 0.3 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
-
Timeout - Specify the amount of time (in seconds) to wait for the activity to be executed before throwing an error. The default value is 30 seconds.
-
Continue on error - Specifies if the automation should continue even when the activity throws an error. This field only supports
Booleanvalues (True, False). The default value is False. As a result, if the field is blank and an error is thrown, the execution of the project stops. If the value is set to True, the execution of the project continues regardless of any error.
UiPath.Semantic.Activities.NFillForm
Description
Uses AI to seamlessly populate the designated form with information extracted from the provided data source.
This activity must be added inside a Use Application/Browser activity.
Project compatibility
Windows | Cross-platform
Configuration
- Data Source - Select a
data source of the following supported types:
DocumentData,Dictionary,DataEntity,DataRow,DataTable, JSON file, unstructured text files or even custom types, like those produced as output by the Extract Form Data activity.Note:ForDataTable, only the data in the first row is used to fill the form.To useIEnumerableas a Data Source, you need to add the activity inside a For Each activity. - Enable validation - Enables execution validation for the run-time value if the toggle button is set to On. An exception is thrown if the internal validation mechanism detects an invalid value after the execution. The default option is Off.
Additional options
Timings
-
Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
-
Delay after - Delay (in seconds) between the time this activity is completed and the time the next activity begins any operations. The default value is 0.3 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
-
Timeout - Specify the amount of time (in seconds) to wait for the activity to be executed before throwing an error. The default value is 30 seconds.
-
Continue on error - Specifies if the automation should continue even when the activity throws an error. This field only supports
Booleanvalues (True, False). The default value is False. As a result, if the field is blank and an error is thrown, the execution of the project stops. If the value is set to True, the execution of the project continues regardless of any error.
UiPath.Semantic.Activities.NSetValue
Description
Uses AI to seamlessly set a value on any input field: input box, drop-down, check box, radio buttons, date picker.
It can be used only on Chrome Edge, Firefox, and Safari.
Uses the same functionality as Fill Form, but on element level.
Instead of using other activities for specific input fields (for instance, Type Into for input boxes, Check/Uncheck for check boxes, etc.), you can use a single generic Update UI Element activity. It can also deal with date pickers and groups of radio buttons.
This activity must be added inside a Use Application/Browser activity.
This activity is compatible with Object Repository.
Project compatibility
Windows | Cross-platform
Configuration
-
Indicate target on screen - Indicate the UI element to use as target.
After you indicate the target, select the plus button to access the following options:
- Indicate target on screen - Indicate the target again.
- Edit target - Open the selection mode to configure the target.
-
Value - The value set on the field. This field accepts only
Stringvariables.If the indicated target is a checkbox, these are the accepted values:- True:
"true","yes","checked","x" - False:
"false","no","unchecked"
- True:
- Enable validation - Enables execution validation for the run-time value is the toggle button is set to On. An exception is thrown if the internal validation mechanism detects an invalid value after the execution. The default option is Off.
Options
- Healing Agent mode - Configures the behavior of the Healing Agent for this activity, based on Governance or Orchestrator settings at process
or job level.
- Disabled - Turns off the Healing Agent for this activity.
- Inherit job settings - Applies the settings defined in Governance or in Orchestrator at the process or job level.
- Recommendation only - Enables the Healing Agent to provide recommendations on how to fix UI automation issues on running jobs, but only if permitted by Governance or Orchestrator settings.
- Same as App/Browser - Inherits the configuration from the parent Use Application/Browser activity.
-
Continue on error - Specifies if the automation should continue even when the activity throws an error. This field only supports
Booleanvalues (True, False). The default value is False. As a result, if the field is blank and an error is thrown, the execution of the project stops. If the value is set to True, the execution of the project continues regardless of any error.
Target
- Expand target properties - Display target and anchor properties. This will hide the other activity properties and display only those related to the target.
- Collapse target properties - Hide target and anchor properties. This will hide target related properties and display the activity properties.
- Targeting methods - The selector types that you want to use for identifying the element. This property can be set to any combination from the
drop-down menu:
- None
- Strict selector - The strict selector generated for the target UI element.
- Fuzzy selector - The fuzzy selector parameters.
-
Computer Vision - To use Computer Vision as a targeting method, you need to go to Project Settings > Computer Vision and make sure to set the correct server endpoint. You can leave this field as it is and use UiPath Cloud, which is filled-in by default, or you can enter another value either by selecting any of the other public endpoints specific to a server region or your own on-premises server.
Another way of using Computer Vision as a targeting method is to set Use Local Server to True. This implies that the UiPath.ComputerVision.LocalServer package is installed in your project. When Local Server is enabled, the server value is over-written. However, the Local Server is not available in cross-platform projects.
- CV Control type - Indicates the type of control identified using Computer Vision.
- CV Text - Indicates the text identified using Computer Vision.
- CV Text occurrence - Indicates a specific occurrence to be used, when multiple matches are found.
- CV Text accuracy - Indicates the accuracy level for OCR text matching.
- Semantic selector - Leverages large language models (LLMs) to generate functional, human-like descriptions of UI elements. These descriptions are highly reliable making them adaptable to UI changes and significantly improving automation resilience when traditional selectors fail.
- Image (Windows only)
- Image accuracy - Indicates the accuracy level for image matching.
- Image occurrence - Indicates a specific occurrence to be used, when multiple matches are found.
- Native text (Windows only)
- Native text - The text to find to identify the UI element.
- Native text occurrence - Indicates a specific occurrence to be used, when multiple matches are found.
- Window selector (Application instance) - Selector for the application window. Only applicable when window attach mode is set to application instance.
- Wait for page load - Before performing the action, wait for the application to becaome ready to accept input. The options are:
- None - Does not wait for the target to be ready.
- Interactive - Waits until only a part of the app is loaded.
- Complete - Waits for the entire app to be loaded.
- Visibility check - When enabled, the activity also checks whether the UI element is visible or not. You can choose one of three options from
the drop-down menu:
- None - Does not check for visibility.
- Interactive (for Fuzzy Selector) - Default option. Checks if the element is potentially visible, ignoring page scroll and obstructions by other apps, or the fact that the application is minimized. This check is useful when trying to ensure that you are not targeting invisible elements that exist in the DOM but are hidden.
- Fully visible - Checks if the UI element is visible or not.
- Responsive websites - Enable responsive websites layout.
Timings
-
Delay before - Delay (in seconds) between the time the previous activity is completed and the time this activity begins performing any operations. The default value is 0.2 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
-
Delay after - Delay (in seconds) between the time this activity is completed and the time the next activity begins any operations. The default value is 0.3 seconds. Adding a delay between activities ensures that one activity has enough time to complete before the next activity begins.
-
Timeout - Specify the amount of time (in seconds) to wait for the activity to be executed before throwing an error. The default value is 30 seconds.
