用户界面自动化活动

上次更新日期 2025年11月18日

About Semantic Selectors

Semantic Selectors are a new targeting method within Unified Target, designed to identify UI elements based on meaning, not position or structure. They use AI-driven understanding of an element’s role, purpose, and context to make automations resilient to UI changes based on the DOM information.

Note: Image-based semantic selectors are currently not supported.

In Unified Target, Semantic Selectors serve as the main fallback targeting method for selectors similarly to Computer Vision, but significantly more powerful and adaptable. They can also be configured as a primary targeting method, enabling a fully semantic approach to UI element detection.

Traditional selectors rely on rigid attributes or hierarchies. Semantic Selectors capture what the element represents, allowing automations to adapt automatically when UIs evolve.

Best practices and benefits

This section outlines best practices and benefits for using Semantic Selectors.

最佳实践

Use Semantic Selectors in the following scenarios:

UIs change frequently or are built dynamically (e.g., React, Angular);
You automate across environments (staging vs. production);
You want higher reliability without constant maintenance;
You need a fallback mechanism in Unified Target that performs well under uncertainty.

Note: You can also set Semantic Selectors as the primary targeting method when semantic understanding is the most reliable approach for your application.

Avoid using Semantic Selectors as primary targeting methods in the following scenarios:

You require fixed structural paths for test verification;
The UI is static and easily addressed using traditional selectors.

收益

When automations fail, it is often due to fragile selectors that break after the following causes:

Minor UI layout or theme updates;
Dynamic element IDs;
Label or class name changes.

Semantic Selectors overcome these challenges by interpreting intent instead of syntax.

Using Semantic Selectors include the following benefits:

Resilience – keep working even when the UI changes.
Adaptability – identifying equivalent elements across environments.
Consistency – aligning automation with user intent and functional purpose.
Integration – working seamlessly within Unified Target, alongside traditional and Computer Vision targeting.

Example of using the Semantic Selector

In the following example, the Semantic Selector targets the The button that submits the form button by meaning, rather than by its position or HTML attributes. Even if the button moves or its label changes, the automation still succeeds.

Comparing selector capabilities

The following table captures the differences between other types of selectos and Semantic Selectors:

功能	Traditional selectors	Computer Vision	Semantic Selectors
UI change tolerance	低	中	高
Context awareness	受限	仅用于视觉效果	Strong (semantic)
Maintenance effort	高	中	低
Integration in Unified Target	是	是	Yes (fallback or primary)
Human-like interpretation	否	Partial	Full