- Release notes
- Before you begin
- Managing access
- Getting started
- Integrations
- Working with process apps
- Working with dashboards and charts
- Working with process graphs
- Working with Discover process models and Import BPMN models
- Showing or hiding the menu
- Context information
- Export
- Filters
- Sending automation ideas to UiPath® Automation Hub
- Tags
- Due dates
- Compare
- Conformance checking
- Process simulation
- Root cause analysis
- Simulating automation potential
- Starting a Task Mining project from Process Mining
- Triggering an automation from a process app
- Viewing Process data
- Creating apps
- Loading data
- Transforming data
- Autopilot™ for SQL (preview)
- Structure of transformations
- Tips for writing SQL
- Exporting and importing transformations
- Viewing the data run logs
- Merging event logs
- Configuring Tags
- Configuring Due dates
- Configuring fields for Automation potential
- Activity Configuration: Defining activity order
- Making the transformations available in dashboards
- Data models
- Adding and editing processes
- Customizing process apps
- Publishing process apps
- App templates
- Notifications
- Additional resources
Process Mining
By default, all extractions you can download for your app template are full extractions. To configure incremental extractions, you need to reconfigure the tables for which you want to enable incremental extraction in Xtract Universal Designer. Incremental extractions are mainly useful for transactional data tables, as these typically contain large amounts of data.
The section Input tables and fields in the documentation page of the app template contains a list of the transactional tables for the app template. Refer to App Templates.
When using full extractions, joins are sometimes applied to filter on properties that are not available in the table itself (for example, creation dates, which are usually stored at the header level. By joining to the header table, item-level tables can also be filtered).
With incremental extractions, only a single table can be extracted. This may result in more data being loaded during the initial run, which can increase execution time.
If you encounter an issue with an incremental extraction in Theobald Xtract Universal, you can reset the incremental settings by opening the extraction and selecting Delete CDC Resources.
This removes the record of the last successful extraction. The next time the extraction runs, it will perform a full load of the table. Subsequent extractions will then run incrementally again.
You can also use this option when modifying the extraction configuration (for example, when adding a new field).
extraction_tables_incremental setting in the extraction script. Check out Configuring the extraction script for more information.
Follow these steps to configure an incremental extraction for a (transactional) table.
- Remove the existing extraction.
- Create a new extraction.
- Select your source connection.
- Define the same name as before for the extraction.
- Select Table CDC from the RFC extractors list.
- Select OK.
- In the Table CDC dialog, first select the Source table.
Tip: Search the table and select it.
- Make sure Extract table on first run is selected.
- Make sure only the required fields are checked in the fields section.
Note: The required fields are listed in the Input tables and fields section of the app template documentation page. App Templates.
- Go to the WHERE clause tab and create the
WHEREclause.Note: The condition for each table is listed in the app template documentation. App Templates.Note: You can use a date field to define a start date for the extractions.- Make sure the start date is the same in all extractions.
- Select the specified start date carefully. You cannot make any changes to the
WHEREclause after the table has been initialized. - Dates are specified in
yyyymmddformat.
- Select OK to save the settings and close the Table CDC dialog.
- Select your newly added extraction and open the Destination settings.
- Select the correct destination configuration to be used.
- Make sure the File Name is set to Custom. Fill out the name of the to be extracted file, make sure to enter the table name (e.g. "CDPOS").
- Make sure Convert dates is turned off.
Follow these steps:
- Go to the Data transformations and select Manage input data. Check outManaging input data for more information.
- For each table that you defined to be incremental, update the settings:
- Select Incremental as the Load type.
- Add the primary keys for the table.
Note: The primary keys for each table are listed in the app template documentation. App Templates.
- For the timestamp field, select TS_TIMESTAMP.
