- Overview
- Getting started
- Building models
- Consuming models
- Model Details
- Public endpoints for Automation Cloud and Test Cloud
- Public endpoints for Automation Cloud and Test Cloud Public Sector
- 1040 - document type
- 1040 Schedule C - document type
- 1040 Schedule D - document type
- 1040 Schedule E - document type
- 1040x - document type
- 3949a - document type
- 4506T - document type
- 709 - document type
- 941x - document type
- 9465 - document type
- ACORD125 - document type
- ACORD126 - document type
- ACORD131 - document type
- ACORD140 - document type
- ACORD25 - document type
- Bank Statements - document type
- Bills Of Lading - document type
- Certificate of Incorporation - document type
- Certificate of Origin - document type
- Checks - document type
- Children Product Certificate - document type
- CMS 1500 - document type
- EU Declaration of Conformity - document type
- Financial Statements - document type
- FM1003 - document type
- I9 - document type
- ID Cards - document type
- Invoices - document type
- Invoices2 - document type
- Invoices Australia - document type
- Invoices China - document type
- Invoices Hebrew - document type
- Invoices India - document type
- Invoices Japan - document type
- Invoices Shipping - document type
- Packing Lists - document type
- Payslips - document type
- Passports - document type
- Purchase Orders - document type
- Receipts - document type
- Receipts2 - document type
- Receipts Japan - document type
- Remittance Advices - document type
- UB04 - document type
- US Mortgage Closing Disclosures - document type
- Utility Bills - document type
- Vehicle Titles - document type
- W2 - document type
- W9 - document type
- Supported languages
- Insights dashboards
- Data and security
- Logging
- Licensing
- How to
- Troubleshooting
Document Understanding user guide
Migrating classic projects
Use the instructions from this page to migrate a classic project or a project based on AI Center. There are two main steps in migrating a project:
- Export the dataset from the classic project or the project based on AI Center.
- Import the dataset into the modern project.
Current limitations
- Currently, importing datasets larger than 5000pages is not supported. Only the initial 5000 pages will be successfully imported, with any additional pages failing to do so. For example, if your dataset consists of 4999 pages and you try to import a document of 4 pages, the process will not succeed.
- Batch names and corresponding batch results are not currently available. If your data is organized into batches, this information is not displayed yet, but it is saved.
- Exports from AI Center are not supported. Only exports from Document Manager are supported.
Exporting a dataset from a classic project
- Navigate to the classic project you want to migrate and open it.
- Go to the document type you want to export and select Open document type.
Figure 1. Open document type
- From the Filter documents drop-down list, select Training and validation set.
Figure 2. Training and validation set
- Select Export.
- Leave Current search results selected and fill in a name for your export job.
- Select Download.
Figure 3. Download export
Export a dataset from a project based on AI Center
-
Open AI Center and navigate to the Data Labeling page.
-
Select the Data Labeling Session you want to migrate.
-
Once Document Manager is open, from the Filter documents drop-down list, select Training and validation set.
Figure 4. Training and validation set
- Select Export.
- Leave Current search results selected and fill in a name for your export job.
- Select Download.
Figure 5. Download export
Importing a dataset
- Navigate to and open the project into which you want to import data.
- Select Add document type and create a new custom document type.
Figure 6. Add document type
- On the new custom document type, select Upload and choose the zip file of the classic project you exported. Wait for the upload to finish.
Note:
Exports from AI Center are not supported. Only exports from Document Manager are supported.
Figure 7. Upload processing
Once the upload is finished, the documents are available for training.
Model training
Once the dataset is imported, the model training starts. After the training is complete, the model score is displayed. To check detailed model scores, select the score, and then Detailed model scores.
This action takes you to the Measure page where you can access detailed model metrics.
When the same dataset is used to train an ML twice, you can observe slightly different model metrics. This can occur due to a few reasons:
- Initialization: Machine learning uses optimization methods that need initial guesses to trigger the optimization algorithms. Different initial guesses during each training could lead to various outcomes due to the unpredictable nature of these algorithms.
- Random state: Some algorithms use randomness in their operations. For instance, when training a neural network, procedures like stochastic gradient descent and mini-batch gradient descent introduce randomness. Therefore, even with identical initial model parameters and datasets, the performance of models may vary in different runs.
- Regularization: Certain algorithms include a penalty term that encourages the model to maintain smaller weights. Due to the randomness involved, the model could operate with a different weight set each time.
However, it's vital to note that these minor differences don't necessarily imply that one model is superior or inferior to another. Even with slightly varying metrics, the models' ability to comprehend data essentially remains the same, provided the differences are not significantly large. Moreover, repeating this process numerous times and taking an average should yield similar performance metrics.
Change the base model in Document type manager
If there's a significant difference between the model results from your classic project and the modern one, it could be caused by a different base model. To change the base model, proceed with the following steps:
-
Select the three-dot menu from your custom document type and choose Document type manager.
-
Navigate to the Settings tab.
-
Select the desired model from the Base model drop-down list.
-
After making your selection, select Save. To exit, select Back.
Export types
For classic projects, there are various methods for exporting data. Not all types of exported data are compatible for importing into modern projects. To compare the model results across both project types,filter documents by Training and validation set and select Choose search results to export the dataset. For more information on each option, check the following table.
| Type of export | Exported data | What happens to imported data |
|---|---|---|
| Current search results | Exports the current filtered dataset. Use it together with the Training and validation set filter. | Documents tagged as training are used to train the model. Documents tagged as validation are used to measure the model performance. Tip: To compare model results between two project types, always export and import the dataset as Train and validation . |
| All labeled | Exports all annotated documents from the dataset:
|
|
| Schema | Exports the list of fields and their respective settings. | A schema is imported is imported if there is none. If a schema is already defined, importing fails. |
| All | Exports all annotated and unannotated documents. |
|
Importing schemas
You can import schemas along with datasets into modern projects. Follow these steps to import a schema:
- Create a custom document type in the Build section.
- Import the zip file that holds the schema.
Note:
- Schema imports are limited to custom document types with no pre-existing schemas.
- If you import a schema into a document type that already contains a schema, the import will fail.
Migrate the automation workflow
Migrating from a classic DU project to a modern one in your RPA automation requires a single change: replace the ML Extractor Activity inside the Data Extraction Scope with a Document Understanding Project Extractor. No other activities need to change — digitization, validation, and training activities remain the same.
If your workflow uses document classification, also replace the existing classifier with a Document Understanding Project Classifier. See Migrate classification below.
Replace the ML Extractor Activity
- In your Studio project, open the Data Extraction Scope activity.
- Remove the existing ML Extractor Activity.
- Add a Document Understanding Project Extractor inside the Data Extraction Scope.
- Select Get or refresh extractor capabilities to open the configuration wizard.
- Under Design time credentials, enter your App Id, App Secret, and Tenant Url.
- Select Get Projects to load the list of available modern projects.
- For Project, select your desired modern project from the dropdown list.
- For Version, select a deployed version of the project. Alternatively, select a Tag linked to a specific version. Version and Tag are mutually exclusive.
- Select Get Capabilities.
- Make sure Update Activity Arguments is checked.
If you connect to a project in a different tenant, configure the Authentication properties of the activity — Runtime Credentials Asset and Runtime Tenant Url — to match the credentials used in the wizard.
For full configuration details, see Document Understanding Project Extractor.
Migrate classification
If your automation uses document classification, replace the existing classifier with a Document Understanding Project Classifier inside the Classify Document Scope. The configuration steps mirror those of the extractor: open the Configure Classifiers Wizard, enter your design time credentials, select your project and version or tag, then select Get Capabilities.
For full configuration details, see Document Understanding Project Classifier.
- Current limitations
- Exporting a dataset from a classic project
- Export a dataset from a project based on AI Center
- Importing a dataset
- Model training
- Change the base model in Document type manager
- Export types
- Importing schemas
- Migrate the automation workflow
- Replace the ML Extractor Activity
- Migrate classification