- Introduction
- Setting up your account
- Balance
- Clusters
- Concept drift
- Coverage
- Datasets
- General fields
- Labels (predictions, confidence levels, label hierarchy, and label sentiment)
- Models
- Streams
- Model Rating
- Projects
- Precision
- Recall
- Annotated and unannotated messages
- Extraction Fields
- Sources
- Taxonomies
- Training
- True and false positive and negative predictions
- Validation
- Messages
- Access Control and Administration
- Manage sources and datasets
- Understanding the data structure and permissions
- Creating or deleting a data source in the GUI
- Uploading a CSV file into a source
- Preparing data for .CSV upload
- Creating a dataset
- Multilingual sources and datasets
- Enabling sentiment on a dataset
- Amending dataset settings
- Deleting a message
- Deleting a dataset
- Exporting a dataset
- Using Exchange integrations
- Model training and maintenance
- Understanding labels, general fields, and metadata
- Label hierarchy and best practices
- Comparing analytics and automation use cases
- Turning your objectives into labels
- Overview of the model training process
- Generative Annotation
- Dastaset status
- Model training and annotating best practice
- Training with label sentiment analysis enabled
- Training chat and calls data
- Understanding data requirements
- Train
- Introduction to Refine
- Precision and recall explained
- Precision and Recall
- How validation works
- Understanding and improving model performance
- Reasons for label low average precision
- Training using Check label and Missed label
- Training using Teach label (Refine)
- Training using Search (Refine)
- Understanding and increasing coverage
- Improving Balance and using Rebalance
- When to stop training your model
- Using general fields
- Generative extraction
- Using analytics and monitoring
- Automations and Communications Mining™
- Developer
- Exchange Integration with Azure service user
- Exchange Integration with Azure Application Authentication
- Exchange Integration with Azure Application Authentication and Graph
- Fetching data for Tableau with Python
- Elasticsearch integration
- Self-hosted Exchange integration
- UiPath® Automation Framework
- UiPath® Marketplace activities
- UiPath® official activities
- How machines learn to understand words: a guide to embeddings in NLP
- Prompt-based learning with Transformers
- Efficient Transformers II: knowledge distillation & fine-tuning
- Efficient Transformers I: attention mechanisms
- Deep hierarchical unsupervised intent modelling: getting value without training data
- Fixing annotating bias with Communications Mining™
- Active learning: better ML models in less time
- It's all in the numbers - assessing model performance with metrics
- Why model validation is important
- Comparing Communications Mining™ and Google AutoML for conversational data intelligence
- Licensing
- FAQs and more
Communications Mining user guide
Overview
This page covers how Communications Mining™ uses natural language processing (NLP) and machine learning (ML) to turn unstructured messages into structured data, enabling insight, automation, and scalability. It walks through the fundamentals of NLP, how Communications Mining works end-to-end, from connecting to data to training and validating models, and highlights the business value, integration capabilities, and benefits for both users and enterprises.
Natural language processing (NLP) is a field of machine learning (ML) focused on building tools that can automatically understand and process natural language data similarly to the way humans can.
A key focus of NLP tools is taking unstructured communications data and turning it into actionable, structured data by understanding the intents, themes, and concepts within and extracting key data points.
Business is built on communication, meaning that almost every process at some stage requires a conversation. Communications across email, tickets, and CRM systems are integral to the completion of a process.
From support to sales, to finance, and services, communicating with each other is how business gets done.
- Volume too high to keep pace with each day without wasting time and resources.
- Growing exponentially as employees and customers communicate more than ever.
- Multi-channel for employees and customers, such as email, support tickets, surveys, chat, and phone.
- Understood and actioned manually by staff through costly, inaccurate, and inefficient processes.
The number of emails, tickets, and messages is rising every year, and this is bringing services to a breaking point. It is increasing the cost to serve and damaging employee and customer experiences.
Thankfully, NLP, a branch of AI that focuses on helping machines read and understand human language, has seen massive advances in recent years.
NLP has come of age, and it is now beating humans in language understanding and reading comprehension.
This generates new solutions and opportunities for the enterprise.
With NLP, it is now possible to understand communications at scale, enabling businesses to:
- Understand what every customer wants.
- Track and measure all service demand in real time.
- Automate every transactional request.
The advantage of NLP is that it can free highly skilled employees from administrative work, reducing the cost to serve and improving experience.
Communications Mining is a field that focuses on understanding and extracting value from communications data.
The practice of converting the unstructured information that this data contains into structured, machine-readable data can then be used for analytics and automation.
- Process Mining captures end-to-end business processes. It extracts raw data from core business applications, such as ERP and CRM, and turns them into intuitive process graphs and dashboards to discover process optimization and automation opportunities.
- Task Mining captures individual tasks or process steps of a specific sub-process. It records and captures the tasks performed directly on the desktop of a user and visualizes their workflow to identify repetitive activities and variations, which are the prime candidates for automation.
- Communications Mining captures conversational data, such as emails, tickets, notes, transcripts, survey responses, and so on. It transforms digital communication data into a structured format to generate insight and enable downstream automation.
Communications Mining automates the interpretation of communications, helping businesses understand and automate messages at speed and at scale, on any channel.
This solution combines machine learning (ML), natural language processing (NLP), and employee-led supervised learning in a powerful, no-code solution that anyone can use.
It provides complete visibility into channels such as emails, tickets, and customer feedback, helping businesses better understand their customers and where improvements will have the biggest impact.
Moreover, it enables intelligent automation from communications, as it generates the structured data required by downstream automations to action requests without human intervention.
- Increase efficiency.
- Enhance customer and client experience.
- Improve governance and control.
All of the previously listed points help deliver value quickly and at scale, in hours, instead of months.
The following image contains an overview of the typical journey that your data goes on within the platform:
- Pre-built connectors for ingestion into historic communications store. Proprietary ML segmentation and cleaning engine to clean data.
- Proprietary Deep Learning Sentence models extract semantics for data-efficient learning.
- Proprietary Unsupervised Learning models identify common intents and constantly search for new ones.
- Train bespoke supervised models efficiently in our Proprietary Active Learning engine and interface.
- Real-time aggregate statistics for meaning-based management information and analytics.
- Real-time model validation and model lifecycle management.
The following overview outlines the steps involved in automating your data with Communications Mining:
- Connect - connects to your channels of unstructured communications data.
- Discover - identifies and puts together groups of communications that share similar themes and concepts.
- Train - builds the training data for the model.
- Predict - predicts structured labels and general fields for every communication.
- Validate - before relying on predictions to drive decisions, the platform ensures full transparency by automatically validating model performance with each retraining.
- Analyze - once satisfied with model performance, the platform delivers actionable insights by combining predictions with metadata, uncovering hidden processes and communication channels.
- Automate - deploy trained models to enable production automations.
First, Communications Mining connects to your channels of unstructured communications data, such as shared email inboxes, workflow tickets, collections of survey responses, and many more.
Getting this data into the platform can be done through:
- Live integration with pre-built connectors, for channels such as Microsoft Exchange or Salesforce.
- Building API integrations.
- Uploading historic data, through CSV or our API.
Discover
Once you have uploaded the data, the platform automatically kicks off the discovery process.
It uses unsupervised learning to cluster together groups of communications that share similar themes and concepts.
These clusters can link to repetitive processes, requests, issues, and sentiments. They can both help define the right structure for our model and speed up the first stage of model training.
The platform begins to form a clear understanding of the data by reviewing these clusters and applying labels and fields that capture relevant concepts and data points.
Train
Next, we use a variety of training modes to build out the training data for our model.
The following image depicts how we are teaching the platform to confidently identify these labels and fields across all our available data.
These training modes are designed to maximize the impact of training actions and minimize the time spent training. Meanwhile, the zero-code interface of the platform means that a Model Trainer can be any business user working in the communication channel. No data scientists or engineers are required.
With every training action, the platform continuously retrains, improving its understanding of each concept and data point, and updating its predictions in real time.
By annotating a small, representative sample of training data, the platform is able to apply its understanding of each label and general field at scale, automatically interpreting and making predictions across the entire dataset.
Predict
The end result is structured labels and general fields predictions, each with their own confidence scores, for every communication. An example is the following image depicting how Communications Mining interprets an email and extracts the relevant structured data from it:
These predictions are made available for analytics in the platform or via the API, which UiPath® robots and other tools can consume for automation or further analysis.
Validate
Before we rely on these predictions to influence decisions or enable action, we need to know how our model is performing.
The validation functionality of the platform provides full transparency when it comes to performance, validating your model automatically each time it retrains.
We can easily understand if our model is performing as it should across key performance factors that are aggregated into a single model rating for simplicity.
The platform also guides Model Trainers to make improvements as needed with recommended next-best actions.
Analyze
Once we are happy with the performance of our model, we can very quickly generate valuable and actionable insights from these business conversations.
The platform aggregates all of the predictions for labels and fields with key metadata to provide a wealth of queryable data, providing visibility into previously hidden processes and channels.
This allows users to perform one of the following actions, among others:
- Create custom dynamic dashboards to track key metrics and the quality of service delivered to customers and clients.
- Run analyses to identify opportunities to improve processes or customer experience.
- Set up alerts to monitor channel performance and risk events.
Alongside analytics, we can deploy trained models to enable production automations.
UiPath® robots and downstream systems can use the structured data that Communications Mining™ created to extend automation into service and conversation-based processes. This allows businesses to automate transactional requests and workflows.
UiPath can now automate end-to-end tasks, such as triaging emails, updating customer information, and case creation.
Leading enterprises trust Communications Mining to analyze and automate their communications for several key reasons:
- No code - We make NLP accessible to business users. The zero-code interface of the platform provides a guided user experience that any employee can use, regardless of technical ability.
- Fully customisable - You can create fully custom models that extract the precise intents, themes, and sentiments that your business needs.
- Accurate - You can train accurate models with minimal training data, with full transparency on model performance to avoid unexpected outcomes in production.
- Fast to train - You can keep costs and effort low, and confidence high, with models that are quick to train and fast to adapt.
- Hyper scalable - You need to start small to succeed, but you will need a solution that can rapidly scale with your needs for bigger use cases, which our platform does.
- Real-time monitoring and alerts - With configurable dashboards, email alerts, and reports tracking key metrics in real-time, you will have all the data you need to make proactive, informed decisions.
- Secure - Above all, you need a solution that you can trust your data with, and our permissioning and encryption of the platform ensure customer data is secure and protected.
- Easy to integrate - We have pre-built integrations for key communications channels, easy-to-work- with APIs, and connectors for workflow and RPA, helping it easily fit into your technology stack.
Our solution opens up the power of AI and NLP to all business users, not just data scientists and engineers. Some of the benefits are:
- Understand your customers better than ever - Discover at scale the issues driving customer demand and the actions that lead to better customer outcomes.
- Enjoy more interesting work - Benefit from powerful automations that take care of the boring, repetitive comms work.
- Focus on the work that matters - Devote more time to customers and the workflows that create real business value.
Communications Mining provides enterprises with full operational visibility, letting you understand and augment your business like never before.
- Grow the ROI of digital transformation - realize faster, more accurate MI capture and identify the most valuable change opportunities with confidence.
- Transform the customer experience - understand the drivers of workflow and customer contact at the source and notice what creates customer success.
- Enhance operational efficiency and performance - scale your operations rapidly with the automation of communications-based work.
- Natural language processing (NLP)
- Understanding the importance of NLP for enterprises
- NLP solutions
- Communications Mining
- Differences between Communications Mining™, Task Mining™, and Process Mining™
- Understanding Communications Mining
- Customer impact
- How Communications Mining works
- The detailed process of Communications Mining
- Why enterprises rely on Communications Mining
- What Communications Mining means to you
- What Communications Mining means to your business
