- Introduction
- Setting up your account
- Balance
- Clusters
- Concept drift
- Coverage
- Datasets
- General fields
- Labels (predictions, confidence levels, label hierarchy, and label sentiment)
- Models
- Streams
- Model Rating
- Projects
- Precision
- Recall
- Annotated and unannotated messages
- Extraction Fields
- Sources
- Taxonomies
- Training
- True and false positive and negative predictions
- Validation
- Messages
- Access Control and Administration
- Manage sources and datasets
- Understanding the data structure and permissions
- Creating or deleting a data source in the GUI
- Uploading a CSV file into a source
- Preparing data for .CSV upload
- Creating a dataset
- Multilingual sources and datasets
- Enabling sentiment on a dataset
- Amending dataset settings
- Deleting a message
- Deleting a dataset
- Exporting a dataset
- Using Exchange integrations
- Model training and maintenance
- Understanding labels, general fields, and metadata
- Label hierarchy and best practices
- Comparing analytics and automation use cases
- Turning your objectives into labels
- Overview of the model training process
- Generative Annotation
- Dastaset status
- Model training and annotating best practice
- Training with label sentiment analysis enabled
- Training chat and calls data
- Understanding data requirements
- Train
- Introduction to Refine
- Precision and recall explained
- Precision and Recall
- How validation works
- Understanding and improving model performance
- Reasons for label low average precision
- Training using Check label and Missed label
- Training using Teach label (Refine)
- Training using Search (Refine)
- Understanding and increasing coverage
- Improving Balance and using Rebalance
- When to stop training your model
- Using general fields
- Generative extraction
- Using analytics and monitoring
- Automations and Communications Mining™
- Developer
- Exchange Integration with Azure service user
- Exchange Integration with Azure Application Authentication
- Exchange Integration with Azure Application Authentication and Graph
- Fetching data for Tableau with Python
- Elasticsearch integration
- Self-hosted Exchange integration
- UiPath® Automation Framework
- UiPath® Marketplace activities
- UiPath® official activities
- How machines learn to understand words: a guide to embeddings in NLP
- Prompt-based learning with Transformers
- Efficient Transformers II: knowledge distillation & fine-tuning
- Efficient Transformers I: attention mechanisms
- Deep hierarchical unsupervised intent modelling: getting value without training data
- Fixing annotating bias with Communications Mining™
- Active learning: better ML models in less time
- It's all in the numbers - assessing model performance with metrics
- Why model validation is important
- Comparing Communications Mining™ and Google AutoML for conversational data intelligence
- Licensing
- FAQs and more
Communications Mining user guide
Training with label sentiment analysis enabled
Label sentiment analysis is a feature that allows labels to be assigned with either positive or negative sentiment, depending on how that label concept is expressed within the message.
Every assigned label needs to be given a positive or negative sentiment, since there is no neutral sentiment, while multiple labels assigned on the same message can have different sentiments depending on how they are expressed.
The benefits of this feature is being able to report on the sentiment within a dataset for specific topics, and a number of charts are available in the Reports tab that relate to sentiment.
Label sentiment analysis is only appropriate for customer feedback related datasets. This is because they contain many more identifiable expressions of sentiment than other datasets, which tend to be much more neutral by nature.
Make sure that label sentiment analysis is appropriate for your use case, as once enabled during dataset creation, it cannot be disabled on that dataset.
The platform does have a pre-trained tone analysis model available, which predicts the overall tone, that is, the sentiment, of a message. This is typically appropriate and sufficient for all other use cases, for example, email inbox analysis and automation.
Label sentiment analysis is enabled at dataset creation and cannot be changed later. As you go through the dataset setup flow, you have the option to enable label sentiment analysis.
Tone analysis, which provides an overall sentiment score from -10 to 10 for a message, can be enabled at dataset creation, or later through the dataset settings.
Assigning labels with sentiment is very similar to assigning labels without sentiment. Check steps 1, 2, and 3 in the following image, which demonstrate annotating a message from a dataset of customer hotel reviews.
The main difference is in step 2, where after typing the label name, you must always select either positive or negative sentiment, denoted by the green or red face icons. This step has been repeated for both the Price and the Room > Size labels.
When applying labels with sentiment, make sure you create a taxonomy with neutral label names, where possible. For example, Price has been used in the previous example, instead of Expensive. This is because Price is neutral, whereas Expensive is inherently negative.
The selection of negative sentiment for a label with a neutral name would capture the instances where the message is expressing a negative perception of the label.
Most of the time it will be obvious which sentiment you should choose when you apply a label, based on the inherent positivity or negativity of the language, for example, the Price and Room > Size from the previous examples.
For certain labels, the concept may not lend itself to a neutral name and will be inherently negative or positive, and thus always always be applied with only one sentiment. For example, Error related labels will typically all be applied with negative sentiment. This is fine, but should be applied consistently.
Sometimes, however, it can be quite unclear. If the language in a message is very neutral in tone, you must think more carefully about which sentiment to apply.
You should consider message metadata and consistency of application:
Message metadata
The first is to look at the metadata of the message. For messages related to customer feedback, which is the most common type of data in a sentiment-enabled dataset, there will often be a certain kind of score or rating associated with a message, for example, NPS score. You can often use these scores to estimate whether a message that appears neutral in tone, is more positive or negative in sentiment, for example, a customer rarely leaves an NPS score of 10 if they are unhappy.
If you consistently apply label sentiment for messages that are neutral in tone, based on a score metadata field, the model can learn to pick up on this and predict the sentiment accordingly.
Consistency of application
The second is to be consistent in how you apply the sentiment for a label when it isquite neutral in tone, and there's no other differentiator, for example, a score related metadata field.
If it is more common for feedback to be positive for a certain label, assume it is positive unless the message is explicitly negative, and vice versa. If you are not consistent, however, the model will struggle to predict the sentiment.
Another important thing to consider when using sentiment analysis is that the model applies each label, that is, root and leaf, independently, so you can have two leaf labels from the same parent label that have different sentiments.
In these instances you must then judge what the overall sentiment for the parent label is. In this example below, the parent label Room is positive overall.
If both leaf labels have the same sentiment, then the model will infer that the parent label also has a negative sentiment and only the leaf labels will be shown as pinned labels, although that implies the parent label is also applied.
