Training with label sentiment analysis enabled

Label sentiment analysis is a feature that allows labels to be assigned with either positive or negative sentiment, depending on how that label concept is expressed within the message.

Every assigned label needs to be given a positive or negative sentiment (there is no neutral sentiment), whilst multiple labels assigned on the same message can have different sentiments depending on how they are expressed.

The benefits of this feature is being able to report on the sentiment within a dataset for specific topics, and a number of charts are available in Reports that relate to sentiment.

Label sentiment analysis is typically only appropriate for customer feedback related datasets. This is because they contain many more identifiable expressions of sentiment than other datasets, which tend to be much more neutral by nature.

It's important to be sure that label sentiment analysis is appropriate for your use case, as once it's enabled during dataset creation, it cannot be disabled on that dataset.

The platform does have a pre-trained tone analysis model available, which predicts the overall tone, i.e. sentiment, of a message. This is typically appropriate and sufficient for all other use cases, e.g. email inbox analysis and automation.

Label sentiment analysis is enabled at dataset creation and cannot be changed later. As you go through the dataset setup flow, you have the option to enable label sentiment analysis.

Tone analysis, which provides an overall sentiment score from -10 to 10 for a message, can be enabled at dataset creation, or later via the dataset settings. This can be turned off and on, so it's not important if you don't enable it during setup.

Assigning labels with sentiment is very similar to assigning labels without sentiment – see Steps 1, 2 and 3 below, which demonstrate annotating a message from a dataset of customer hotel reviews.

The main difference is in Step 2, where after typing the label name, you must always select either positive or negative sentiment, denoted by the green or red face icons (this step has been repeated for both the ‘Price’ and the ‘Room > Size’ labels).

One important thing to remember when applying labels with sentiment, is that it's important to create a taxonomy with neutral label names (where possible). For example, 'Price' has been used above rather than 'Expensive', as 'Price' is neutral whereas 'Expensive' in inherently negative.

The selection of negative sentiment for a label with a neutral name would capture the instances where the message is expressing a negative perception of the label.

Much of the time it will be obvious which sentiment you should choose when you apply a label, based on the inherent positivity or negativity of the language (e.g. the 'Price' and 'Room > Size' examples above).

For certain labels, the concept may not lend itself to a neutral name and will be inherently negative or positive, and thus always always be applied with only one sentiment. For example, 'Error' related labels will typically all be applied with negative sentiment. This is fine, but should be applied consistently.

Sometimes, however, it can be quite unclear. If the language in a message is very neutral in tone, we need to think more carefully about which sentiment to apply. Here there are two main things to consider:

Message metadata

The first is to look at the message's metadata. For customer feedback related messages (the most common type of data in a sentiment-enabled dataset), there will often be some kind of score or rating associated with a message (e.g. NPS score). You can often use these scores to gauge whether a message that appears neutral in tone, is more positive or negative in sentiment - i.e. a customer rarely leaves an NPS score of 10 if they're unhappy.

If you consistently apply label sentiment for messages that are neutral in tone, based on a 'score' metadata field, the model can learn to pick up on this and predict the sentiment accordingly.

Consistency of application

The second is to be consistent in how you apply the sentiment for a label when it's quite neutral in tone, and there's no other differentiator (e.g. a 'score' related metadata field).

If it's more common for feedback to be positive for a certain label, assume it's positive unless the message is explicitly negative, and vice versa. If you're not consistent, however, the model will struggle to predict the sentiment.

Another important thing to consider when using sentiment analysis is that the model applies each label (root and leaf) independently, so you can have two leaf labels from the same parent label that have different sentiments.

In these instances you must then judge what the overall sentiment for the parent label is. In this example below, the parent label ‘Room’ is positive overall.

If both leaf labels have the same sentiment, then the model will infer that the parent label also has a negative sentiment and only the leaf labels will be shown as pinned labels (though that implies the parent label is also applied).