Communications Mining user guide

Coverage is a term frequently used in machine learning (ML), and relates to how well a model covers the data it used to analyse. In Communications Mining™, this relates to the proportion of messages in the dataset that have informative label predictions, and is presented in the Validation page as a percentage score.

Informative labels are those labels that the platform understands to be useful as standalone labels, by looking at how frequently they are assigned with other labels. Labels that are always assigned with another label are down-weighted when the score is calculated. For example, parent labels that are never assigned on their own, or Urgent, if it is always assigned with another label.

The following visual shows what low versus high coverage would look like across an entire dataset. Imagine the shaded circles are messages that have informative label predictions:

As a metric, coverage is a very helpful way of understanding if you have captured all of the different potential concepts in your dataset, and whether you have provided enough varied training examples for them so that the platform can effectively predict them.

In almost all cases, the higher the coverage of a model is, the better it performs, but you should not take it into account in isolation when checking model performance.

It is also very important that the labels in the taxonomy are healthy, meaning that they have high average precision, and no other performance warnings, and that the training data is a balanced representation of the dataset as a whole.

If your labels are unhealthy or the training data is not representative of the dataset, then the coverage of your model that the platform calculates will be unreliable.

Your model having high coverage is particularly important if you are using it to drive automated processes.

For more details on model coverage, and how to check the coverage of your model, check Understanding and improving model performance.