insights
latest
false
- Release Notes Cloud Insights
- Getting Started
- Access and Permissions
- Notifications
- Interacting with Insights
- Action Center Integration
- Automation Hub Integration
- Autopilot integration
- Document Understanding Integration
- Healing Agents integration
- License monitoring integrations
- Maestro Integration
- Test Manager Integration
- Real-time Monitoring
- Real Time Data Export
- Licensing
- Troubleshooting
Insights
Last updated Sep 24, 2025
In this topic you can learn how to leverage the Insights real-time data export feature with AWS SQS to send data to ElasticSearch and use it to populate real-time monitoring data into Kibana dashboard.
Before you can start viewing real-time monitoring data in Kibana, the following hosts must be set up:
There is no out-of-the-box support for ElasticSearch consuming SNS messages. To do this, an SNS -> SQS -> Elastic pipeline needs to be configured. An SQS queue can be configured to be the output of an SNS topic. To do so, follow the instructions from the official AWS documentation.
To configure a new LogStash pipeline from the UI, login to Kibana user the
elastic username and navigate to and use the following example for the pipeline
template:input {
sqs {
access_key_id => "<access_key_id>"
secret_access_key => "<secret_access_key>"
queue => "hztest_sns_sqs_output"
region => "us-west-2"
}
}
filter {
}
output {
elasticsearch {
index => "test_aws_sqs"
}
}input {
sqs {
access_key_id => "<access_key_id>"
secret_access_key => "<secret_access_key>"
queue => "hztest_sns_sqs_output"
region => "us-west-2"
}
}
filter {
}
output {
elasticsearch {
index => "test_aws_sqs"
}
}Note:
- The
regionsparameter is optional, but if missing the default region is set tous-east-1, and the configuration will fail if the SQS is defined in a different region. - For secrets like ElastiscSearch password or AWS IAM credentials, consider using logstash-keystore.
- If a new index is created to be able to discover it do make sure to add the index to the indexes pattern via .
To test the configuration, run the following command on the LogStash, where
test_config is a file with the pipeline definition above.
/usr/share/logstash/bin/logstash --config.test_and_exit -f ./test_config/usr/share/logstash/bin/logstash --config.test_and_exit -f ./test_config
