Amazon Kinesis Data Firehose monitoring integration

Importante

Enable the AWS CloudWatch Metric Streams integration to monitor all CloudWatch metrics from your AWS services, including custom namespaces. Individual integrations are no longer our recommended option.

New Relic includes an integration for collecting your Amazon Kinesis Data Firehose data. This document explains how to activate this integration and describes the data that can be reported.

Features

Amazon Kinesis Data Firehose provides a simple way to capture and load streaming data. It can capture, transform, and load streaming data into Amazon Kinesis Data Analytics, Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service, enabling near real-time analytics.

New Relic's Kinesis Data Firehose integration reports data such as indexed records and bytes, counts of data copied to AWS services, the age and freshness of records, and other metric data.

Activate integration

To enable this integration follow standard procedures to Connect AWS services to New Relic.

Configuration and polling

You can change the polling frequency and filter data using configuration options.

Default polling for the Amazon Kinesis Data Firehose integration:

New Relic polling interval: 5 minutes
Amazon CloudWatch data interval: 1 minute

Find and use data

To find your integration data, go to one.newrelic.com > All capabilities > Infrastructure > AWS and select one of the Kinesis Data Firehose integration links.

You can query and explore your data using the QueueSample event type, with a provider value of KinesisDeliveryStream.

For more on how to use your data, see Understand and use integration data.

Metric data

This integration collects the following metrics:

Name	Description
`deliveryToElasticsearchBytes`	The number of bytes indexed to Amazon ES over the specified time period.
`deliveryToElasticsearchRecords`	The number of records indexed to Amazon ES over the specified time period.
`deliveryToElasticsearchSuccess`	The sum of the successfully indexed records over the sum of records that were attempted.
`deliveryToRedshiftBytes`	The number of bytes copied to Amazon Redshift over the specified time period.
`deliveryToRedshiftRecords`	The number of records copied to Amazon Redshift over the specified time period.
`deliveryToRedshiftSuccess`	The sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands.
`deliveryToS3Bytes`	The number of bytes delivered to Amazon S3 over the specified time period.
`deliveryToS3DataFreshness`	The age (from getting into Kinesis Firehose to now) in seconds of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to the S3 bucket.
`deliveryToS3Records`	The number of records delivered to Amazon S3 over the specified time period.
`deliveryToS3Success`	The sum of successful Amazon S3 put commands over the sum of all Amazon S3 put commands.
`incomingBytes`	The number of bytes ingested into the Kinesis Firehose stream over the specified time period.
`incomingRecords`	The number of records ingested into the Kinesis Firehose stream over the specified time period.
`putRecordBatchLatency`	The time taken in milliseconds per `PutRecordBatch` operation, measured over the specified time period.
`putRecordBytes`	The number of bytes put to the Kinesis Firehose delivery stream using `PutRecord` over the specified time period.
`putRecordLatency`	The time taken in milliseconds per `PutRecord` operation, measured over the specified time period.