The Data ingestion tab is located in the Data management UI. The data ingestion UI shows a summary of your usage by data source. For organizations with multiple accounts, you can also view the usage of specific accounts.
For detailed definitions of the metrics shown in the chart, see Data ingest sources.
To see the underlying NRQL query used to create that chart, click View NRQL.
For a video showing a brief tour of this and other administration UIs, see Account settings.
Get more detail
To get more detail about a specific data source shown in the chart, hover over the band in the chart and click it. A modal will open up, as shown in the image below.
When you click on a band in the ingestion chart, you'll get more details about that data.
For some technical details about how the chart displays data and error messages, see Chart details.
How you manage your New Relic data will depend on a lot of factors specific to your organization and your needs. That said, here are some general tips for managing data ingest and avoiding unexpected spikes:
Assign team members. Assign team members who'll be in charge of reviewing your ingest on a cadence, and managing it. Ensure they understand the data-related billing factors, including what does and doesn't count towards billing.
Get to know your data. Spend some time getting acquainted with your data. Get to know its ebbs and flows, and understand where it's coming from.
Monitor when you make significant changes. When you first activate a new data-reporting tool, or when you upgrade an agent or integration, or when you make any big change in your system, you should monitor your ingest to ensure there's no unexpected spike in data.
Set up alerts. If you're concerned about specific scenarios causing sudden spikes in data, set up an alert condition to notify you if that occurs. For tips on that, see Usage queries.
Reducing ingest
Below are some tips for common approaches New Relic customers take to reduce the ingest of data that's not helpful to them.
All New Relic solutions have various configuration options that give you control over how data is reported to New Relic. Below we have some popular methods for reducing data ingest, but to learn about all the options available, we recommend reading the docs for specific tools you're using.
Options for adjusting data include:
Configure the sampling rate for transaction events. See agent configurations for Java, .Net, Go, NodeJS, PHP, Python, or Ruby.
Automatic logs in context: Disable or enable via the UI or API, or adjust client-side configuration settings.
Log data forwarding: Unfiltered logs can sometimes result in a large amount of log data being reported. You can adjust the log forwarder configuration to filter log events from the log-sending side.
Drop log data: Manage data ingest via the UI or API.
On ingest, we apply data dropping rules so you won't be charged for data that's not useful. But you can also set your own data dropping rules.
For unexpected increases in network monitoring data, you may consider adjusting the polling interval.
If you have agents or integrations that you don't want at all, you can uninstall/delete those tools. For instructions, see the specific docs for that tool. Keep in mind that if you think there's a chance you might use that tool in the future, simply disabling it might be a better solution than uninstalling it completely.
Data ingestion sources
The data ingestion UI chart shows you a high level breakdown of your billable data usage. The table below explains those sources. In this table, "usage metric group" refers to the value of that source's usageMetric attribute value on the NrConsumption event.
Metric timeslice data averages to one-hour periods after eight days. After 90 days, the permanent metric data continues to be stored in one-hour periods. We currently store the raw metric data for 30 days.
You are only billed for the initial ingest volume. You are not billed for subsequent rollups.
This includes APM events, like Transaction and TransactionError.
This data is reported via the SystemSample, NetworkSample, and StorageSample events.
The usage metric group is InfraHostBytes.
Information related to your servers and virtual machines coming from infrastructure agents, including storage and network data.
Infrastructure processes
This data is stored in the ProcessSample event.
The usage metric group is InfraProcessBytes.
Data related to each process running on the hosts running the infrastructure agent. This feature is turned off by default. For more information, see Process metrics.
Infrastructure integrations
Usage metric group: InfraIntegrationBytes.
Performance data related to applications and services, typically managed by the customer, including data related to Docker containers, Windows services, Nagios checks, and cloud integrations such as managed services in AWS, Azure, and GCP.
Includes logs and any Log_<value> custom data partition that exists.
The usage metric group is LoggingBytes.
Log records are stored on the Log data type by default. Additional custom data partitions will create new data types, which are always prefixed with Log_ and are counted as part of the overall set of log data stored.
With LogExtendedRecord, log messages longer than 4KB are split into multiple events that, when needed, are stitched together to display the original message; this reduces the size of message data. For more on how this data is stored, see our log blobs docs.
Mobile events, including the general Mobile event, MobileRequestError, MobileBreadcrumb, MobileSession, MobileHandledException, MobileCrash, MobileRequest, and MobileJavaScriptError.
Usage metric group: MobileEventsBytes.
This includes the Span data type and OpenTelemetry's SpanEvent.
The usage metric group is TracingBytes.
DistributedTraceSummary events do not count towards ingest.
These are events, including the namespaces of Browser, Browser:EventLog, Browser:JSErrors, and PcvPerf (PageView timing).
Change tracking activity is stored in the Deployment namespace. Individual changes you track average around 200 to 250 bytes toward your ingest.
Events reported by the security features of New Relic are stored in the Security namespace. These are primarily vulnerability reports provided by the Vulnerability Management feature, but may be expanded to include additional products in the future.
Expected event volumes of this type are very low, as the occurrence of vulnerability reports are rare.
Security features still in public preview use a separate Security:Preview namespace and are not billable.
Displays are estimates. The ingest value shown on the ingest chart can vary slightly from the actual amount you may see when running your own query. This is because the calculation used for the chart is an estimate.
No chart data available. The data ingestion chart can display a slightly longer time frame than that covered by your data retention settings. For this reason, you may get message that there's no chart data available.
Chart time buckets. If an account has less than a terrabyte of data, we compute the volume over a 24-hour period; otherwise, we compute it for a 1-hour period.
