Reduce data ingested

If you've reviewed your data ingestion and want to send less data and reduce the data ingested, we have these solutions for you:

• Set global.lowDataMode

• Change the scrape interval

• Filter namespaces

Let's explain each option.

Reduce data ingested by setting global.lowDataMode

If you're looking to cut costs, you can modify the global.lowDataMode parameter value. This will cut down on the number of metrics you send. The modification of this parameter will depend on the method you used to install Kubernetes:

• Guided install (New Relic CLI, Helm, manifest). The installation command includes lowDataMode by default because it has these parameters: NRI_CLI_LOW_DATA_MODE, NRI_CLI_PROMETHEUS_AGENT_LOW_DATA_MODE, and NRI_CLI_LOGGING_LOW_DATA_MODE.

• Helm. Set the global.lowDataMode parameter in the nri-bundle chart to true.

• Manifest. LowDataMode is only for Helm charts, and Helms charts are templates applied to manifests. When someone applies LowDataMode in the Helm chart, it makes a few small changes to the final manifest. The changes in the manifests are about specific settings, but not for a parameter named LowDataMode.

The lowDataMode parameter affects these specific components of the nri-bundle chart:

[
  {
    "cluster_name": "api-test",
    "kubernetes": {
      "annotations": {
        "kubernetes.io/psp": "eks.privileged"
      },
      "container_hash": "fryckbos/test@sha256:5b098eaf3c7d5b3585eb10cebee63665b6208bea31ef31a3f0856c5ffdda644b",
      "container_image": "fryckbos/test:latest",
      "container_name": "newrelic-logging",
      "docker_id": "134e1daf63761baa15e035b08b7aea04518a0f0e50af4215131a50c6a379a072",
      "host": "ip-192-168-17-123.ec2.internal",
      "labels": {
        "app": "newrelic-logging",
        "app.kubernetes.io/name": "newrelic-logging",
        "controller-revision-hash": "84db95db86",
        "pod-template-generation": "1",
        "release": "nri-bundle"
      },
      "namespace_name": "nrlogs",
      "pod_id": "54556e3e-719c-46b5-af69-020b75d69bf1",
      "pod_name": "nri-bundle-newrelic-logging-jxnbj"
    },
    "message": "[2021/09/14 12:30:49] [ info] [engine] started (pid=1)\n",
    "plugin": {
      "source": "kubernetes",
      "type": "fluent-bit",
      "version": "1.8.1"
    },
    "stream": "stderr",
    "time": "2021-09-14T12:30:49.138824971Z",
    "timestamp": 1631622649138
  }
]

[
  {
    "cluster_name": "api-test",
    "container_name": "newrelic-logging",
    "namespace_name": "nrlogs",
    "pod_name": "nri-bundle-newrelic-logging-jxnbj",
    "message": "[2021/09/14 12:30:49] [ info] [engine] started (pid=1)\n",
    "stream": "stderr",
    "timestamp": 1631622649138
  }
]

HTTP_SPAN_LIMIT: 750
DB_SPAN_LIMIT: 250
COLLECT_INTERVAL_SEC: 15

Reduce data ingested by changing the scrape interval

The New Relic Kubernetes integration allows you to change the scrape interval when metrics are gathered from the cluster. This lets you choose the right balance between data resolution and usage. We suggest choosing a scrape interval between 15 and 30 seconds for the best experience.

The way you modify the scrape interval depends on the method you used to install Kubernetes:

• Guided install (New Relic CLI, Helm, manifest): You can't modify this value following our guided install flow.

• Helm: Set the scrape interval in the nri-bundle chart to the value you want.

• Manifest: Set the scrape interval in the manifest configuration YAML file to the value you want. If you followed our installation instructions, the file was called newrelic-manifest.yaml.

If you're using Helm or manifest and want to change the scrape interval, just add the interval value under the newrelic-infrastructure section. Take a look at this example of the values-newrelic.yaml file to see how it looks for Helm:

global:
  licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
  cluster: _K8S_CLUSTER_NAME_

# ... Other settings

# Configuration for newrelic-infrastructure
newrelic-infrastructure:
  # ... Other settings
  common:
    config:
      interval: 25s

Reduce data ingested by filtering namespaces

You can label namespaces to filter which ones the Kubernetes integration scrapes. All namespaces scrape by default.

We use the namespaceSelector in the same way Kubernetes does. To include only namespaces matching a label, just change the namespaceSelector. Add the following to your values-newrelic.yaml file, under the newrelic-infrastructure section:

common:
  config:
    namespaceSelector:
      matchLabels:
        key1 : "value1"

Examples with namespaces

global:
licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
cluster: _K8S_CLUSTER_NAME_

# ... Other settings 

# Configuration for newrelic-infrastructure
newrelic-infrastructure:
# ... Other settings 
common:
    config:
    namespaceSelector:
        matchLabels:
        newrelic.com/scrape: "true"

common:
config:
    namespaceSelector:
    matchExpressions:
    - {key: newrelic.com/scrape, operator: NotIn, values: ["false"]}
    - {key: key1, operator: In, values: ["value1"]}

global:
licenseKey: _YOUR_NEW_RELIC_LICENSE_KEY_
cluster: _K8S_CLUSTER_NAME_

# ... Other settings 

# Configuration for newrelic-infrastructure
newrelic-infrastructure:
# ... Other settings
common:
    config:
    namespaceSelector:
        matchExpressions:
        - {key: newrelic.com/scrape, operator: NotIn, values: ["false"]}

You can see a full list of settings that you can modify in the chart's README file.

How can I find out which namespaces are excluded?

The K8sNamespace sample shows all the namespaces within the cluster. The nrFiltered attribute determines whether we're going to scrape the data related to the namespace. This query shows you which namespaces are being monitored:

FROM K8sNamespaceSample SELECT displayName, nrFiltered WHERE clusterName = <clusterName> SINCE 2 MINUTES AGO

What data is being discarded from the excluded namespaces?

These samples won't be available for the excluded namespaces:

• K8sContainerSample
• K8sDaemonsetSample
• K8sDeploymentSample
• K8sEndpointSample
• K8sHpaSample
• K8sPodSample
• K8sReplicasetSample
• K8sServiceSample
• K8sStatefulsetSample
• K8sVolumeSample