Observability

This page explains how to configure the application observability.

It provides details on how to:

• Configure logging

• Configure metrics

• Disable alerts and dashboards

Configure logging

You can configure the log level of your application at installation time using the values below. kdb Insights Enterprise uses log components as a way to identify the origin of a log message. The routings object allows you to set the log level for different components with DEFAULT applying to anything that's not set explicitly.

global:
  logging:
    routings:
      DEFAULT: INFO
      InfoService: DEBUG

The example above sets all components to INFO with the exception of the InfoService component, which is set to DEBUG.

Refer to the logging documentation for more details on logging stacks to deploy.

Configure metrics

kdb Insights Enterprise generates metrics and includes pre-packaged alerts and monitoring dashboards to allow you to check system performance and quickly identify potential issues. You can monitor data flows, component errors, and a variety of other metrics.

By default these are disabled for kdb Insights Enterprise since the application does not deploy with the Prometheus stack. Refer to deploying Prometheus for details on how to deploy the Prometheus stack.

To enable metrics, alerts and dashboards for your deployment, update your Helm values file and set global.metrics.enabled to true.

This is created as part of the install setup.

global:
  metrics:
    enabled: true
    serviceMonitor:
      enabled: true
      interval: 30s
      additionalLabels:
        release: kx-prom

When you have updated the values file, run kxi install run --filepath values.yaml to apply the change.

This deploys Prometheus Service Monitor resources, which scrape kdb Insights Enterprise components and publish metrics to the backend.

Enable Prometheus scraping

There are two methods of enabling Prometheus scraping:

• Prometheus ServiceMonitors

• pod annotations

ServiceMonitors

The global.metrics.serviceMonitor object configures the use of the Prometheus ServiceMonitor object within the application. With this mode enabled, the application creates a set of ServiceMonitor resources, which Prometheus uses to scrape metrics.

The variables in the table above are described as follows:

• enabled - enables the ServiceMonitor mode

• interval - metrics polling interval

• additionalLabels.release - this is a label to match your Prometheus deployment name.

Pod annotations

The other method of enabling the scaping metrics is to use pod annotations instead of using ServiceMonitors. This is useful in situations where the Prometheus operator cannot be used, or another approach is preferred.

The values file below shows the required configuration.

global:
  metrics:
    enabled: true
    useAnnotations: true
    serviceMonitor:
      enabled: false

Setting global.metrics.useAnnotations to true adds Prometheus annotations to the application pods. These annotations are prefixed with prometheus.io.

Disabling alerts and dashboards

To scrape metrics but disable both the install of alerts and dashboards, include the following in the metrics section of your Helm values file:

kxi-monitoring:
 enabled: false

To disable alerts but enable dashboards, include the following in the metrics section of your Helm values file:

kxi-monitoring:
 metrics:
 alerts: false

To disable dashboards but enable alerts, include the following in the metrics section of your Helm values file:

kxi-monitoring:
 grafana:
 dashboards: false