Aggregation

This page describes how to configure aggregations in kdb Insights Enterpriseto enable efficient data summarization and analysis.

Aggregation

Aggregation is performed by the Aggregator which collects partials results into a single unified response for a query. Similarly to the routing components, the Aggregator is deployed outside of a package so that it can aggregator across packages. It is primarily configured by setting environment variables on the service.

Environment Variables

Configuration can be supplied to the Aggregator using environment variables. Environment variables are configured differently depending on the method of deployment. In all cases, the variables are always string values.

Docker

Kubernetes

In Docker, environment variables are supplied using under an environment key for the target service as a list of key-value pairs.

YAML

Copy

services:
 agg:
 environment:
 - KXI_NAME=agg

In a Kubernetes deployment, environment variables are passed to components using a values file. For the Aggregator, these values are supplied under an aggregator key.

Warning

When running in a kdb Insights Enterprise deployment, values are nested under either a service-gateway or qe-gateway key in the values file.

See enterprise deployment configuration for more details.

YAML

Copy

service-gateway:
 aggregator:
 env:
 KXI_SG_TIMEOUT: "50000"

name	description
`KXI_NAME`	Process name.
`KXI_PORT`	Port.
`KXI_SG_RC_ADDR`	A comma-delimited list of RC process(es) to connect to in the format `host:port,host:port`
`KXI_CUSTOM_FILE`	File containing custom code to load into Agg process.
`KXI_GC_FREQ`	Frequency in milliseconds to run garbage collect in a timer (default: `600000`, set to `0` to disable).
`KXI_SG_TIMEOUT`	Default request timeout in milliseconds (default: `30000`).
`KXI_SG_CONN_TIMEOUT`	Timeout on connection open.
`KXI_SG_REQ_DEL_FREQ`	Time in milliseconds to run the request table delete routine (default: `10000`).
`KXI_HB_FREQ`	Time in milliseconds to run the heartbeat to connected processes (default: `30000`).
`KXI_HB_TOL`	Number of heartbeat intervals a process can miss before being disconnected (default: `3`).
`KXI_ENABLE_FLUSH`	Set to `true` to enable async flush on messages from RC to DA/Agg and from Agg to GW (default `false`).
`KXI_DISC_MODE`	Discovery mode: `"kubernetes"` (currently, the only listed mode available; modes will be extended in future releases).Not required if `KXI_SG_RC_ADDR` is used.
`KXI_RC_LABEL_SELECTOR`	Label selector to discover RCs via Kubernetes-based discovery (default: `app.kubernetes.io/name=resource-coordinator`).
`KXI_RC_CONTAINER_NAME`	RC container name for Kubernetes-based discovery (default: `resource-coordinator`).
`KXI_ORDINAL`	Override Aggregator's ordinal. See ordinals.
`KXI_RETURN_PARTIALS`	Set to `true` to have the Aggregator send the caller the partial results on a failed aggregation. (default: `false`)