Resources

This page for kdb Insights Enterprise explains how to configure and manage compute and storage resources for databases and pipelines, enabling users to optimize performance and scalability through the web interface or CLI.

Each component of the application is assigned a default set of resources. These resources dictate how Kubernetes schedules the application pods by setting requests and limits. This ensures the application components are given a minimum amount of resources while protecting against any rogue workloads over-consuming. These resource requests and limits can be tuned according to the size of your workload.

Kubernetes container resources

The install values file snippet below shows an example of resources for the keycloak component.

It is configured to request a minimum CPU of 100m (0.1 CPUs) and a limit of 500m (0.5 CPUs). Similarly it's configured with a memory request of 1024 Mib and a limit of 3072Mib.

keycloak:
  resources:
    requests:
      cpu: 100m
      memory: 1024Mi
    limits:
      cpu: 500m
      memory: 3072Mi

The kdb Insights Enterprise application consists of a base deployment, and dynamically deployed packages.

The base components can be configured as above by copying the structure above and using the chart name instead of keycloak.

Each package can define the resources for its components, however you may want to configure the defaults across all packages. These would be used if a package is deployed withoutresources defined for a component.

Sidecar resources

Some components are deployed with a sidecar container for discovery or metrics integrations. If you need to configure the sidecar resources, you can set them globally using the snippet below.

global:
  sidecar:
    resources:
      requests:
        cpu: 0.1
        memory: 128Mi
      limits:
        cpu: 0.1
        memory: 128Mi

Scratchpad resources

The scratchpad CPU and memory can be configured with the following snippet within your values.yaml. Note: As of Insights 1.11, the top-level kxi-scratchpad field has been removed. Each user has their own scratchpad pod, so resource limits should be lower than previous versions where all scratchpads ran in a single pod.

kxi-scratchpad-manager:
  kxi-scratchpad:
    resources:
      requests:
        cpu: 100m
        memory: 100Mi
      limits:
        cpu: 500m
        memory: 500Mi
    tmpVolume:
      sizeLimit: 50Mi
      medium: ""

Query resources

The default values for components related to the query path are:

To set resource limits for the resource-coordinator, aggregators, and service gateway, set resources in your install configuration.

service-gateway:
  aggregator:
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  sg-gateway:
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  resource-coordinator:
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

Storage manager and data access processes

To set requests and limits for your databases set a k8sPolicy for the SM and DAP elements of your assemblies.

spec:
  elements:
    sm:
      eoi:
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "2Gi"
            cpu: "200m"
      eod:
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "2Gi"
            cpu: "200m"
      dbm:
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "500Mi"
            cpu: "200m"
      k8sPolicy:
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "500Mi"
            cpu: "200m"
    dap:
      instances:
        hdb:
          k8sPolicy:
            resources:
              requests:
                memory: "64Mi"
                cpu: "250m"
              limits:
                memory: "128Mi"
                cpu: "500m"

Query environments

If query environments are enabled their resources are set to match the resource limits of the main query processes. Refer to the Query environments setup documentation for details on how to enable or disable these resources.

For example, the query environment hdb pod hdb-qe will get the same limits as the hdb pod.

spec:
  queryEnvironment:
    enabled: true
    size: 1
  dap:
    instances:
      hdb:
        k8sPolicy:
          resources:
            requests:
              memory: "64Mi"
              cpu: "250m"
            limits:
              memory: "128Mi"
              cpu: "500m"