Package Shared Objects

A number of configuration objects within the package are shared by different elements. This page details the specifications of these shared objects.

k8sPolicy

Each element defined within the package has a k8sPolicy object. This object allows additional Kubernetes configurations to be applied to the deployment objects created by the package.

Refer to the following database and pipeline examples:

databases/xxx/shards/xxx-shard.yaml:

YAML

Copy

daps:
  instances:
    idb:
      k8sPolicy: {}
    hdb:
      k8sPolicy: {}
sm:
  k8sPolicy: {}
sequencers:
  north:
    k8sPolicy: {}
  south:
    k8sPolicy: {}

/pipelines/examplePipeline.yaml:

YAML

Copy

name: examplePipeline.yaml
worker:
  k8sPolicy: {}
controller:
  k8sPolicy: {}

Within the k8sPolicy, the instances ServiceAccount and PodSecurityContext can be defined.

Additional advanced configuration can be set as detailed in the sections below.

`k8sPolicy.serviceAccount`

The serviceAccount field allows the user to define a service account name to be used.

YAML

Copy

k8sPolicy:
  serviceAccount: "my-svc-acc"

This may be a new service account name, or the name of a preexisting service account. If preexisting, the create field of serviceAccountConfigure should be set to false.

`k8sPolicy.serviceAccountConfigure`

The serviceAccountConfigure field allows the user to override defaults for a component's service account.

YAML

Copy

k8sPolicy:
  serviceAccountConfigure:
    create: false
    automountServiceAccountToken: false

By default, the KXI Operator creates a service account for each component.

This configuration allows for the creation to be disabled, and the automatic mounting of a service account token to also be disabled. Refer to configuring a service account for additional information.

name	type	required	description
`automountServiceAccountToken`	`boolean`	No	Mount the service account token to the container
`create`	`boolean`	No	Create a service account for component

`k8sPolicy.resources`

The resources field allows the user to define container resources.

YAML

Copy

k8sPolicy:
  resources:
    applyResources: true
    requests:
      memory: "64Mi"
      cpu: "250m"
    limits:
      memory: "128Mi"
      cpu: "500m"

Resource configuration allows limits and requests to be set for containers. The Kubernetes scheduler attempts to place the pod on a node with sufficient resources to meet a request and enforces limits on a pod.

Warning

When a process in the container tries to consume more than the allowed amount of memory, the system kernel terminates the process that attempted the allocation, with an out of memory (OOM) error.

name	type	required	description
`applyResources`	`boolean`	No	Where resources have not been defined, they may be defaulted. Setting to `false` disables the defaults from being applied to the container.
`limits.cpu`	`string`	No	Enforced CPU usage limit in units of Kubernetes CPUs.
`limits.memory`	`string`	No	Enforced maximum memory in bytes. You can express memory as a number of bytes or a number with a unit. See managing resources for more details
`requests.cpu`	`string`	No	Requested container CPU in units of Kubernetes CPUs.
`requests.memory`	`string`	No	Requested container memory in bytes. You can express memory as a number of bytes or a number with a unit. See managing resources for more details
`tmpDirSize`	`string`	No	Each container requires a `tmp` directory mounted to the container. This optional field allows you to request a specific size for the attached volume using a volume capacity

Tip

Memory and CPU requests are the baseline that your process is guaranteed to have available to it for processing. In general, it is recommended that if a limit is specified, it should match the request. For processes that require a significant amount of memory, such as the Storage Manager, it is recommended that no limit is set.

`k8sPolicy.nodeSelector`

The nodeSelector field allows the user to define nodeSelector configuration for workloads.

YAML

Copy

k8sPolicy:
  nodeSelector:
    disktype: ssd

name	type	required	description
`nodeSelector`	`object`	No	Map of key-value pairs

`k8sPolicy.affinity`

The affinity object allows the user to define affinity and anti-affinity configuration for workloads.

YAML

Copy

k8sPolicy:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: topology.kubernetes.io/zone
            operator: In
            values:
            - antarctica-east1
            - antarctica-west1
    ...
    podAffinity:
    ...
    podAntiAffinity:
    ...
    podAntiAffinityPreset: "soft"

Pod affinity allows for more control over pod scheduling. Where nodeSelector may be used for simple scheduling, podAffinity allows for greater range of constraints.

There are currently two types of Node affinity, called requiredDuringSchedulingIgnoredDuringExecution and preferredDuringSchedulingIgnoredDuringExecution

requiredDuringSchedulingIgnoredDuringExecution only schedules pods on nodes matching criteria, whereas preferredDuringSchedulingIgnoredDuringExecution attempts to schedule, but on failure runs on non-matching nodes.

Inter-pod affinity and anti-affinity allow you to constrain which nodes your pod is eligible to be scheduled to run on.

The rules are of the form "this pod should (or, in the case of anti-affinity, should not) run in an X if that X is already running one or more pods that meet rule Y". Y is expressed as a LabelSelector with an optional associated list of namespaces.

As with nodeAffinity there are two types of pod affinity and anti-affinity, called requiredDuringSchedulingIgnoredDuringExecution and preferredDuringSchedulingIgnoredDuringExecution.

The affinity object also includes a podAntiAffinityPreset. This allows the user to configure none, hard, hard-az, soft or soft-az without the need to populate the podAntiAffinity object. See the table below for definitions of the preset affinity values. Setting this field tells the Operator to default the podAntiAffinity object for that component.

affinity	description
`none`	No preset affinity. Pods can be scheduled on the first available node.
`soft`	Prefer scheduling workloads on separate nodes to avoid cascading node failure. If there are no more nodes available, this preset allows scheduling matching workloads on the same node.
`hard`	Require scheduling of workloads to be on separate nodes. If no more nodes are available, workload fails scheduling.
`soft-az`	Prefer scheduling workloads in separate availability zones. If there are no more availability zones, this preset allows scheduling matching workloads in the same zone.
`hard-az`	Require scheduling of workloads to be in separate availability zones. If no more availability, zone workload scheduling fails.

Note

Where podAntiAffinity has been set, this overrides any configuration set within the podAntiAffinityPreset field.

name	type	required	description
`nodeAffinity`	`object`	No	Node Affinity is a set of conditions for a node to meet for Pod scheduling on a node
`podAffinity`	`object`	No	Pod Affinity is a set of conditions for additional workloads to be met for Pod scheduling on a node
`podAntiAffinityPreset`	`string`	No	When set, the operator uses preset anti-affinity configuration for workloads. `none`, `hard`, `soft`, `hard-az` or `soft-az` anti-affinity may be set. See the table above for details on the presets.
`podAntiAffinity`	`object`	No	Pod Anti-affinity is a set of conditions for additional workloads to be met for Pod scheduling on a node

`k8sPolicy.tolerations`

The tolerations field allows the user to define tolerations configuration for workloads.

YAML

Copy

k8sPolicy:
  tolerations:
  - key: "example-key"
    operator: "Exists"

Nodes may be tainted, as means to prevent certain pods from being scheduled on that node. A taint is a key-value pair. Several taints are pre-existing and set by Kubernetes.

Tolerations are applied to pods, and allow (but do not require) the pods to schedule onto nodes with matching taints.

name	type	required	description
`tolerations`	`object`	No	Pod node taint tolerations

`k8sPolicy.terminationGracePeriodSeconds`

The terminationGracePeriodSeconds field allows the user to define terminationGracePeriodSeconds configuration for workloads.

YAML

Copy

k8sPolicy:
  terminationGracePeriodSeconds: 60

This sets the length of time in seconds the container is allowed to shut down before the pod is killed.

name	type	required	description
`terminationGracePeriodSeconds`	`integer`	No	The amount of time in seconds to wait for a workload to complete before forcefully killing it.

`k8sPolicy.podSecurityContext`

The podSecurityContext field allows for the pod level security context to be configured for the element pods.

YAML

Copy

k8sPolicy:
  podSecurityContext:
    runAsUser: 1000
    fsGroup: 1000

It holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of securityContext take precedence over field values of podSecurityContext.

`k8sPolicy.securityContext`

The securityContext allows for a container level security context to be configured.

YAML

Copy

k8sPolicy:
  securityContext:
    runAsUser: 1000
    fsGroup: 1000
    readOnlyRootFilesystem: true
    allowPrivilegeEscalation: false