Upgrading kdb Insights Enterprise

This page provides the steps to upgrade kdb Insights Enterprise from a current install.

If you are upgrading kdb Insights Enterprise on Azure Marketplace, reach out to KX Support.

Prerequisites

Warning

The upgrade process requires downtime to ensure data integrity can be guaranteed. This means that before any upgrade is scheduled, a number of pre-flight checks are necessary.

Ensure the latest version of kxi-cli is installed
Pre-upgrade Versions

The currently running versions of kdb Insights Enterprise and Operator should be noted for a rollback scenario.

Operator

bash

Copy

helm ls -n kxi-operator

bash

Copy

NAME        NAMESPACE       REVISION    UPDATED                                 STATUS      CHART                       APP VERSION
insights    kxi-operator    1           2022-04-01 15:39:44.955236379 +0000 UTC deployed    kxi-operator-1.0.0          1.0.0

Note

Overriding the Operator version

The CLI installs the latest kdb Insights Operator minor version that matches the specified $INSIGHTS_VERSION. You can override this by passing the --operator-version command line option.

Application

bash

Copy

helm ls -n $INSIGHTS_NAMESPACE

bash

Copy

NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
insights kxi 1 2022-04-01 16:57:00.144126754 +0000 UTC deployed insights-1.0.0 1.0.0

kubectl

You must have the kubectl tool installed on your machine and configured to access the same cluster as the kdb Insights Enterprise deployment you are upgrading.

Variables

Have the following information to hand:

variable	example	further details
INSIGHTS_NAMESPACE	kxi	Namespace of existing deployment
INSIGHTS_VERSION	1.13.0	Version to upgrade to
INSTALL_CONFIG_FILE	values.yaml	Install configuration
PRE_UPGRADE_INSIGHTS_VERSION	1.12.0	Current application version
PRE_UPGRADE_OPERATOR_VERSION	1.12.0	Current Operator version

Retrieve install configuration

Retrieve the INSTALL_CONFIG_FILE from the existing installation with the following command.

bash

Copy

kxi install get-values --namespace $INSIGHTS_NAMESPACE > $INSTALL_CONFIG_FILE

Configure a default profile

If you are upgrading to kdb Insights Enterprise version 1.13 or higher, you must configure a default profile for the environment you are upgrading, or you must specify a profile override.

Other version-specific considerations

Refer to the release notes for any specific upgrade considerations for your version of kdb Insights Enterprise. This may include customizing the install configuration file retrieved previously.

Stop ingestion feeds

Stop all feeds into the application to avoid data loss with inflight data.

Upgrade with CLI

Upgrading using the CLI does not remove the persisted data backing kdb Insights Enterprise configuration or databases. These remain as persisted volumes within the Kubernetes cluster.

The steps performed by the CLI are:

Backup packages and any running assemblies
Teardown packages
Upgrade application, Operator, and CRDs
Convert assemblies that were running prior to the upgrade to packages, if you have chosen to convert them
Restore the packages

Note

The use of kxi assembly to deploy assembly yaml files has been deprecated. As part of the upgrade, you are prompted to choose if you want to convert any running assemblies to packages and redeploy using kxi pm after the upgrade.

If you choose not to convert these assemblies, the CLI does not redeploy them after the upgrade, they must be manually deployed.

If there are no packages or assemblies running, the CLI upgrades the base system and skips the package and assembly steps.

To upgrade with a previously configured default profile, run the following command which steps through the upgrade, prompting you as required.

bash

Copy

kxi install upgrade --filepath $INSTALL_CONFIG_FILE --version $INSIGHTS_VERSION

bash

Copy

Backing up packages
Persisted package artifact for kxi-db to <.kxi BACKUP FILE>
Persisted package artifact for insights-demo to <.kxi BACKUP FILE>

...
Upgrading insights and operator
...
Reapplying packages
...
Upgrade to version x.x complete
...

Note

The previous example shows two sample packages; you may have different packages, or none at all.

To upgrade with a profile override, run:

bash

Copy

kxi --profile <profile name> install upgrade --version 1.13.0 --operator-version 1.13.0 --filepath values.yaml

Note

If you want to configure a new profile, run kxi configure --profile <name of new profile> command.

Note

If you have not previously set encryption of data in transit, you will be prompted to enable it during upgrade.

If you have previously enabled encryption for data in transit, the default behavior is to keep the existing, pre-upgrade setting. To change this setting, add the --encryption true/false command line parameter.

Post-upgrade checks

Check the system is running correctly.

Expected version of kdb Insights Enterprise is running

bash

Copy

kubectl describe cm insights-config | grep version
version: insights-1.1.1

Operator installed

bash

Copy

helm ls -n kxi-operator

bash

Copy

NAME         NAMESPACE         REVISION     UPDATED                                     STATUS         CHART                     APP VERSION
insights     kxi-operator     1             2022-04-11 15:39:44.955236379 +0000 UTC     deployed     kxi-operator-1.1.0         1.1.0

Application installed

bash

Copy

helm ls -n $INSIGHTS_NAMESPACE

bash

Copy

NAME         NAMESPACE     REVISION     UPDATED                                     STATUS         CHART             APP VERSION
insights     kxi         1             2022-04-11 16:57:00.144126754 +0000 UTC     deployed     insights-1.1.1     1.1.1

Operator in running state

bash

Copy

kubectl get pods -n kxi-operator

bash

Copy

NAME                                         READY     STATUS         RESTARTS    AGE
insights-kxi-operator-6df4bcfddc-gnnjj         1/1     Running     0             85m
insights-kxi-operator-6df4bcfddc-lqmrl         1/1     Running     0             85m
insights-kxi-operator-6df4bcfddc-z45wq         1/1     Running     0             85m

Application in running state

bash

Copy

kubectl get pods -n $INSIGHTS_NAMESPACE

bash

Copy

NAME                                     READY     STATUS         RESTARTS     AGE
insights-aggregator-9959bdc87-4pg7z     2/2     Running     0             9m47s
...
...
insights-sg-gateway-596b486d9f-pmhf4     1/1     Running     0             9m47s
insights-sg-gateway-596b486d9f-qvzbr     1/1     Running     0             9m47s

Assembly CRDs redeployed

bash

Copy

kubectl get crds | grep insights.kx.com

bash

Copy

assemblies.insights.kx.com             2022-04-11T15:23:53Z
assemblyresources.insights.kx.com     2022-04-11T15:23:54Z

Ensure packages are up and running successfully. The following example shows two sample packages; you may have different packages, or none at all.

bash

Copy

kxi pm list

bash

Copy

╭────────────-──┬─────────┬────────┬─────────────────────┬──────────────────────────────────────╮
│ name          │ version │ access │ owner.name          │ id                                   │
├─────────────-─┼─────────┼────────┼─────────────────────┼──────────────────────────────────────┤
│ kxi-db        │ 1.0.0   │ ARWX   │                     │ 529ab8df-3f1a-4b5b-8d69-29ecee6ca2f5 │
│ insights-demo │ 0.0.1   │ ARWX   │                     │ add333ad-bdd2-41a9-bcc7-a24fdbee598f │
╰──────────────-┴─────────┴────────┴─────────────────────┴──────────────────────────────────────╯

If all post-upgrade checks pass, you can restart any feeds into the application that were stopped before the upgrade.

Rollback

Note

Upgrade the KXI CLI to 1.14 first

Ensure you are using version 1.14 of the kdb Insights Enterprise CLI before following the process outlined below.

You can use the kxi install rollback command to roll back to an older version of kdb Insights Enterprise.

Run kxi install history to review the release history and choose which revision you want to roll back to:

shell

Copy

$ kxi install history --show-operator
REVISION    UPDATED                     STATUS          CHART               APP VERSION     DESCRIPTION
1           Tue Feb 28 14:10:03 2025    deployed        insights-1.12.0     1.12.0          Install complete
2           Tue Feb 28 14:16:41 2025    failed          insights-1.13.0     1.13.0          Upgrade complete
3           Tue Feb 28 14:25:55 2025    failed          insights-1.13.1     1.13.1          Upgrade "insights" failed: post-upgrade hooks failed: timed out waiting for the condition

1           Tue Feb 28 14:09:52 2025    superseded      kxi-operator-1.12.1  1.12.1         Install complete
2           Tue Feb 28 14:15:37 2025    superseded      kxi-operator-1.13.0  1.13.0         Upgrade complete
3           Tue Feb 28 14:24:54 2025    superseded      kxi-operator-1.13.2  1.13.2         Upgrade complete

If you only want kdb Insights Enterprise revisions, remove the --show-operator from the above command.

Important

To initiate a rollback, it's important to note that the rollback command relies on the REVISION value rather than the APP VERSION. In order to revert back to version 1.3.0 of kdb Insights Enterprise, you must identify the correct REVISION number, which in this case is simply 1.

Note

The version of kdb Insights Operator and kdb Insights Enterprise must match.

If the currently installed version of kdb Insights Operator does not align with the rollback version of kdb Insights Operator, it's necessary to perform the rollback on both.

Run kxi install rollback with the chosen revisions:
shell
Copy
$ kxi install rollback $INSIGHTS_REVISION --operator-revision $OPERATOR_REVISION Rolling Insights back to version 1.12.0 and revision 1. And operator back to version 1.12.0 and revision 1 [y/N]: y Backing up packages No packages to back up ...
Note

If you do not provide an explicit revision, kxi install rollback chooses the most recent one.

Important

Rolling back kdb Insights Enterprise only

Omit --operator-revision if you only want to rollback kdb Insights Enterprise.

Warning

The rollback process fails if it finds any persistent volumes with IDB or HDB suffix without a valid database structure. If these are not needed, delete them and retry the rollback. If they are needed, ensure they have a valid database structure and retry the rollback.

Failed upgrades

The following is an example output from a failed upgrade attempt from version 1.12.0, where only assemblies were running before the upgrade.

text

Copy

...
Persisted assembly definitions for ['dfx-assembly', 'iot-assembly'] to /tmp/tmp7ny84esa-kxi-assembly-state.yaml
...
Error: Command '['helm', 'upgrade', '--install', '-f', './values.yaml', 'insights', 'kx-insights/insights', '--version', '1.13.0', '--namespace', 'insights']' returned non-zero exit status 1.

State files are preserved in the event of an upgrade failure.

After recovering the state of kdb Insights Enterprise, you can restore state files as follows.

Restoring assemblies

Restoring packages

Where your upgrade attempt was from any 1.12 version or below, you can restore assembly state files to their running state by executing the following command:

bash

Copy

kxi assembly create --filepath /tmp/tmp7ny84esa-kxi-assembly-state.yaml

Packages can be restored using package manager by executing a command such as:

bash

Copy

kxi pm deploy kxi-db

Updating your application configuration

You can apply advanced configurations to your running instance of kdb Insights Enterprise by upgrading with an updated install config file.

Retrieve your install configuration from the existing application and update the resulting INSTALL_CONFIG_FILE.
Upgrade your installation with the updated values file, passing the same version number as the running installation.