Stream Settings

This page describes the configuration options available in the Stream Settings tab in the kdb Insights Enterprise Web Interface.

A stream in kdb Insights Enterprise is a deployment of a Reliable Transport (or RT) instance. It enables highly available, high performance messaging within and beyond the cluster.

Stream Settings tab

The Stream Settings tab is accessed when you create a new database or open an existing database in the web interface.

Through these settings you can configure the following types of streams:

• Primary stream

• Additional streams

Refer to Resources, for details about configuring the resources used by streams.

Stream Settings

The following table describes the stream properties. These apply to both primary and additional streams.

property	Description	Default	Required
Name	Specifies the name of the stream internally. This name must be unique in this database.   You can rename the stream by clicking the current name and entering a new one. However, be aware that renaming a stream after deployment may result in a loss of data.   The stream name is used for name-based data routing, so ensure it follows the required format, using alphanumeric characters and dashes.   This name identifies the stream that a kdb Insights Stream reader or writer node subscribes or publishes to.	The primary stream is named after the database in which it resides.   Additional streams are automatically named using the pattern: database-name-incremental-number. For example: insights-demo-1, insights-demo-2	Required for Primary Stream.
External data ingress	Click the toggle to configure the stream to allow for external ingress connections. This is an optional endpoint for ingesting data into the stream from an external data source. Enable this if you need to send data into a stream from an external source, such as a feedhandler or custom publisher. Either accept the default name, or enter a new name. The name must be unique across all databases. Refer to the note on external reference below.	Disabled by default When enabled, the reference name field is automatically populated with a default value in the format database-name-ingr.	Optional
External data egress	Click the toggle to configure the stream to allow for external egress connections. This is an optional endpoint for publishing data from the stream to an external subscriber. Enable this if you need to send data out of a stream to an external source. Either accept the default name or enter a new name. The name must be unique across all databases. Refer to the note on external reference below.	Disabled by default. When enabled, the reference name field is automatically populated with a default value in the format database-name-egr.	Optional

The following shows the Stream Settings with External data ingress and External data egress enabled for the Primary Stream.

External reference

The reference name used for the external data ingress into and egress out of streams is used as the SDK_CLIENT_UID configuration value by external clients to reference the stream for publishing or subscribing from outside of the kdb Insights Enterprise cluster.

The reference may only contain lower case alphanumeric characters, - or ., and must start and end with an alphanumeric character.

These names are used when enrolling a client who wishes to publish or subscribe to an RT stream from an external application and must be unique. The enrollment steps are documented here.

Configure the primary stream

In kdb Insights Enterprise, all data written to the database goes through the primary stream which is created automatically when you create a new database. Each database must have a primary stream.

To configure the primary stream:

1. When you create a new database navigate to the Stream Settings tab.



2. Configure the properties, as described in Stream Settings.

3. Click Save.

Configure additional streams

Additional optional streams handle all data that needs to be passed into and out of the cluster as well as between components in the cluster. These streams do not write directly to the database.

For example, a publishing client external to the cluster, such as a feedhandler, might send data in CSV or JSON format, which needs to be decoded and have a schema applied before being ingested into the database.

You can configure additional streams to:

• Ingest raw data that requires transformation (through a pipeline)

• Subscribe to data from outside the cluster

• Manage load received by a subscriber

Additional streams are deployed with identical resource specifications as the primary stream.

1. Click + Add Stream to display the settings for the new stream.



2. Configure the settings as described in Stream Settings.

3. Click Save.

Delete an additional stream

To delete an additional stream:

1. Click the trashcan icon to the right of the additional stream.

2. A note is displayed saying the stream has been marked for deletion and will be deleted once you click Save.

3. Click Restore Stream to undo this deletion.

Further reading

• Create a database

• Configure database schema

• Configure database settings

• Configure database resources

• Deploy a database