Bloomberg B-Pipe Feed Handler Install and Setup

This page explains how to install and set up the Bloomberg B-Pipe feed handler, and provides an example of how to subscribe to additional fields from Bloomberg B-Pipe.

Download the Bloomberg B-Pipe feed handler from the KX downloads portal.

Download the Bloomberg B-Pipe feed handler

# Add the Kx helm repo
helm repo add kx https://portal.dl.kx.com/assets/helm --username <INSERT_EMAIL> --password <INSERT_BEARER_TOKEN>

# Get the rt-bpipe-pub helm charts
helm fetch kx/rt-bpipe-pub --version  1.0.0 --untar --devel

                    

Install the Bloomberg B-Pipe feed handler

The Bloomberg Entitlements accelerator contains a configuration that is used when installing the Bloomberg B-Pipe feed handler, ensuring the feed correctly connects to the accelerator. Unpack the accelerator kxi file to view and update the configuration file as necessary:

# Unpack the accelerator
kxi package unpack fsi-app-bbg-entl-1.0.0.kxi

# View the config file
cat fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml 

                    

Update the configuration file fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml. The following values should be updated as a minimum:

• imagePullSecrets - the name of the secret used to pull images from portal.dl.kx.com.

• bpipeAppName - the app name used to authenticate with the Bloomberg B-Pipe feed.

For more advanced configuration options, refer to additional configuration options.

Install the feed handler using your updated config file:

# Install using updated config file.
helm install rt-bpipe-pub ./rt-bpipe-pub -f fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml
                    

Applying configuration changes to the Bloomberg B-Pipe feed handler

To apply configuration changes to the feed handler, save your changes in your configuration file and use the helm upgrade command as below:

helm upgrade rt-bpipe-pub ./rt-bpipe-pub -f fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml
                    

Stopping the Bloomberg B-Pipe feed handler

The Bloomberg B-Pipe feed handler is stopped using a helm uninstall command:

helm uninstall rt-bpipe-pub
                    

Additional configuration options for the Bloomberg B-Pipe Feed

The configuration settings used specified in fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml are a small sub-set of available settings. All default configuration settings used by the helm chart can be seen in rt-bpipe-pub/values.yaml.

For maintainability, it is recommended to use a custom values.yaml if you wish to override any configuration settings. Example command:

# Upgrade the feed to use accelerator overrides as well as my own custom overrides.
helm upgrade rt-bpipe-pub ./rt-bpipe-pub -f fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml -f my-custom-values.yaml
                    

Using a custom file to override the configuration is recommended for the following reasons:

• It makes version control easy.

• It allows for flexibility, allowing you to use slightly different values across dev, staging, and production environments.

Example: Subscribing to additional fields from Bloomberg B-Pipe

Adding/removing fields to those ingested by the Bloomberg B-Pipe feed handler is a more complex task as several files need to be updated including:

• The feed handler values.

• The accelerator table definitions.

• The fsi-app-bbg-entl-bbg-realtime-l1-1-spwork-0 stream processor (SP) config file: fsi-app-bbg-entl/src/bbg.realtimel1.config.q

In this example, we subscribe to the field PER_TRADE_VWAP_REALTIME and store this in a column called perTradeVwap in the Trade table.

1. Unpack the accelerator file

Unpack the kxi file, as follows:

kxi package unpack fsi-app-bbg-entl-1.0.0.kxi
                    

2. Create a custom values file with the required overrides

• Create a file, for example my-custom-values.yaml.

• Copy the definition of the Trade table from the file fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml.

• Paste this definition into my-custom-values.yaml and add PER_TRADE_VWAP_REALTIME as shown in the below example:

bpipe:
  bpipeTableField3: "trade:EVT_TRADE_IDENTIFIER,EVT_TRADE_BUY_BROKER_RT,EVT_TRADE_SELL_BROKER_RT,TRADE_RPT_PARTY_SIDE_RT,EVT_TRADE_RPT_PARTY_TYP_RT,EVT_TRADE_BIC_RT,EVT_TRADE_MIC_RT,EVT_TRADE_ESMA_TRADE_FLAGS_RT,EVT_TRADE_AGGRESSOR_RT,EVT_TRADE_RPT_CONTRA_TYP_RT,EVT_TRADE_REMUNERATION_RT,EVT_TRADE_ATS_INDICATOR_RT,EVT_TRADE_PRICE_RT,EVT_TRADE_SIZE_RT,EVT_TRADE_CONDITION_CODE_RT,EVT_TRADE_LOCAL_EXCH_SOURCE_RT,TICK_SEQUENCE_NUMBER,PER_TRADE_VWAP_REALTIME"

• Save the file.

• Here we have added PER_TRADE_VWAP_REALTIME to the end of the list of values.

2. Update the Trade table definition in the accelerator using an overlay

It is recommended that overlays are used to customize schemas to ensure upgradability.

• Add an overlay called MyBloombergOverlay:

kxi package add --to fsi-app-bbg-entl patch --name MyBloombergOverlay
                    

• In the newly created file fsi-app-bbg-entl/patches/MyBloombergOverlay.yaml add the column perTradeVwap to the Trade table.

• The file contents look like below:

kind: Package
apiVersion: pakx/v1
metadata:
  name: target
spec:
  uuid: 14fa19e7-0e16-42d7-9602-89526c6b9e64
  manifest: {}
  tables:
    uuid: beb9b6a1-d3d1-4d9f-a2fc-8d6c9ffc93ad
    schemas:
    - name: Trade
      columns:
      - name: perTradeVwap
        type: float
  databases: []
  pipelines: []
  router: null
  views: []
  deployment_config: null
  data_files: {}
  version_info: ''
                    

• Apply the overlay to the package:

kxi package overlay fsi-app-bbg-entl fsi-app-bbg-entl/patches/MyBloombergOverlay.yaml 
                    

• TODO: Temporary workaround, remove incorrect files created by overlay logic

rm -rf fsi-app-bbg-entl/databases/fsi-core-db/tables/
                    

3. Add custom config to be loaded by the ingest pipeline fsi-app-bbg-entl-bbg-realtime-l1-1-spwork-0

• If a file named bbg.realtimel1.config.custom.q is present in fsi-app-bbg-entl/src/, the ingest pipeline fsi-app-bbg-entl-bbg-realtime-l1-1-spwork-0 loads its contents, allowing customization of real-time data ingest behavior.

• Create a file named bbg.realtimel1.config.custom.q to the directory fsi-app-bbg-entl/src/.

• Add the new bloomberg field and accelerator column to the file fsi-app-bbg-entl/src/bbg.realtimel1.config.custom.q as shown below:

// Add the new bloomberg field to the list `.bbg.cols.Trade` 
.bbg.cols.Trade,:`PER_TRADE_VWAP_REALTIME;

// Add the mapping of the bloomberg field name to the accelerator column name to the dictionary `.bbg.bbgToFsiColMap.Trade`
.bbg.bbgToFsiColMap.Trade,:(enlist `perTradeVwap)!(enlist `PER_TRADE_VWAP_REALTIME);

                    

4. Repack and deploy the updated package

• Uninstall the feed handler, if it is already running:

helm uninstall rt-bpipe-pub
                    

• Teardown the accelerator if it is already running:

kxi pm teardown fsi-app-bbg-entl
                    

• Repack the accelerator with your overlaid changes from the previous steps:

kxi package packit fsi-app-bbg-entl --tag 
                    

• Push the package:

kxi pm push --force ${KX_ARTIFACT_PATH}/fsi-app-bbg-entl-1.0.0.kxi
                    

• Deploy the package

kxi pm deploy fsi-app-bbg-entl 
                    

Install the feed handler using both the accelerator custom overrides after the package has been fully deployed:

helm install rt-bpipe-pub ./rt-bpipe-pub -f fsi-app-bbg-entl/config/feed/bbg-bpipe-feed-values.yaml -f my-custom-values.yaml