Deploying the ICE Consolidated Feed Handler Standalone

This page details how to install and run the KX ICE Consolidated Feed Handler standalone, publishing to a singular vanilla kdb+/q process.

Architecture

Prerequisites

Ensure the following prerequisites are met before running the standalone version of the KX ICE Consolidated Feed Handler:

  • Docker is installed.

  • Docker-Compose is installed.

  • You have login credentials for the ICE Consolidated Feed.

  • You have login credentials for docker registry portal.dl.kx.com.

  • You have a kdb+/q license file (kc.lic or k4.lic).

  • You have a local installation of pykx or kdb+/q, depending on your preferred method of accessing the data published by the ICE feed.

Login to Docker registry

To access the images needed for this walkthrough, you must login to the KX Docker registry:

Shell

Copy 
docker login portal.dl.kx.com -u <insert registry username> -p <insert registry password>

Download the ICE Standalone example

Download the ICE standalone example:

Shell

Copy 
version=1.0.1;\
curl -L  https://portal.dl.kx.com/assets/raw/kxi-ice-fh/${version}/ice-standalone-q-example-${version}.tar.gz -o ice-standalone-q-example.tgz

Extract the contents and navigate to the resultant directory:

Shell

Copy 
tar -xvzf ice-standalone-q-example.tgz;\
cd ice-standalone-q-example

Configure the ICE Standalone example

Within the ice-standalone-q-example directory, find the docker-compose-q-proc.yaml file and the env file. Update the env file with the following:

  • Set q_install_dir to the path of your kdb+/q license directory.

  • Set KXFEED_ICE_USERNAME and KXFEED_ICE_PASSWORD with your ICE username and password.

Information

To enable direct publishing to a kdb+/q process, ensure "isDirectConnect":1 is set in the dbConfigFile. This setting should be present in this example configuration.

File permissions

To avoid permission issues with containers mounting directories, run the following command in the ice-standalone-q-example directory (you may need sudo access):

Shell

Copy 
chmod -R 777 */

or with sudo:

Shell

Copy 
sudo chmod -R 777 */

Deploy the ICE Standalone example

Once the configuration has been updated, the ICE standalone example can be deployed:

Shell

Copy 
docker-compose --env-file env -f docker-compose-q-proc.yaml up

Query the data with kdb+/q

To access data published by the ICE feed, start a kdb+/q session locally and connect to the q process that captures the ICE data:

Shell

Copy 
q)h: hopen 2345

You can then query the data by running: For the quote table:

Shell

Copy 
q)h"select from quote"

For the trade table:

Shell

Copy 
q)h"select from trade"

Query the data with pykx

To query ICE data using pykx, launch the Python interpreter and import pykx:

Shell

Copy 
import pykx as kx

Open a connection to the q process that captures the ICE data:

Shell

Copy 
q = kx.SyncQConnection('localhost', 2345)

Then, query the data by running: For the quote table:

Shell

Copy 
q('select from quote')

For the trade table:

Shell

Copy 
q('select from trade')

View the logs

To validate that everything started successfully, check the logs using the following command in the ice-standalone-q-example directory:

Shell

Copy 
docker-compose --env-file env -f docker-compose-q-proc.yaml logs | less

Note

Logs are also visible during the docker-compose up command, but they may scroll quickly. Using docker-compose logs makes it easier to read and search through the logs.

Tear down the ICE Standalone example

To teardown and delete the containers of the ICE standalone example, run this command within the ice-standalone-q-example directory:

Shell

Copy 
docker-compose --env-file env -f docker-compose-q-proc.yaml down