Equity Analytics Framework - Util Functions

This page provides an overview of the Equity Analytics Framework and its utility functions.

The Equity Analytics Framework calculates a wide number of Order Analytics. This framework is built with flexibility in mind, and can easily be extended with custom analytics.

A number of utility functions have been written to enable development of custom analytics.

.eqea.util.simpleAnalytics

The Util function enables users to run "simple" analytic, that is, analytics that make use of columns already present in the OrderAnalytics table.

The function .eqea.util.simpleAnalytics runs on a subset of the .eqea.analytics.cfg.

Analytics that make use of .eqea.util.simpleAnalytics should have at a minimum the following values populated in .eqea.analytics.cfg:

• analytic

• analyticType

• funcName

• aggClause

Parameters:

Returns:

Example usage:

Writing example custom function .example.orderExecution.simpleCustomAgg:

• Populating required values in .eqea.analytics.cfg:

.eqea.analytics.cfg:flip `analytic`analyticType`funcName`aggClause`marketDataTabName`joinTimeOffset! flip (

    // Examples of adding simple custom analytic types (simple aggregation on already calculated values in our OrderAnalytics table)
    (`strikeToCompletionBidMidPrice ;`simpleCustomAgg   ;`.example.orderExecution.simpleCustomAgg   ; (%;(+;`arrivalAskPrice;`endAskPrice);2) ; `       ; 0Nt );
    (`strikeToCompletionAskMidPrice ;`simpleCustomAgg   ;`.example.orderExecution.simpleCustomAgg   ; (%;(+;`arrivalAskPrice;`endAskPrice);2) ; `       ; 0Nt )

    );    

• Contents of example function: .example.orderExecution.simpleCustomAgg

.example.orderExecution.simpleCustomAgg:{[OrderAnalyticsRes]

    // We can use a util function `.eqea.util.runSimpleAnalytic` to run analytics that are dependent on columns that already exist in the OrderAnalytics table
    cfg:select from .eqea.analytics.cfg where analyticType=`simpleCustomAgg;
   .eqea.util.simpleAnalytics[OrderAnalyticsRes;cfg]

    };

.eqea.util.simpleAnalyticsWithWc

Util function to enable users to run "simple" analytics when a where clause is in the cfg i.e. analytics that make use of columns already present in OrderAnalytics table.

The function .eqea.util.simpleAnalyticsWithWc runs on a subset of the .eqea.analytics.prevailing.cfg table.

Analytics that make use of .eqea.util.simpleAnalyticsWithWc should have at a minimum the following values populated in the table passed in to the cfg argument:

• whereClause

• aggClause

Parameters:

Returns:

.eqea.util.tickData.getDataAndAggFromCfg

The Util function is used to query tick data and aggregate based on a subset of the .eqea.analytics.cfg table. Analytics that make use of .eqea.util.tickData.getDataAndAggFromCfg should have the minimum following values populated in .eqea.analytics.cfg:

• analytic

• analyticType

• funcName

• aggClause

• marketDataTabName

Parameters:

Returns:

Example usage:

Writing example custom function .example.orderExecution.tickDataAgg:

• Populating required values in .eqea.analytics.cfg:

.eqea.analytics.cfg:flip `analytic`analyticType`funcName`aggClause`marketDataTabName`joinTimeOffset! flip (

    // Examples of adding more complex custom analytics (aggregations on tick data with multiple where clauses)
    (`countPriceUnderLimitPrice              ;`tickDataAgg  ;`.example.orderExecution.tickDataAgg  ; (count;`i)                              ; `Trade   ; 0Nt );
    (`sumVolumeUnderLimitPrice               ;`tickDataAgg  ;`.example.orderExecution.tickDataAgg  ; (sum;`volume)                           ; `Trade   ; 0Nt )

    );    

• Contents of example function: .example.orderExecution.tickDataAgg

.example.orderExecution.tickDataAgg:{[OrderAnalyticsRes]

    // We can use `.eqea.util.runTickDataAnalytics` to run analytics on tick data tables
    cfg:select from .eqea.analytics.cfg where analyticType=`tickDataAgg;
    // Our where clause changes depending on whether the order is a buy or a sell.
    // By generating wcList from our input OrderAnalytics table we also ensure the limitPrice value relates to the Order we are analyzing.
    wcList:exec whereClause from select whereClause:(((((`BUY`SELL)!(<=;>=)) orderSideCode),\:(`price)),'limitPrice) from OrderAnalyticsRes;
    OrderAnalyticsRes:.eqea.util.tickData.getDataAndAggFromCfg[OrderAnalyticsRes;cfg;wcList;`strikeTime;`orderCompletedTime;0b];
    OrderAnalyticsRes

    };

.eqea.util.tickData.getDataAndAggregate

The Util function can be used to query tick data and aggregate based on input arguments.

This is a low-level function, essentially a wrapper for a functional select, to be used for complex use cases that are incompatible with .eqea.util.tickData.getDataAndAggFromCfg.

Parameters:

Returns:

.eqea.util.asof.ajFromCfg

This function calculates configured analytics that are based on as-of join logic.

This function:

1. Builds a time rack for each order based on whats configured in the joinTimeOffset column of the .eqea.analytics.cfg table.

2. Performs an as-of join, joining the time rack to the table specified by the marketDataTabName column of the .eqea.analytics.cfg table.

3. Pivots the result of the as-of join such that the data is pivoted to a single row per order, under column names as defined by the analytic column of the .eqea.analytics.cfg table.

4. Returns the result to be joined onto our input OrderAnalytics data.

Analytics that make use of .eqea.util.asof.ajFromCfg should have the minimum following values populated in .eqea.analytics.cfg:

• analytic

• analyticType

• funcName

• aggClause

• marketDataTabName

• joinTimeOffset

Parameters:

Returns:

Example usage:

Writing example custom function .example.orderExecution.ajExample:

• Populating required values in .eqea.analytics.cfg:

.eqea.analytics.cfg:flip `analytic`analyticType`funcName`aggClause`marketDataTabName`joinTimeOffset! flip (

    // Examples which use as of joins
    (`myArrivalTradePrice           ;`ajExample   ;`.example.orderExecution.ajExample                ; `price                                ; `Trade     ; 00:00:00 );
    (`myArrivalTradePrice_5         ;`ajExample   ;`.example.orderExecution.ajExample                ; `price                                ; `Trade     ; 00:00:05 );
    (`myArrivalTradePrice_10        ;`ajExample   ;`.example.orderExecution.ajExample                ; `price                                ; `Trade     ; 00:00:10 )

    );    

• Contents of example function: .example.orderExecution.ajExample

.example.orderExecution.ajExample:{[OrderAnalyticsRes]

    // We can use `.eqea.util.runTickDataAnalytics` to run analytics which retrieve values from tick data using an as of joins
    cfg:select from .eqea.analytics.cfg where analyticType=`ajExample;
    OrderAnalyticsRes:.eqea.util.asof.ajFromCfg[OrderAnalyticsRes;cfg;`strikeTime;()];
    OrderAnalyticsRes

    };

.eqea.util.asof.ajTickDataToInput

Efficient as-of join, joining an input data set to tick-data.

It is intended to be used for complex analytics, which are incompatible with .eqea.util.asof.ajFromCfg.

Parameters:

Returns:

.eqea.util.pivotByCfg

This function pivots a dataset, using the contents of .eqea.analytics.cfg to define output column names.

It is intended to be used to pivot data produced by .eqea.util.asof.ajTickDataToInput.

Parameters:

Returns:

.eqea.util.timeRackFromCfg

This function creates a time rack table from inputData based on the joinTimeOffset column from .eqea.analytics.cfg.

The time rack table is generated using joinOrderTimeCol + exec distinct joinTimeOffset from cfg where marketDataTabName=tickTabName.

It is intended to create a table of timestamps and instrumentIDs as required as an input in .eqea.util.asof.ajTickDataToInput.

Parameters:

Returns: