Stats

.qsp.stats.describe[fields; stats]

Parameters:

Statistic Options

Note: some statistics do not support categorical data and will return generic null for said data

For all common arguments, refer to configuring operators

This example computes the min, max, and average on a batch of data

.qsp.run
 .qsp.read.fromCallback[`publish]
 .qsp.stats.describe[`y; `minimum`maximum`average]
 .qsp.write.toVariable[`output];
publish ([] x: til 5; y: 10 13 1 9 8)
output
Expected output: ([] minimum_y: enlist 1; maximum_y: enlist 13; average_y: enlist 8.2)

This example demonstrates how to use the percentiles option The operator below will compute the mode and skew along with the 90^th, 95^th and 99^th percentile.

.qsp.run
 .qsp.read.fromCallback[`publish]
 .qsp.stats.describe[`x; (`mode; `skew; (`percentiles; 0.9 0.95 0.99))]
 .qsp.write.toVariable[`output];
publish ([] x: til 100)
output

sp.stats.describe('price', 'average')

Parameters:

Returns: A pipeline comprised of a describe operator, which can be joined to other pipelines.

A list of all supported statistic options can be found below:

*calculated using the Fisher-Pearson coefficient of skewness

>>> from kxi import sp
>>> import pykx as kx
>>> import pandas as pd
>>> sp.run(sp.read.from_callback('publish')
| sp.stats.describe('x', 'average')
| sp.write.to_variable('out'))
>>> data = pd.DataFrame({
'x':[5,1,4,2,3],
'y':[100,100,200,50,50]
})
>>> kx.q('publish', data)

average_x
---------
3

Using percentiles along with other stats

>>> from kxi import sp
>>> import pykx as kx
>>> sp.run(sp.read.from_expr('([] x: 1 2 2 3 3 3 4 4 4 4)')
| sp.stats.describe('x', ['mode', 'skew', ('percentiles', [0.9, 0.95, 0.99])])
| sp.write.to_variable('out'))
>>> kx.q('out')

mode_x skew_x    percentile_0.9_x percentile_0.95_x percentile_0.99_x
---------------------------------------------------------------------
4      -0.512289 4                4                 4

.qsp.stats.ema[X; alpha; y]

Parameters:

For all common arguments, refer to configuring operators

This example replaces the columns x and y with their exponential moving averages.

.qsp.run
 .qsp.read.fromCallback[`publish]
 .qsp.stats.ema[`x`y; .33]
 .qsp.write.toConsole[];
publish ([] x: til 10; y: 0 1 4 2 5 3 6 7 9 8)

sp.stats.ema('volume', 0.33, 'res')

Parameters:

Returns: A pipeline comprised of a ema operator, which can be joined to other pipelines.

>>> from kxi import sp
>>> import pandas as pd
>>> import pykx as kx
>>> sp.run(sp.read.from_callback('publish')
| sp.stats.ema('x', 0.33, 'res')
| sp.write.to_variable('out'))
>>> data = pd.DataFrame({
'x': [1, 50, 3, 4, 5, 6]
})
>>> kx.q('publish', data)

x  res
-----------
1  1
50 17.17
3  12.4939
4  9.690913
5  8.142912
6  7.435751

.qsp.stats.sma[X; n; y]

Parameters:

For all common arguments, refer to configuring operators

This calculates, for each data point, the arithmetic mean of a moving window including that point and the n-1 prior data points.

This example replaces each value in y with the simple moving average of that value and the nine prior values.

.qsp.run
 .qsp.read.fromCallback[`publish]
 .qsp.stats.sma[`y; 10]
 .qsp.write.toConsole[];
publish ([] x: til 10; y: 0 1 4 2 5 3 6 7 9 8)

sp.stats.sma('price', 60, 'movingAvgPrice')

Parameters:

Returns: A pipeline comprised of a sma operator, which can be joined to other pipelines.

>>> from kxi import sp
>>> import pandas as pd
>>> import pykx as kx
>>> sp.run(sp.read.from_callback('publish')
| sp.stats.sma('x', 3, 'res')
| sp.write.to_variable('out'))
>>> data = pd.DataFrame({
'x': [1, 50, 3, 4, 5, 6]
})
>>> kx.q('publish', data)

x  res
-------
1  1
50 25.5
3  18
4  19
5  4
6  5

.qsp.stats.twa[X; times; range; y]

Parameters:

For all common arguments, refer to configuring operators

This calculates, for each data point, the arithmetic mean of a moving window including that point and the n-1 prior data points weighted by the time deltas found in times.

This example replaces each value in y with the time weighted average of that value and the nine prior values using weights derived from the time column.

.qsp.run
 .qsp.read.fromCallback[`publish]
 // The windowing is to ensure that records are sorted by timestamp
 .qsp.window.tumbling[00:01:00; `time; .qsp.use `sort`lateness!(1b; 00:00:10)]
 .qsp.stats.twa[`data; `time; 10]
 .qsp.write.toConsole[]
publish ([] time: 0p + 00:00:01 * 0 5 6 17 14 21 57 58 71;
 data: 10 20 10 9 11 8 21 10 9)

This example replaces each value in c and in d with the time weighted average of the values within a and b respectively and four prior values using the times column as a series of times.

.qsp.run
 .qsp.read.fromCallback[`publish]
 .qsp.window.tumbling[00:00:01; `time; .qsp.use `sort`lateness!(1b; 00:00:01)]
 .qsp.stats.twa[`a`b; `time; 5; `c`d]
 .qsp.write.toConsole[];
publish ([] time: 0p + 00:00:00.1 * 0 8 13 17 19 21; a: 1 7 8 7 7 8; b: til 6);

sp.stats.twa('price', 'time', 60, '1minMovingAvgPrice')

This calculates, for each data point, the arithmetic mean of a moving window including that point and the n-1 prior data points weighted by the time deltas found in times.

Returns: A pipeline comprised of a twa operator, which can be joined to other pipelines.

Examples:

>>> from kxi import sp
>>> from datetime import timedelta
>>> import pandas as pd
>>> import pykx as kx
>>> sp.run(sp.read.from_callback('publish')
| sp.stats.twa('x', 'time', 3, 'res')
| sp.write.to_variable('out'))
>>> data = pd.DataFrame({
'x': range(1,6),
'time': [timedelta(seconds=x) for x in [0, 5, 6, 14, 17]]
})
>>> kx.q('publish', data)

x  time                 res
--------------------------------
1  0D00:00:00.000000000 1
2  0D00:00:05.000000000 2
3  0D00:00:06.000000000 2.166667
4  0D00:00:14.000000000 3.214286
5  0D00:00:17.000000000 4.166667