Jupyter q Magic Command Notebook
This notebook demonstrates how to use the q Magic command in a Jupyter notebook.
The Jupyter q magic command in PyKX allows you to execute q code within a Jupyter notebook. It provides seamless integration with the q programming language.
This example Notebook has the following sections:
1. Import PyKX
To run this example, first import the PyKX library:
Python
import pykx as kx2. Create the external q process
You can run an external q process by using the following Python code:
Python
import subprocess
import time
try:
with kx.PyKXReimport():
proc = subprocess.Popen(
('q', '-p', '5000')
)
time.sleep(2)
except:
raise kx.QError('Unable to create q process on port 5000')Or execute this command in a terminal:
PowerShell
$ q -p 50003. Execute against Embedded q
To execute q code within PyKX's EmbeddedQ module, type %%q at the beginning of the cell:
After %%q you can further add two execution options:
|
EXECUTION OPTION |
Description |
|---|---|
|
|
Prints the q backtrace before raising a QError if the cell gives an error. |
|
|
Calls display rather than the default print on returned objects. |
4. SQL interface
The s) syntax runs SQL queries against local tables within the q process.
Note
To use the SQL interface, first you need to load the s.k_ library.
Python
%%q
\l s.k_
tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);
s) select * from tab where a>500 and b<250.0 limit 5Python
a b c
-----------------
916 149.4238 AAPL
898 97.74763 MSFT
900 45.35602 MSFT
677 150.6862 MSFT
741 151.1293 MSFT5. q namespaces
You can use q namespaces, and switch between them with \d.
Note
The namespace is reset back to the base namespace . between cells.
Python
%%q
\d .example
f: {[x] til x};6. q over IPC
After %%q you can include connection information, if you wish to connect to a remote q process over IPC.
The list of supported connection parameters is below. The rule is:
-
If they have a type, it must be followed by a second value/parameter.
-
If there's no type after them, you can use them as a standalone flag.
|
PARAMETER |
OBJECT TYPE AND Description |
|---|---|
|
|
(string) The host to connect to. |
|
|
(integer) The port to connect over. |
|
|
(string) The username to use when connecting. |
|
|
(string) The password to use when connecting. |
|
|
(float) The time in seconds before the query times out. Defaults to no timeout. |
|
|
Disable messages over 2GB being sent / received. |
|
|
Use a tls connection. |
|
|
Use a unix connection. |
|
|
(integer) How many reconnection attempts to make. |
|
|
Disable the context interface. |
Connect to a q server running on localhost at port 5000 as user using password password and disable the context interface.
Python
%%q --host localhost --port 5000 --user user --pass password --noctx
til 10Python
0 1 2 3 4 5 6 7 8 9All connection arguments are optional, except the --port argument. If --host is not provided localhost is the default host.
Python
%%q --port 5000
tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);Note that it's possible to execute q code spanning multiple lines:
Python
# Shutdown the q process we were connected to for the IPC demo
proc.kill()7. q first mode
q first mode can be enabled by importing PyKX after setting the environment variable PYKX_JUPYTERQ to true, or at runtime use:
Python
kx.util.jupyter_qfirst_enable()Python
PyKX now running in 'jupyter_qfirst' mode. All cells by default will be run as q code.
Include '%%py' at the beginning of each cell to run as python code. Once enabled, you can call q code without needing to include %%q at the beginning of a cell.
In this state, you can execute Python code as well, but those cells must include %%py.
Python
%%py
for fruit in ['apple', 'orange', 'banana']:
print(fruit)Python
apple
orange
bananaIf you wish to exit q first mode, simply run the following code and the notebook will revert back to default, Python first execution.
Python
%%py
kx.util.jupyter_qfirst_disable()Python
PyKX now running in 'python' mode (default). All cells by default will be run as python code.
Include '%%q' at the beginning of each cell to run as q code. To enable qfirst mode from q, run the following.
Python
%%q
.pykx.enableJupyter()Python
PyKX now running in 'jupyter_qfirst' mode. All cells by default will be run as q code.
Include '%%py' at the beginning of each cell to run as python code. And to return to Python first execution run the code below.
Python
.pykx.disableJupyter()Python
PyKX now running in 'python' mode (default). All cells by default will be run as python code.
Include '%%q' at the beginning of each cell to run as q code. 8. Saving code blocks
The --save feature allows user to save code in a cell as a q file.
To use this feature, include --save followed by the path of the file.
Note
If the q script errors the file will not be saved.
Using --save on an IPC connection cell will save the file on the remote host.
Python
%%q --save ../../new_file.q
vals:til 10
vals * 3Python
0 3 6 9 12 15 18 21 24 27
Cell contents saved to '../../new_file.q'.If the user wants to save a code block without executing them first, they can include --execute False at beginning of a cell.
Note
Nothing is outputted when the code below is ran.
Python
%%q --save ../../new_file.q --execute False
new_val:3 6 9
new_valPython
Cell contents saved to '../../new_file.q' without cell logic being run. To run the cell remove '--execute False'.File paths that end in .q_ will automatically be created as locked files without the need for any additional input.