New Jupyter client library


#1

Jupyter is a project aiming to “develop open-source software, open-standards, and services for interactive computing across dozens of programming languages.” [source]. It is typically known for the “notebook” and is often used for data science and exploratory data analysis.

The project decouples the notebook frontend from the “kernel” via an abstraction, which allows for many languages to implement these kernels.

I am starting to write a new frontend, which led me to develop a Rust-based kernel-abstraction (or “client”). I wrote a blog post about this here.

I’d love some feedback.


#2

I think you could also post this in the Jupyter discourse, they probably have good feedback too! https://discourse.jupyter.org/


#3

Great, thanks for the suggestion! I’ve posted to reddit and got a response from a Jupyter dev - I’m hoping to contribute to the documentation for the wire protocol.


#4

I kind of want to like this, because i like both rust and jupyter, but what i cannot imagine a situation where i would use this library. What’s the intended use case?


#5

I’ll admit I haven’t thought too far down that road. It could be used for alternative Jupyter frontends. I also can imagine Jupyter kernels taking the role of execution engines, perhaps run on demand.

For me it was a great learning experience to learn the Jupyter frontend -> Kernel communication protocol.


#6

Cool, I guess that’s real dedication then! :)


#7

Maybe running jupyter notebooks on a CI server would be a use case for a jupyter rust client? I hear netflix has lots of notebooks as tests.

I think there’s a lot of milage in Jupyter notebooks as automated “Test Stories”.


#8

Have you seen https://github.com/google/evcxr ? How does your project compare?

Edit:
Ah nevermind… I see I was confusing kernel with client.


#9

I think that’s a nice idea. I had the idea that using Jupyter kernels of many languages as some form of work-farm would be useful, and to build some form of job scheduler in Rust. The problem to this however is that Docker would be a potentially useful approach in that use case, rather than Jupyter. I feel I may be too late with the Jupyter-as-job-execution idea…


#10

Wouldn’t docker solve a completely different problem here (isolation of processes (docker) versus having nice-looking output from your tasks (jupyter))?


#11

I’m thinking the way processing pipeline steps can be baked into a docker container (e.g. Pachyderm)


#12

Pachyderm sure looks interesting… on a different note, this is making me think about my dockershell project. Maybe dockershell would work well as a jupyter kernel - a different take to the bash kernel.