Numerics & math foundation

[mdup]

There’s a great Twitter retransmission of the currently-happening #rustconf, and I wish I could be there right now!

This slide suggests there’s a need for a good fundation for Numerical Computing on Rust. I personally believe there’s so much power in Rust that it could become an awesome platform for the work currently done in Numpy.

We know the popularity of the most used scientific libs, such as Scikit-learn, R, etc. comes from two things:

1. a big community able to write the common and less-common scientific algorithms in <language>;
2. solid numerical frameworks on which those are built.

I think we could address point 2 here. We can get inspired from (non exhaustive iist):

• NumPy (scientific computing for python esp. great numerical arrays)
• TensorFlow (CPU/GPU tensor computations)
• Eigen (C++ matrices)

Where should we orient our efforts? I think a good milestone could be having an user-friendly, vector-enabled (SIMD) equivalent of NumPy multidimensional arrays, along with a couple of core primitives.

We could also address 1. in parallel, writing some ML code, matrix manipulation including advanced decompositions, etc. (I have a working SVM taking the dust in my home dir + a stats library I’m working on, but nothing public yet.)

Thoughts?

[willi_kappler]

I personally believe there’s so much power in Rust that it could become an awesome platform for the work currently done in Numpy.

That’s exactly what I’m thinking the whole time

Maybe we can also have a “arewenumericyet” or “arewescientificyet” like the other “arewe…” pages. Is s.o. already working on this ?

I’m currently working on smaller things like:

willi-kappler/natural_constants

willi-kappler/simple_units

(both at a very early stage and still in development)

I also made a comparison of parallelization crates for rust:

https://github.com/willi-kappler/mandel-rust (yeah I have to update this…)

I hope to have more time to work on these (and more projects) in the future.

1. a big community able to write the common and less-common scientific algorithms in <language>;
2. solid numerical frameworks on which those are built.

As you already said both could be addressed more or less in parallel (of course most of 1. would require 2.)

In addition to what you mentioned:

• read / write support for scientific data formats (HDF5, image, 2D, 3D, …)
• plotting (gnuplot, matplotlib)
• MPI / cluster support

I know that there are Rust wrapper for most of these but having them native would be even better (and a lot of work).

What we also need are tutorials and documentation.

I gave a short introduction to Matlab for students at beginning of this year (together with a colleague):

willi-kappler/Matlab_Kurs

I can try to translate it into English and see what can be done in Rust today and update it as soon as we have newer / better crates.

[pachi]

There’s the nice https://github.com/bluss/rust-ndarray project, to develop an N-dimensional array similar to what Numpy provides (it could potentially import/export data in a numpy ndarray compatible way using the C API).

The project is active and is already useful, and developing a Dataframe type on top of this could be a nice next step.

[Razican]

I also have a small crate that implements the VSOP87 algorithm in all of its versions in 100% pure Rust: https://crates.io/crates/vsop87

It calculates positions of planets in the solar system for a given time.

[sharez]

I believe OpenMP is also important, which is one of the common and efficient methods of shared memory programming.

Update:
I found this one which seems exciting: https://github.com/nikomatsakis/rayon

[willi_kappler]

Yes OpenMP support would be nice, LLVM is getting better with every new release. But I think that would be a lot of work to include OpenMP support into Rust.

I found this one which seems exciting: https://github.com/nikomatsakis/rayon

There’s also Jobsteal. I’ve made a comparison of some parallelization crates here (I need to update it…).

[willi_kappler]

Looks like the plotting stuff is being worked on:

https://www.reddit.com/r/rust/comments/54fq1p/dataplotlib_an_earlystage_hasslefree_plotting/

[leonardo]

There’s also Jobsteal. I’ve made a comparison of some parallelization crates here (I need to update it…).

I tried that Mandelbrot benchmark some time ago and it run fine. Now I have installed it again and I see a problem (latest working Nightly):

   Compiling crossbeam v0.1.6
error[E0512]: transmute called with differently sized types: [usize; 32] (2048 bits) to [std::sync::atomic::AtomicBool; 32] (256 bits)
  --> ...\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-0.1.6\src\sync\seg_queue.rs:34:29
   |
34 |             ready: unsafe { mem::transmute([0usize; SEG_SIZE]) },
   |                             ^^^^^^^^^^^^^^

error: aborting due to previous error

Removing some of the uncompilable code, it gives (the CPU has 4+4 cores):

...>mandel.exe --num_threads 8
Configuration: re1: -2.00, re2: 1.00, img1: -1.50, img2: 1.50, max_iter: 2048, img_size: 1024, num_threads: 8
Time taken for this run (serial): 1441.46419 ms
Time taken for this run (rayon_join): 185.18138 ms
Time taken for this run (rayon_par_iter): 201.04410 ms
Time taken for this run (job_steal): 264.69656 ms
Time taken for this run (job_steal_join): 185.36490 ms

Here job_steal_join doesn’t seem faster than rayon_join.

[willi_kappler]

Hi leonardo,

thanks for your feedback and sorry about the compile error: I need to update to a newer version of crossbeam. I’ll try to fix it in the next couple of days.

Rayon and JobSteal are quite close to each other but in most cases (at least on my machines) JobSteal was a bit faster.

[willi_kappler]

I’ve released a new version of mandel-rust: v0.4. I had to remove the code that didn’t compile.

[leonardo]

During the compilation it gives a warning:

   ...
   Compiling time v0.1.35
warning: constant `compiler_version` should have an upper case name such as `COMPILER_VERSION`, #[warn(non_upper_case_globals)] on by default
 --> ...\mandel_util-04bb748c3137f731\out/compiler_version.rs:1:1
  |
1 | pub const compiler_version: &'static str = "1.14.0-nightly";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

   Compiling mandel_method v0.4.0 (file:.../mandel-rust-master/mandel_method)

Then at run-time it gives an error:

...>mandel.exe
Configuration: re1: -2.00, re2: 1.00, img1: -1.50, img2: 1.50, max_iter: 4096, img_size: 2048, num_threads: 2
mandel-rust version: 0.4.0
Number of repetitive runs: 2
Rustc version: 1.14.0-nightly
Time taken for this run (serial): 11406.37904 ms
thread 'main' panicked at 'I/O error while writing benchmark results: Error { repr: Os { code: 3, message: "Can't find specified path." } }',
../src/libcore\result.rs:799
stack backtrace:
   0:           0x47a95e - <unknown>
   1:           0x478a21 - <unknown>
   2:           0x47c87c - <unknown>
   3:           0x47c7ea - <unknown>
   4:           0x48ad8b - <unknown>
   5:           0x42b273 - <unknown>
   6:           0x42aaf5 - <unknown>
   7:           0x401ab5 - <unknown>
   8:           0x402104 - <unknown>
   9:           0x4013b4 - <unknown>
  10:           0x4014e7 - <unknown>
  11:     0x7ff9d95b13d1 - <unknown>
Command exited with non-zero status 101

[willi_kappler]

Thanks for reporting!

Looks like it can’t write the results to the plot/ folder. I’ve added a system dependent path separator. The warning is also removed.

Can you please try it again ?

[leonardo]

The warning is gone but the run time error is still present:

Time taken for this run (serial): 11459.00911 ms
thread 'main' panicked at 'I/O error while writing benchmark results: Error { repr: Os { code: 3, message: "Impossible to find the specified path." } }',
../src/libcore\result.rs:799
stack backtrace:

[willi_kappler]

Now I know were the problem was: it expects a folder called “plot”.
The updated version checks if that folder is available and if not creates it.
Sorry for the trouble!

[leonardo]

Now it works. Results on an older CPU with two cores:

...>mandel --num_threads=1
Configuration: re1: -2.00, re2: 1.00, img1: -1.50, img2: 1.50, max_iter: 4096, img_size: 2048, num_threads: 1
mandel-rust version: 0.4.0
Number of repetitive runs: 2
Rustc version: 1.9.0-nightly
Time taken for this run (serial): 36374.66885 ms
Time taken for this run (scoped_thread_pool): 40570.48019 ms
Time taken for this run (rayon_join): 36260.96306 ms
Time taken for this run (rayon_par_iter): 36209.13873 ms
Time taken for this run (rust_scoped_pool): 39575.40944 ms
Time taken for this run (job_steal): 41676.19083 ms
Time taken for this run (job_steal_join): 47219.79330 ms

...>mandel
Configuration: re1: -2.00, re2: 1.00, img1: -1.50, img2: 1.50, max_iter: 4096, img_size: 2048, num_threads: 2
mandel-rust version: 0.4.0
Number of repetitive runs: 2
Rustc version: 1.9.0-nightly
Time taken for this run (serial): 43767.00963 ms
Folder 'plot' does not exist, creating it...
Time taken for this run (scoped_thread_pool): 23223.63988 ms
Time taken for this run (rayon_join): 23976.00798 ms
Time taken for this run (rayon_par_iter): 23189.08940 ms
Time taken for this run (rust_scoped_pool): 22900.26987 ms
Time taken for this run (job_steal): 32971.58480 ms
Time taken for this run (job_steal_join): 33164.70669 ms

On a newer bigger CPU:

...>mandel.exe --num_threads=8
Configuration: re1: -2.00, re2: 1.00, img1: -1.50, img2: 1.50, max_iter: 4096, img_size: 2048, num_threads: 8
mandel-rust version: 0.4.0
Number of repetitive runs: 2
Rustc version: 1.14.0-nightly
Time taken for this run (serial): 11419.05091 ms
Folder 'plot' does not exist, creating it...
Time taken for this run (scoped_thread_pool): 1467.54240 ms
Time taken for this run (rayon_join): 1444.94264 ms
Time taken for this run (rayon_par_iter): 1468.72888 ms
Time taken for this run (rust_scoped_pool): 1446.48753 ms
Time taken for this run (job_steal): 1447.19408 ms
Time taken for this run (job_steal_join): 1460.25664 ms

[fommil]

I wrote https://github.com/fommil/netlib-java which brings highly optimised linear algebra BLAS/LAPACK to the JVM and is used by big frameworks like Apache Spark. I’d absolutely love to do the same for Rust but I honestly don’t have the bandwidth to do this in my spare time

[willi_kappler]

Wow and welcome to Rust!

That would be definitely useful and a lot of work, too.

A quick search revealed these posts / projects:

https://athemathmo.github.io/2016/03/23/linear-algebra-in-rust.html

Bindings vs from scratch for BLAS? • r/rust

blas-lapack-rs/blas

blas-lapack-rs/lapack

mikkyang/rust-lapack

mikkyang/rust-blas

https://github.com/wellposed/rust-OpenBLAS (seems unmaintained ?)

Unfortunately I didn’t have the time to use them (yet), so I can’t comment on how good these crates work.

[fommil]

Oh wow, I’d seems it’s already been done. A great shame that linalg is
reinventing the wheel instead of building on the existing BLAS/LAPACK
optimised routines. It seems there is Sur a need for a high level maths
library like Scala’s Breeze.

[anowell]

Maybe we can also have a “arewenumericyet” or “arewescientificyet” like the other “arewe…” pages. Is s.o. already working on this ?

I love all the arewe… pages. Given that a solid foundation with numerics and scientific computing correlates with strong ML ecosystems, there are already sections of arewelearningyet carved out for for scientific computing, and some numeric bits also live in the data structures category. A solid numerics & math foundation might not be a true subset of the ML ecosystem, but I’m still open to ideas on how to better organize and feature these foundational pieces on arewelearningyet. Of course, I’d also be happy to link to an arewenumericyet or arewescientificyet should such sites come to exist.

[p_aryo]

I think one important piece which we are currently missing is the type level integers. Yes we have typenum and crates built on it, but in my opinion it’s an ersatz solution which is not suited for foundation role. At first glance this RFC looks quite promising, it’s also provides additional capabilities for compile time checks which certainly will be useful for numerical and math crates.