BenchmarksGame and RosettaCode

#1

I have seen that Rust is lucky to be on this site (unfortunately unlike D language):

http://benchmarksgame.alioth.debian.org/u64/rust.php

I am not yet good enough to improve/write BenchmarksGame Rust entries. That site ranks languages according to run-time and also to de-commented&compressed code length. That site allows multiple entries for each language, so I suggest to put there two or more entries for each Rust benchmark, one entry designed to be short or/and idiomatic, and a second version designed just for speed. This allows to better see the various balances Rust offers to the programmer.

Example: this version is as fast as the best C entry, but it’s quite ugly looking and quite long (and it uses more memory than the C entry):
http://benchmarksgame.alioth.debian.org/u64/program.php?test=pidigits&lang=rust&id=1

I am not suggesting to remove or rewrite that Rust entry, but I suggest to add a second shorter and much nicer entry, using the num bigints, and it should be as nice and short as this one (this Python3 entry also uses GMP, so it’s just 1.3 times slower than the Rust entry):

http://benchmarksgame.alioth.debian.org/u64/program.php?test=pidigits&lang=python3&id=5

I am still rather new to Rust, but while I learn Rust I’ll probably write, improve, fix and update some RosettaCode (http://rosettacode.org ) entries. For me it’s a way to learn many corners of a language, and the resulting examples (if they are well written) are useful for other Rust newbies.

#2

Hello, you have a very nice name! If you’re looking to contribute to the benchmarks game code, you should look here. For rosetta code, this is the repo.

#3

That’s not how the benchmarksgame works. The results are taken from the fastest entry. Speaking of which, teXitoi, Veedrac and yours truly have already done some work to speed up the benchmarks. They’re on the tracker waiting for approval.

1 Like
#4

That site has secs, KB RAM, gz, cpu, and cpu load columns. If you go in the page of a benchmark, like this one:

http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=knucleotide

You can click on the “gz”, and the site shows you:

http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=knucleotide&sort=gz

So you can sort your data according to various metrics. You can open the top item according to “gz”, for this benchmark it’s the second Perl entry:

http://benchmarksgame.alioth.debian.org/u64q/program.php?test=knucleotide&lang=perl&id=2

It’s not golfed because this site rightfully doesn’t like golfed entries, and because “gz” means the bytes after removing comments and after compression with gz. So lot of the redundancy (like repeated long tokens) is removed.

If you take look at the entries in various benchmarks you can see clearly that some entries are really really designed to be short&simple above all else, like:

http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexdna&lang=jruby&id=1

The Rust entry for PiDigits is really ugly, it contains (wrapped) raw calls to GMP, like __gmpz_mul_ui(&mut self.mpz, &a.mpz, b); This makes the Rust entry as fast as the fastest, but noisy, very long, and quite unreadable. It’s also quite not-idiomatic. Rust is designed for low-level programming, but it also has plenty of ammunition to allow a higher level of coding. Including nice to use multi-precision integers.

The point of the Benchmarks Game site is not to just show the performance of the various entries, it allows various entries for each language for each benchmark to allow programmers to show various trade-offs while you implement your solution. It’s wise to add two or more Rust entries (if there’s a reason to do it) to show that in Rust you can write compact readable
code, and low-level code. Before the Benchmarks Game site owner removed the D language, many benchmarks had two D entries, to show a short nice version, and a longer faster version.

#5

I think you’re way too quick to dismiss this and bring the “non-idiomatic” judgement to bear.

Much more than half code is the GMP bindings, and wrapping FFI calls like this is 110% idiomatic: wrapped C FFI calls—with the clear ownership and resource management semantics Rust brings—is really the best way to interact with C APIs, even better than doing it from C itself IMO (in a perfect world, GMP would be used via an external crate, but, understandably, it’s presumably easier for the site maintainer if cargo isn’t needed). On a more fundamental level, with Rust’s zero-overhead FFI, reusing existing optimised libraries is not unidiomatic. Also, the code uses iterators and context objects with methods (etc.) in a great way.

Also, I think the big-int API it is exposing is perfectly idiomatic: caring about allocations and memory and performance is a major part of being idiomatic in Rust, as a low-level language. Arbitrary-precision arithmetic performance is quite sensitive to doing allocations, especially in loops like this: being able to reuse storage allocated in previous iterations is vital. That said, the types could definitely have an operator overload or two, but there’s actually only one that would be useful for the code written there (i.e. wouldn’t force unnecessary allocations): Mpz * c_ulong, to replace mul_from_ui.

Of course, if we step away from just being idiomatic, and instead focus on making the code as readable as possible, we can go crazy with operator overloads, even if that forces (for instance) O(n) allocations where previously there were only O(1).

3 Likes
#6

Recently the Rust k-nucleotide benchmark was improved, so this is the current BenchmarksGame situation (the numbers are run-times in seconds):

n-body:
  Fortran Intel 8.11
  Rust 22.92

mandelbrot:
  C gcc 1.78
  Rust 4.96

binary-trees:
  C gcc 3.28
  Rust 7.54

spectral-norm:
  C gcc 1.98
  Rust 4.00

fannkuch-redux:
  C gcc 9.07
  Rust 13.34

reverse-complement:
  C gcc 0.42
  Rust 0.72

fasta:
  C gcc 1.36
  Rust 1.50

k-nucleotide:
  C gcc 6.46
  Rust 8.97

pidigits:
  Chapel 1.60
  Rust 1.74

regex-dna:
  PHP 2.23
  Rust 2.34
1 Like
#7

The situation is recently improved (I’ve omitted the entries that are fastest):

spectral-norm:
  C gcc #4 1.98
  Rust #2 3.97

n-body
  Fortran Intel #6 8.11
  Rust #2 13.22

binarytrees:
  C gcc #3 3.28
  Rust #2 4.76

fannkuch-redux:
  C gcc #5 9.07
  Rust #3 10.63	

mandelbrot:
  C gcc #6 1.64
  Rust #4 1.93

fasta:
  C gcc #7 1.36
  Rust #2 1.49

pidigits:
  Chapel 1.60
  Rust 1.74
3 Likes
#8

The situation is getting better:

binary-trees:
  C++ g++ #9 2.34
  Rust #2 4.27

regex-redux:
  C gcc #4 1.89
  Rust 3.28

n-body:
  Fortran Intel #6 8.11
  Rust #2 13.08

spectral-norm:
  C gcc #4 1.98
  Rust #3 2.36

fannkuch-redux:
  C gcc #5 9.07
  Rust #3 10.80

mandelbrot:
  C gcc #6 1.64
  Rust #4 1.90

fasta:
  C gcc #7 1.36
  Rust #2 1.49

pidigits:
  Chapel 1.60
  Rust 1.75

reverse-complement:
  C gcc #6 0.42
  Rust #2 0.45
2 Likes
#9

New results with rustc V.1.26.0:

regex-redux:
  Rust 3.01
  C 1.50

n-body:
  Rust 13.26
  Fortran 8.16

binary-trees:
  Rust 4.13
  C 2.54

k-nucleotide:
  Rust 5.28
  C++ 3.66

fannkuch-redux:
  Rust 9.60
  C 8.72

pidigits:
  Rust 1.74
  Chapel 1.62

mandelbrot:
  Rust 1.75
  C++ 1.51

spectral-norm:
  Rust 2.30
  C++ 1.98

fasta:
  Rust 1.47
  C 1.32

reverse-complement:
  Rust 1.62
  C gcc 1.75
2 Likes