Crate of the Week


#221

Tantivy is a full text search engine written in Rust. The author has a learned from other search engines, notably Lucene. The code is a joy to work with and very fast.

For those interested in the internals, there’s a series on it.

So I’d like to nominate Tantivy.


#222

In the good tradition of tooting one’s own horn, I nominate optional, a crate containing a few space saving (and also fast) Option-like types for bools, integers and floats.


#223

I would like to nominate librespot. I searched for a spotify connect server for the office last week. Was so happy to find one in rust :slight_smile:


#224

I would like to nominate djangohashers, yes, it is a bit of self-promotion, but I’m really proud of this little library.

This library is a Rust port of the password primitives used in Django Project. This is useful in 2 scenarios:

  1. You have an existing Django app and want to migrate or integrate with Rust, but you have no way to reuse the old authentication scheme because of the password hashes.
  2. You just want use a battle-tested password hashing scheme for a new pure-Rust project, even if no Django integration is needed.

I translated all relevant unit tests from Django 1.4 to 1.11, and just added Argon2 support in the last version. There is also an option to use a PBKDF2 implementation 28% faster than CPython.


#225

derive_builder: automatically implement the builder pattern for arbitrary structs. Now with macro 1.1 support (custom derive since Rust 1.15).


#226

I would like to nominate pipeliner.

It allows you to use .with_threads(num) anywhere in your iterator chain. And it just works.

The documentation is clear and straight to the point.


#227

I would like to nominate twiddle. If one needs to do some low level hardware programming it makes one’s life suck less :slight_smile:


#228

This is so kool it seems too good to be true


#229

I’d like to second the nomination of config. :wink:


#230

I wonder if I could try to nominate my own crate CDRS for next week. It’s a client for Apache Cassandra written completely in Rust. Currently it is compatible with 4-th version protocol specification and implements the features described there. Hope this crate will help Rust to feel more comfortable in the world of Big Data.


#232

I second this nomination.

Cassandra is widely used in the OLTP big data world, and having fast native access to it will encourage a bunch of people from that world to dip their toe in rust.


#233

nalgebra. Linear algebra library for Rust.

It was also nominated back in September, but never received the honour.


#234

How is pipeliner different than rayon?


#235

I only tried rayon for a short time before giving up on it. So what I’m about to write might not be accurate.

The use-case was parallelizing network requests(client side), to avoid accumulating latencies. Those requests are not CPU-bound. So, requesting multiple parallel threads on a dual core device makes sense!

  • Rayon

  • Does not work with iterators directly.

  • It’s not clear, at least from a cursory look at the docs, how to configure the number of threads used with par_iter().

  • Errors can contain hard to decipher unsatisfied trait bounds.

  • [minor] Wildcard/glob use required.

  • Pipeliner

  • Works with iterators directly.

  • You can pass .with_threads(num) anywhere in your iterator chain. You can even pass it multiple times with different numbers.

  • Errors are easy to decipher (your typical borrow checker errors. Unsatisfied 'static requirements).

  • [minor] No wildcard/glob use required.

In short, pipeliner may not have a fancy work-stealing algorithm implemented behind the scenes. But it provides more flexibility and easier integration with iterators.


#236

Cargo-fuzz. Command line wrapper for using libFuzzer.


#237

PumpkinDB. PumpkinDB is a compact event sourcing database, featuring fast on-disk storage, flexible approach to event structure and encoding, sophisticated event indexing and querying.


[TWiR] Call for Participation
#238

μtest. Unit testing for microcontrollers and other no_std system.


#239

I’d like to nominate @BurntSushi’s FST crate .
This provides a very compact in-memory representation of maps, in the same ballpark as gzip, but queryable!
The compact representation makes analyses possible that would otherwise explode the RAM of even the biggest compute nodes.

In Andrew’s typical style, the crate, the theory behind it, and the design trade-offs are extremely well documented via blogpost. After reading it, I felt like I had learned something new about theoretical computer science, and had a good feeling for when to use the crate.

My department is currently using the crate to analyse genome regulation, by identifying the counts of all possible 16-mers in the human genome, so 4^16 = 4.294.967.296 keys. Our algorithm was impossibly slow before we encountered FST, producing hundreds of gigabytes of temp-files that had to be processed line-by-line.
Thanks to FST we can do unbiased analysis into what makes humans tick: how our genomes are regulated.

I’ll add that we’re using fst via it’s python bindings, by Johannes Baiter. Johannes was supportive and quickly responded to our questions. (He works at the digitisation centre of Bavarian State Library, so I’m very curious what use-case he found for fst there.)
I include this tidbit to demonstrate that there is already a (minute) user-community around fst, and that it is (apparently) broadly applicable.

All in all: fst is a crate that, through its great implementation, excellent documentation and great API, makes high-performance data structures available to a new audience, enabling real research and bringing humanity forward.

:heart: Andrew Gallant / @BurntSushi

(p.s. also a little :heart: for @ehiggs, who introduced me to it)


#240

duct - “A cross-platform library for running child processes and building pipelines. duct wants to make shelling out in Rust as easy and flexible as it is in Bash. It takes care of gotchas and inconsistencies in the way different platforms shell out.”


#241

Serenity - “Serenity is a Rust library for the Discord API. It enables you to create awesome bots for the Discord chat service in an easy and organized way.”