Announcing elfmalloc, a pure Rust allocator that's competitive on performance with jemalloc

announcements
1

@ezrosent and I are happy to announce elfmalloc, a new allocator written in pure Rust!

Quick links:

This project actually involved a number of crates - 9 in total. They are all in this repository, and many are interesting on their own:

  • bagpipe, a fast concurrent bag datastructure
  • bsalloc, a simple "bootstrapping" allocator useful in implementing other allocators
  • mmap-alloc, an allocator backed by direct mapping of memory pages
  • malloc-bind, a utility to implement the C malloc API in terms of a Rust allocator
  • slab-alloc, an implementation of Jeff Bonwick's slab allocator algorithm
  • object-alloc, a trait for object allocators in Rust
  • object-alloc-test, a test suite for object allocators including memory corruption detection

This project is still in its early phases, there is a lot of work to do, and we would love contributions! See CONTRIBUTING.md for details.

31 Likes
2

Guides for reducing binary sizes with Rust recommend disabling jemalloc to create an apples-to-apples comparison with C. Do you have any predictions or goals for the resulting binary sizes created using elfmalloc?

1 Like
3

We haven't thought about that yet, so I don't have a good answer. I ran cargo build --release in the elfmalloc crate, and the resulting libelfmalloc.rlib is 836K. I'm not sure how that compares to jemalloc, though.

4

FYI the size of an .rlib is pretty far removed from binary size because it includes metadata and doesn't contain machine code for generic and #[inline] functions.

1 Like
5

Fair point. I suppose a statically-compiled binary would work, but we'd need to figure out first how to use elfmalloc as the global Rust allocator, which we haven't yet done.

6

This sounds like you only benchmarked it on WSL? No pure Linux or Windows benchmarks? Especially the latter would be interesting, as AFAIK Rust doesn't use jemalloc on Windows.

7

The highest-core machine we have access to only has a Windows partition. While I should get around to getting a native Linux partition, it hasn't happened yet and I'm not sure when I'll have time to sit down and get everything working.

As for Windows, we are working on getting support for it! The biggest hurdle we will have to address is the way Windows handles uncommitted memory. But we are pretty sure how to support this configuration, so stay tuned!

8

Would it be possible to dual-license under the MIT license as well? That would allow it to be used by the Rust compiler distribution, as well as by the Redox operating system.

1 Like
9

If I'm not mistaken, this shouldn't be a problem - the Apache 2.0 license allows arbitrary use and redistribution so long as there's attribution, so Rust and Redox should both be able to use it. The same is true, for example, with LLVM, which uses neither Apache 2.0 nor MIT, but is still used by Rust.

10

Apache v2 is not compatible with GPL v2, MIT is. This is why most of the Rust ecosystem is dual-licensed Apache2/MIT.

1 Like
11

Ah, I see. I'll need to discuss this with @ezrosent, but we'll consider it.

1 Like
12

OK, we've decided to go ahead with it. Since our contributors hold individual copyright, we'll need their permission first, but assuming we get that, we'll dual-license.

4 Likes
13

Done!

2 Likes
14

It's always nice to see how things can quickly be resolved in the OpenSource world. Not meetings. No self-flagellation. Just discussion, decision, execution!

1 Like