Hey everyone,
I'm quite new to Rust (2 chapters in the Book) so I apologize in advance for the potential Captain Obvious in the room, and also for asking in the wrong place if this is not the right one.
I'm thinking about writing a malloc() replacement--and upgrade--in Rust as part of my PhD in CS. Do you have any related existing projects in mind?* Thanks in advance!
Cheers,
X
* Edit: it is also possible that writing a Rusty malloc() is by definition a bad idea. But given Rust's systems nature and so much loved performance/safety features, I'd bet against that possibility. Right?
Hey, this is a great time to write allocators in Rust! One of the historical stumbling blocks for writing pure-rust allocators was that thread-locals were slow in Rust (due to the need of lazy init and destructors).
There's recent progress on fast thread-locals:
That's still nightly-only, but looks like something stabilizable.
I think the most feature-full existing Rust allocator is elfmalloc: https://github.com/ezrosent/allocators-rs/tree/master/elfmalloc. I'd love to see a direct port of GitHub - microsoft/mimalloc: mimalloc is a compact general purpose allocator with excellent performance. to Rust and, with fast thread-locals, that should be feasible I think.
Well, that's (in-a-good-sense) much more info than what I was hoping to mine. Thank you sir!
Other allocator-like things: bumpalo - Rust, slab - Rust, heapless - Rust
I am working on a memory profiler for Python written in Rust, (Redirecting…), which has some similarities to a memory allocator (I override malloc()/free()/etc.). One relevant lesson, as mentioned above, is that Rust's thread-locals aren't good enough, I'm doing LTO with C thread locals for now until the newer stuff is stable.
Redox OS's system allocator is written in Rust:
*braces oneself*
Just a quick dumb question here, in order not to pollute the GitHub issue:
"Specifically, this allocator will maintain an intrusive doubly linked list of currently allocated heap blocks, and will provide an API to iterate the blocks."
From the little that I know, allocators typically keep an indexing structure for free block-bookkeeping. I understand that your suggestion is to extend this feature to allocated blocks too. A naive and rough heuristic would say that such an extension would double the allocator's response latency compared to a standard mechanism (if "meta" header fields were picked smart enough, memory could be kept about the same).
I can see why live heap feedback would be sweet to have, but is the performance trade-off acceptable? (that is, if I'm not missing something in the first place)
That's a great question, answered Tracking GlobalAllocator · Issue #9309 · rust-lang/rust-analyzer · GitHub.
The TL;DR is that this is specifically for profiling (you'd cfg-out the thing in production), so 2x time/space overhead would be totally acceptable. The current state of the art is using massif, which I think gives way higher overhead.
Yup, makes sense. Thanks again for the feedback! 
Cheers,
X
P.S. Since you brought up massif, you might also wanna check out heaptrack. I've found it to be faster and more practical.
