Is std::alloc::Allocator likely to be stable any time soon?

help
1

In the module std::alloc there is an "experimental" (that is available on nightly only ) trait Allocator.

I was wondering when it might be stabilised ( so available on stable Rust )?

1 Like
2

As with (most? hopefully all.) unstable features there's a tracking issue for it: Allocator traits and std::heap · Issue #32838 · rust-lang/rust · GitHub
(You can find that if you e.g. look at one of the trait functions or in the source code).
Such an issue might give you a first look at what's missing for this to become stable. For the allocator API in particular there's a whole working group. Looks like there's still a bunch of unsolved issues, but the bug tracker there and the Zulip might have more information.

1 Like
3

Thanks, yes, after I posted I started looking at that. It seems stability for Allocator is hard to forecast at this point.

I am not sure I understand exactly what Allocator is for, I think the idea to provide more flexibility when allocating (for example) a BTreeMap, so that it can be allocated not from the global allocator but some other memory allocator. Is that right?

4

Yes. Some types in libstd already are generic over the allocator they use (defaulting to the Global one).
This allows a user to pick a more specific allocator for situations where it would make sense.
For example you can use an arena/bump allocator for a vec, when you precisely know that you need some memory for a period of time, but want to avoid deallocations while running your algorithm.

1 Like
5

IIRC; while this was active beforehand, this is getting quite a push for Rust in the Linux kernel usage (they're not so hot on taking down the whole system just because you opened Chrome) so it's quite a bit less likely to languish for years without anyone to push it forwards than the default for unstable features. (No disrespect to the Rust contributors!)

3 Likes
6

The Rust for Linux project is probably not going to push for allocator_api to be stabilized. It does not really fit what's needed in the kernel. See:

3 Likes
7

I wanted Layout argument to be optional for dealloc, since in the common case of libc free the layout is unused, and only causes unnecessary code bloat. Unfortunately, currently the allocator API can't express it, and LLVM can't optimize out the layout arguments.

Another unresolved problem IMHO is reporting errors from the allocator. The OOM error handler and OOM panic get a Layout, but not any information from the actual allocator. This leads to a situation where the Layout is used for guessing the root cause of allocation failure. This seems silly, given that the allocator could explicitly report the actual reason without guessing. But it needs an API for reporting allocation failures in a way that doesn't slow down the happy path.

8

Could an allocator take advantage of the Layout parameter to save heap space? Is this just a theoretical advantage?

9

Interesting! Sounds like it might still be doing so in spirit though:

Lastly, the series also removes our reliance on the allocator_api
unstable feature.

Long term, we still want to make such functionality available in
upstream Rust, but this allows us to make progress now and reduces our
maintainance burden.

So the literal feature allocator_api might not be landing, but the kernel definitely wants something like it.

10

Allocators generally need this information, but because libc malloc/free don't have it passed explicitly in their API, they have to get it in some other way (e.g. store it at a negative offset before the pointer). So because free() already tracks layout in its own private way, the Layout tracked on the Rust's side is redundant and useless for it.

Other custom allocators can be written to use the Layout arg as their source of truth, so whether this argument is useful or not depends on the allocator used.

11

To explain why I was asking, I have been working on making my BTreeMap support "custom" allocation ( or whatever you want to call it ). I re-packaged it here:

So this crate has pstd::alloc::Allocator and pstd::collections::btree_map::BTreeMap similar to std. I had a bit of a mammoth refactoring session the code so I didn't have to clone the allocator too much ( I didn't want a clone in every Vec, rather just one for the whole BTreeMap ).

One thing that caught me out is you have to call grow and shrink as appropriate instead of realloc, grow is not equivalent to realloc if you are shrinking!

12

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.