How to batch allocate Arc<T> in Rust?

I don't know how to express the question in Rust, so I will express what we do in go first.

There's a common pattern to batch allocate some items in go, like this:

s := make([]*Item, n)
for i := 0; i < n; i++ {
    s[i] = new(Item)
}

This will allocate n times and the address of the Items are not guaranteed to be continuous, and here's an optimized version:

s := make(map[int]*Item, n)
values := make([]Item, n)
for i := 0; i < n; i++ {
    s[i] = &values[i]
}

This will only call mallocgc once and the memory layout of Items are guaranteed to be continuous, which is good for cache locality. The performance is about 2x compares to the previous version.

And here's the question: how can I do the equivalent in Rust?

I know by using let s: Vec<Item> = Vec::with_capacity(n) will only allocate once and the memory layout is guaranteed to be continuous, but I can't find a way to do similar thing for Arc.

For example, If I use let s: Vec<Arc<Item>> = Vec::with_capacity(n), it will allocate space for n * Arc, and then calling something like Arc::new(Item::default()) for n times to initialize the vec will need allocation for n times, and the memory layout are likely to be dispersed. This will lead to a great performance loss.

In async network programming, Arc<T> is indispensable because we are always using patterns such as fan-out or spawn some task to run in the background, and lifetimes doesn't help in those cases.

I've not found a way to convert Vec<Item> to Vec<Arc<Item>> without extra allocation cost.

Would Arc<Vec<Item>> or Arc<[Item]> be a better fit for your usecase? It only makes a single allocation for the Arc and guarantees that the Items are contiguous in memory, but doesn’t carry any identification of a particular item with it— Every copy has shared access to the entire collection.

3 Likes

Arc<[T]> is the closest you can get. It's possible to create it using iter.collect().

The implementation of std::sync::Arc will free its memory when the Arc is dropped, so there's no way to have an array of stdlib's Arc's in one allocation. This just completely breaks how this type is implemented. You would need another type for this.

Additionally, Arc requires to have a guaranteed stable address. A type like Vec<InlineArc> would reallocate its data when the capacity changes, changing addresses of the elements. This would make every other reference to InlineArc dangling.

Maybe you could have triomphe::Arc::from_header_and_iter and then hand out instances OffsetArc, but I haven't checked if these two work together.

2 Likes

Would an arena allocator using one big block of continuous memory solve that problem? If Arc took an allocator as a generic argument like Vec does

1 Like

A hypothetical Arc<T, ArenaAllocator> could make allocations contiguous (with refcounts in between), but it's not available in stable Rust. If you're lucky and allocate them all at once, even the regular global allocator (Arc<T, Global>) could end up placing them next to each other.

OTOH Arc<&'arena T> would be a double indirection, and most likely spoil any performance gain from the arena.

1 Like

sharded_slab can be used as a pool or slab of T. The overall pool can be shared between threads by wrapping it an Arc. An individual T can be shared between threads using get_owned, which increments the pool Arc's ref count. To mutate a T, you'll have to wrap it in a Mutex or RwLock.

Arc<Vec<Item>> adds an extra indirection (performance cost) with no benefits, as Arc only gives shared access to the inside and &Vec<T> has no more functionality than &[T].

For this reason, of these two, only Arc<[Item]> should be used.

1 Like

While I agree with your general point, this is not strictly true— You can get mutable access or take back ownership of the Vec via into_inner and friends.

Also, constructing an Arc<[Item]> from an existing Vec<Item> generally requires copying all of the items into a new location in memory; in some circumstances this may outweigh the access-speed benefits of Arc<[Item]>.

2 Likes

I indeed missed into_inner. It may actually be a legitimate use case for a Vec, although a niche one.

For the construction cost, that's O(n), yeah. From the OP's post I inferred that they want to create an array of Items. In that case creating a Arc<Vec<Item>> or creating an Arc<[Item]> shouldn't be too different. Of course if they already have a Vec of Items, then keeping that is possibly cheaper than copying it over.

2 Likes

Hi, thanks very much! But building an Arc<[T]> will also need n allocations using iter.collect()?

No; it can make a single allocation the correct size in many cases, i.e. where the iterator can report its length before iterating. If the number of elements is not known up-front, I expect it uses a strategy similar to Vec’s, which performs O(log n) allocations.

4 Likes

Thanks! Seems that Arc<[T]> is the only way in rust, though it cannot get a Arc directly, so it needs to pass an extra index arg to the fn which needs to be spawned.

That's what OffsetArc is for in triomphe crate.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.