Guarantee that Vec::default() does not allocate?

help
21

See this playground:

Imagine the loop is an infinite loop. The (unbound) mpsc will be filled up with more and more Vecs.

22

No, it couldn't. A lot of unsafe code already relies on it heap-allocating, which means that the buffer pointer doesn't change when the Vec (ptr, len, cap triple) itself is moved. Breaking this assumption would cause nothing but pain and suffering.

6 Likes
23

No: https://doc.rust-lang.org/std/vec/struct.Vec.html#guarantees

Most fundamentally, Vec is and always will be a (pointer, capacity, length) triplet. No more, no less. The order of these fields is completely unspecified, and you should use the appropriate methods to modify these.

9 Likes
24

For the record, Vec::new()'s documentation states explicitly that it will not allocate:

Constructs a new, empty Vec<T>.

The vector will not allocate until elements are pushed onto it.

However, there's no formal guarantee that Vec::default() calls Vec::new().

25

Yes, that was my point. I think it would be trivial to add though?

(Though I don't need it anymore because of other problems regarding capacity. Instead I'll just inhibit the drop handler from executing, as shown in my above post.)

26

This seems reasonable to me to document that <Vec<T> as Default>::default does the same as Vec::<T>::new.

There’s this sentence in the docs:

In particular, if you construct a Vec with capacity 0 via Vec::new, vec![], Vec::with_capacity(0), or by calling shrink_to_fit on an empty Vec, it will not allocate memory.

And to me this doesn’t appear to be intended to be an exhaustive listing that deliberately not mentions Vec::default. So IMO, a change to the Vec::default docs could also reasonably include an addition of Vec::default into the list in that sentence, to make things even more clear.

Once there’s the guarantee that Vec::default doesn’t allocate, you also know you can always detect it based on capacity, as also made clear by this sentence in the docs.

Vec will allocate if and only if mem::size_of::<T>() * capacity() > 0.

(On an unrelated note, that sentence could IMO be improved to spell “Vec<T>” instead of “Vec”.)

Which means that once you account for zero-sized types (e.g. by handling them specially), you’d be guaranteed that Vec::new() does indeed have capacity 0.

5 Likes
27

So when combining @2e71828's note to use std::mem::replace(&mut v, Vec::with_capacity(0)) instead of std::mem::take, and adding a check for size_of::<Element> > 0, it would solve my problem (formally).

If the guarantees get extended to Vec::default in the future, then I could use std::mem::take again. But even without such an explicit guarantee, it seems safe to assume that Vec::default() doesn't allocate, even if not formally guaranteed as of yet.

But the key point is to handle ZSTs properly.

I think my idea to use ManualDrop could lead to other problems (as I need to take care to not leak any other resources that might be part of the guard). I guess another option would be to add a bool to the guard and check that value in the drop implementation.

I'll think a bit about what's best to do without being too obscure.

28

I think I found a way to be more verbose in what's happening where I don't need to check for a "non-allocated" Vec at all:

#[derive(Debug)]
pub struct BufWriteGuard<T> {
    buffer: Vec<T>,
    recycler: Option<mpsc::UnboundedSender<Vec<T>>>,
}

impl<T> BufWriteGuard<T> {
    fn new(buffer: Vec<T>, recycler: mpsc::UnboundedSender<Vec<T>>) -> Self {
        BufWriteGuard {
            buffer,
            recycler: Some(recycler),
        }
    }
    pub fn finalize(mut self) -> BufReadGuard<T> {
        BufReadGuard::new(take(&mut self.buffer), self.recycler.take().unwrap())
    }
}

impl<T> Drop for BufWriteGuard<T> {
    fn drop(&mut self) {
        if let Some(recycler) = self.recycler.take() {
            let _ = recycler.send(take(&mut self.buffer));
        } else {
            println!("Didn't recycle BufWriteGuard::buffer")
        }
    }
}

(Playground)

When converting the BufWriteGuard to a BufReadGuard, I'll have to take the recycler (or clone it, which would be extra overhead). So if recycler is an Option<UnboundedSender>, then I can use that option to memorize that the drop handler shouldn't recycle anything after finalize has been called (because finalize will set recycler to None).

So I can completely circumvent checking capacity().

Anyway, thanks a lot to everyone explaining to me the behavior of Vec (especially the interesting behavior in the ZST case, address stability which requires heap allocation, and pointing me to the relevant parts in the docs). Maybe if I have time, I'll propose an update to the API gurantees regarding Vec::default; but I'd also be happy if someone else likes to do it. I'm not yet very familiar with contributing yet and extending stability guarantees is something that really needs to be done right!

29

Don't worry. This is why new guarantees like this need both sign-off from the libs-api team as well as a 10-day comment period so the community can also raise concerns.

You can definitely submit a PR, and the reviewer can help if there are any issues.

30

Okay, I gave it a try: #100872.

2 Likes
31

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.