Are these `push` and `pop` operations on `Cell<Vec<T>>` safe?

Sometimes I found myself need to modify a Vec from multiple places, where I find it would be best if I can pop elements from a &Cell<Vec<T>>, then I begin to wonder what kind of operations are safe for a &Cell<Vec<T>>. For example, are these push and pop operations safe?

use std::cell::Cell;
use std::ptr;

fn cell_vec_push<T>(vec: &Cell<Vec<T>>, value: T) {
    // See <https://doc.rust-lang.org/1.80.1/src/alloc/vec/mod.rs.html#1993>.
    unsafe {
        let vec = &mut *vec.as_ptr();
        let len = vec.len();

        // This may panic and calls the destructor of `value`.
        vec.reserve(1);

        ptr::write(vec.as_mut_ptr().add(len), value);
        vec.set_len(len + 1);
    }
}

fn cell_vec_pop<T>(vec: &Cell<Vec<T>>) -> Option<T> {
    unsafe { (*vec.as_ptr()).pop() }
}

Note that these functions can be implemented without unsafe codes by taking the Vec out of Cell, modify it, then put it back:

/// Ensures original `Vec` is restored even if operation panics.
struct RestoreOnDrop<'a, T>(&'a Cell<Vec<T>>, Vec<T>);

impl<T> Drop for RestoreOnDrop<'_, T> {
    fn drop(&mut self) {
        mem::forget(self.0.replace(mem::take(&mut self.1)));
    }
}

fn cell_vec_push_safe<T>(vec: &Cell<Vec<T>>, value: T) {
    RestoreOnDrop(vec, vec.take()).1.push(value);
}

fn cell_vec_pop_safe<T>(vec: &Cell<Vec<T>>) -> Option<T> {
    RestoreOnDrop(vec, vec.take()).1.pop()
}

But I think in this specific case, taking the Vec of the Cell may be unnecessary, and the compiler is not able to optimize the move around operations away. So I wrote those unsafe implementations at the beginning.

Of course there is always RefCell, but accessing it requires runtime check which might not be optimal.

If my unsafe implementations are safe, do you think it is reasonable to add these functions to the standard library?

If any one is interested, my use case is writing a graph traversal algorithm, click here to see a simplified version of it.

The original version:

use std::hint;

struct Node1 {
    value: u32,
    neighbors: Vec<usize>,
}

fn dfs_1(graph: &mut [Node1], node: usize) {
    let value = graph[node].value;

    while let Some(child) = graph[node].neighbors.pop() {
        let child_value = graph[child].value;

        dfs_1(graph, child);

        // Do something with `value` and `child_value`.
        hint::black_box((value, child_value));
    }
}

If I can pop a Cell<Vec<T>> directly, the code above can be simplified to:

struct Node2 {
    value: u32,
    neighbors: Cell<Vec<usize>>,
}

fn dfs_2(graph: &[Node2], node: &Node2) {
    while let Some(child) = cell_vec_pop(&node.neighbors) {
        let child = &graph[child];

        dfs_2(graph, child);

        // Do something with `node.value` and `child.value`.
        hint::black_box((node.value, child.value));
    }
}

Which could avoid some bound checking overhead. Here is the comparison of generated assembly: https://godbolt.org/z/ErjYaKTha.

Here is a playground link for all the code above: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=83c255923ca33ac343cb332c21f5eaa9.

vec: &Cell<Vec>

let vec = &mut *vec.as_ptr();

No. Absolutely not. It doesn’t matter what dance moves you do to try to make it work, casting a shared reference to an exclusive reference is always, 100%, immediate undefined behavior. You’re violating one of the most sacred rules of Rust.

Wrapping in a Cell is a red herring here. It doesn’t, and can’t, do anything, because you’re bypassing the sole reason it exists: to provide a safe and sound API for shared mutability.

Miri would have trivially caught this UB. Please, always write comprehensive tests and run them on Miri if you want to write unsafe code.

1 Like

Do you mean Cell::as_ptr? It returns a *mut Vec<T> pointer, there is no casting from shared reference to exclusive reference.

1 Like

This is not casting a &Vec<T> to a &mut Vec<T>, it's going from &Cell<Vec<T>> to *mut Vec<T> (which internally uses UnsafeCell::get) to &mut Vec<T>.

The use of UnsafeCell::get makes it safe as long as no other reference to the Vec<T> is created while the one just produced exists.

8 Likes

Use of Cell here is at very least very confusing, because it's meant to be a safe interface for Copy types, and this is neither safe nor a Copy type.

You should use UnsafeCell directly.


I think your code has UB, because you're obtaining the pointer without going through UnsafeCell's getters (bypassing Cell.get() too). From the Rust docs:

unsafe fn not_allowed<T>(ptr: &UnsafeCell<T>) -> &mut T {
  let t = ptr as *const UnsafeCell<T> as *mut T;
  // This is undefined behavior, because the `*mut T` pointer
  // was not obtained through `.get()` nor `.raw_get()`:
  unsafe { &mut *t }
}
4 Likes

I didn’t bypass UnsafeCell::get, I called Cell::as_ptr for the pointer, which itself calls UnsafeCell::get.

6 Likes

Pop is sound, but push is not sound because the global allocator could panic, then call cell_vec_push through the panic hook set via set_panic_hook. This would then create aliased &mut. It is unreasonable to expect the global allocator to be panic free, and setting the panic hook is a safe operation

See these two issues for details

Just use a refcell instead.

9 Likes

Edit: It seems OP have covered these basic safe code from the beginning. I'll leave the code for others but you can ignore it.


You don't need any unsafe ops for it.

use std::cell::Cell;

fn cell_vec_push<T>(vec: &Cell<Vec<T>>, value: T) {
    let mut v = vec.take();
    v.push(value);
    vec.set(v);
}

fn cell_vec_pop<T>(vec: &Cell<Vec<T>>) -> Option<T> {
    let mut v = vec.take();
    let res = v.pop();
    vec.set(v);
    res
}
4 Likes

I see. But I think that’s only because that the panic hook runs before unwinding.

How about:

use std::cell::Cell;

// This function does not panic.
fn cell_vec_try_push<T>(vec: &Cell<Vec<T>>, value: T) -> Result<(), T> {
    unsafe {
        let vec = &mut *vec.as_ptr();

        if vec.try_reserve(1).is_err() {
            return Err(value);
        }

        // We have successfully reserved space for pushing, so call will not panic.
        vec.push(value);

        Ok(())
    }
}

// This function could panic, but not before the mutable reference is released.
fn cell_vec_push<T>(vec: &Cell<Vec<T>>, value: T) {
    cell_vec_try_push(vec, value).ok().unwrap();
}

It's pretty much mandate to read the whole docs of the type thoroughly before doing any unsafe ops.

Cell implements interior mutability by moving values in and out of the cell. That is, an &mut T to the inner value can never be obtained, and the value itself cannot be directly obtained without replacing it with something else.

Basically the entire safety guarantee of the Cell<T> is built around that the reference to the inner T is never be obtained. Only operations allowed is to copy out its bytes or overwrite them. You can build your own abstractions on it with your own set of limitations, but in this case Cell is not a right building block. For such advanced use case you may use UnsafeCell instead.

2 Likes

Note that the thing that is unsound due to reentrancy here would be equally unsound if written on UnsafeCell. Cell itself doesn’t contribute to the problem. (I agree with you, though, that it might be less confusing to use an UnsafeCell given the lack of using any Cell operations on this Cell.)

1 Like

I avoided UnsafeCell because I consider push and pop functions user facing library functions. Cell<Vec<T>> looks like a type a user would use, so I want to add functions operate on it, but UnsafeCell<Vec<T>> is more like an implementation detail, making users construct UnsafeCell<Vec<T>> is a little awkward.

1 Like

I'm not talking about OOM, even bugs in the allocator could cause it to panic.

The allocator itself could still run arbitrary code even when succeeding, including accessing a thread-local &Cell<Vec<T>>. (Perhaps the allocator offers a hook to log information about allocations?)

3 Likes

@RustyYato, @kpreid: But the Allocator trait is an unsafe trait, the implementor has the responsibility to implement it correctly. I think the reasoning about buggy allocators only applies if Allocator is a safe trait?

Implementations of unsafe traits are obligated to obey the safety requirements that are written in the trait’s documentation. The safety requirements for Allocator and GlobalAlloc have no rules pertaining to what the allocator may call.

Regarding panicking in particular, GlobalAlloc does say:

  • It’s undefined behavior if global allocators unwind. This restriction may be lifted in the future, but currently a panic from any of these functions may lead to memory unsafety.

So, while a global allocator that panics is currently buggy, this is not guaranteed to be true in the future.

3 Likes

Now I think we have a circular reasoning. In order for the allocator to cause UB, it has to depend on the implementation of cell_vec_push, and in order for cell_vec_push to be unsound, it has to depend on the implementation of the allocator. So when UB happens, which implementation do I blame?

If I were a standard library developer, am I OK to assert that cell_vec_push being sound, and ask the allocator to not causing UB?

in order for cell_vec_push to be unsound, it has to depend on the implementation of the allocator.

But it calls the allocator regardless of what allocator implementation is being used. cell_vec_push could be defined in library crate A which is used by an allocator implementation in library crate B and also binary crate C which uses the allocator in a way that turns out to cause reentrance.

The allocator traits being unsafe traits doesn't mean “if anything goes wrong, it’s the implementor’s fault”. Unsafe traits have specific safety requirements, and implementations are free to call any safe code they like unless otherwise specified. Unsafe trait implementations don't even have to contain any unsafe code!

In general, by providing a safe function that uses unsafe operations, you are making a claim that the function can be called in any way whatsoever, including reentrancy, without causing UB via those unsafe operations. Any other approach would fall apart in the face of people writing complex programs with many transitive dependencies.

6 Likes

OK, how about this one:

use std::cell::Cell;
use std::mem;

fn needs_allocation<T>(vec: &Cell<Vec<T>>) -> bool {
    unsafe {
        let vec = &*vec.as_ptr();

        vec.len() == vec.capacity()
    }
}

fn allocate_one<T>(vec: &Cell<Vec<T>>) {
    struct RestoreOnDrop<'a, T>(&'a Cell<Vec<T>>, Vec<T>);

    impl<T> Drop for RestoreOnDrop<'_, T> {
        fn drop(&mut self) {
            mem::forget(self.0.replace(mem::take(&mut self.1)));
        }
    }

    RestoreOnDrop(vec, vec.take()).1.reserve(1);
}

fn cell_vec_push<T>(vec: &Cell<Vec<T>>, value: T) {
    if needs_allocation(vec) {
        allocate_one(vec);
    }

    unsafe { (*vec.as_ptr()).push(value) };
}

I employed a hybrid approach, which is just a little bit slower when allocation happens. To my understanding, even if reentrancy happens, stack overflow could happen, but no undefined behavior will occur.

That looks ok to me. It's similar to Tokio's internal rc cell.

1 Like