Update a non-Copy value through a mutable reference with a function that takes ownership

Consider the following scenario:

  • I have
    • a mutable reference ptr to a value of type T which is non-Copy (otherwise it's trivial),
    • and a function f with signature fn(T) -> T;
  • I want to replace the current value at *ptr with f(*ptr).

The obvious implementation *ptr = f(*ptr) clearly does not work because we hit error E0507.

The solution I could come up with is the following (link to playground):

pub fn update<T, F>(ptr: &mut T, f: F)
where
    F: FnOnce(T) -> T,
{
    // SAFETY:
    // - a `*const T` derived from a `&mut T` is always valid for reads;
    // - it is always properly aligned;
    // - it always points to a properly initialized value of type `T`.
    //
    // The value is now owned by `old` and we can no longer use `*ptr` directly,
    // see below.
    let old = unsafe { std::ptr::read(ptr) };

    let new = f(old);

    // SAFETY:
    // - a `*mut T` derived from a `&mut T` is always valid for writes;
    // - it is always properly aligned.
    //
    // We cannot assign directly to `*ptr` as that would count as a use
    // (because it attempts to drop the old value at `*ptr`), we must use
    // `std::ptr::write` instead to avoid the implicit drop.
    unsafe { std::ptr::write(ptr, new) };
}

which, disregarding all the comments, is basically a one-liner:

pub fn update<T, F>(ptr: &mut T, f: impl FnOnce(T) -> T) {
    unsafe {
        std::ptr::write(ptr, f(std::ptr::read(ptr)));
    }
}

Question

Is my abstraction update safe? (Sound?)

I know that one should put on a hazmat suit when dealing with unsafe and be extra cautious, and I personally just don't feel sure enough that my code is entirely correct. In particular

  • Should I be thinking about pinned values? Is my code correct only if T: Unpin? I don't think so, but I'm only 99.99% sure.
  • Should I be thinking about exception safety? What worries me is that if the call to f panics maybe things get dropped too many times (both *ptr and old), violating memory safety. Should I then be using ManuallyDrop? This point particularly concerns me, because I can see in an example that there are two drops, presumably one from dropping old in update and one from dropping x in main.
  • Are there any other issues I should be aware of?

Nope. But the general idea is fine. For an existing sound implementation see: replace_with - Rust

The problem with your implementation is indeed that it doesn’t correctly handle the case if f panics.

1 Like

You're reinventing replace_with aka take_mut aka core::mem::replace_with (RFC 1736), which is notoriously hard to get sound, see the discussion of the RFC for details.

1 Like

This, and also: if f panics then you've left *ptr as a moved-out-of (deinitialized) value, which is semantically wrong and can cause all sorts of havoc even if there's no drop glue involved. The referent of an &mut T must always be a valid T.

2 Likes

You never need to worry about pinned values unless you mention Pin yourself. Pinning works by making it unsafe to get an &mut T when the T is supposed to be pinned, so pinning can never be a reason why a function of &mut T is unsound.

3 Likes

That was my worry indeed.

I'll go study replace_with/take_mut, neither of which I was aware of. Thanks everybody!

Honestly, all four answers by the three of you should be accepted answers. I'll mark @steffahn's one just because he's been the fastest. Thanks everybody! :slight_smile:

I would strongly recommend against using unsafe for this. In general, pulling unsafe tricks to circumvent the ownership and borrowing system will lead to unsoundness and UB, unless you really know what you are doing (i.e., don't need to ask).

Just put a T: Default bound on T and use mem::take().

By the way, in case that isn’t clear: a common (and typically preferred, if possible) solution to this problem is to re-write your fn(T) -> T function into a fn(&mut T) function.

Yes, yes, of course! :smiley:

I was deliberately thinking of a situation where for some reasons one already has a function fn(T) -> T and wants to use it for instance to update items in a Vec.

Well, for Vec, Default is super cheap, so you can simply *ptr = f(mem::take(ptr)) :slight_smile:

Also it’s usability concerns like this that motivate that most functions (e.g. all the methods of Vec itself) modifying a Vec<Foo> will take a &mut Vec<Foo> argument instead of accepting and returning an owned Vec<Foo>.

1 Like

Fine, but I wonder how one could learn all the intricacies of unsafe Rust without ever asking such a question!

I feel this is a case where unsafe Rust puts an extra layer of difficulty w.r.t. C for example, because in C one does not have to think about exception safety and implicit drops, hence the solution to this specific problem is trivial.

I agree with you that implementing Copy, or Default, or any other safe solution is better here; I was just trying to study unsafe Rust more in depth and I thought about this problem. Now I know that it's harder than it seems because of exception safety, as I feared.

1 Like

No, no, I meant modifying items inside a Vec: update(&mut some_vec[42], f).

Ah, I misunderstood that, thanks for the clarification.

I guess replace_with exists for a reason, it occasionally has practical use; but otherwise, of course, for the vector item types, the same thing applies, that generally the function f likely could’ve better been written as a fn(&mut T) to begin with :slight_smile:

1 Like

I'm not sure about all, but for most things, it's the Nomicon.

That's indeed the reference from which I try to learn stuff when I find the time. Then I thought about this magical function update, which might come in handy as testified by the very issue https://github.com/rust-lang/rfcs/pull/1736 which says

This common pattern deserves a place in the standard library.

I then tried to implement it, given the limited knowledge that I have. The major problem is that I have no way to check whether I came up with the correct solution precisely because rustc is incapable of validating unsafe code! I had the feeling that it was not exception safe because things could get dropped twice, but didn't know how to address that: I then resorted to asking experts. I will now continue my journey by studying how replace_with can be safely implemented.

I struggle to see the issue in all this.

1 Like

Cases like replace_with/take_mut also stretch our notions of soundness in interesting ways. If you think too deeply about what exactly it means for code to be sound, it’s kind-of hard to define it properly – or put differently, the most proper way to define this that I can think of would, at least a priori, make the API that replace_with/take_mut provide actually unsound.

The argument goes that an API like

pub struct Foo(…private fields…);
impl Foo {
    pub fn with(callback: FnOnce(&mut Foo)) {  ……  }
    pub fn super_unsafe(self) { …UB… }
}

too could – in isolation – be considered sound, if it never allows users to call the super_unsafe function, by the principle of never handing a user an owned value of type Foo.

However, with replace_with, you can obtain an owned value from a mutable reference. So both together are definitely unsound. Without some global community consensus mechanism, it’s however hard to establish which one of replace_with or Foo to blame (so conservatively, arguably, better both be considered unsound).

In the case of this replace_with API, there are no good practical reasons for allowing something like Foo, so as far as I’m aware, the community consensus is that replace_with-style functionality ought to be sound.

7 Likes

Wow this is super interesting, thanks for taking the time to explain this! I always admire people who have such a clear view of these difficult topics :clap:

Well I guess for now I have enough new material to study and think about in the weekend! :sweat_smile:

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.