A function that takes &mut but returns a & borrow

[yshui]

I find myself trying to do this pattern:

1. Takes a mutable references to T.
2. Modify T somewhat.
3. Returns something that contains a immutable reference to T.

But the compiler insists that after 3, T is still being mutably borrowed. Is there a way to make the compiler realized T is only being immutably borrowed?

I know I can split the function into 2 functions, one for (1+2) and one for 3. But they are logically a single operation and splitting them makes the API ugly.

A concrete example:

fn get_or_init<T: Default>(v: &mut HashMap<u32, T>, key: u32) -> &T {
    v.entry(key).or_insert(T::default())
}

[cuviper]

One workaround is to return it back as a pair, something like (&Self, &T), but I don't know of any way to release the original &mut directly.

[yshui]

Maybe I can do this?

fn get_or_init<T: Default>(v: &mut HashMap<u32, T>, key: u32) -> (&HashMap<u32, T>, &T) {
    v.entry(key).or_insert(T::default());
    (v, unsafe {v.get(&key).unwrap_unchecked()})
}

Still feels clunky though

[yshui]

Ah you replied as I was just typing my previous reply!

[yshui]

This also doesn't work very well as an abstraction. If I want to embed a type that supports this kind of operation into a larger type, then I would need to add a similar method to the larger type as well.

[yshui]

OK, what if I have a macro like this:

unsafe fn amnesia_impl<'a, T>(t: *const T) -> &'a T {
    unsafe { &*t }
}
macro_rules! amnesia {
    ($e:expr) => {{
        let tmp: &_ = $e;
        unsafe { amnesia_impl(tmp) }
    }}
}

Would using amnesia! be safe? Hmm, how do i make sure the lifetime is unchanged?

[jessa0]

This feature of "downgrading of &mut T to &T" has been proposed several times before, but has some unresolved issues. One of which is described in this comment:

Consider the existing method get_mut defined on Mutex :

fn get_mut(&mut self) -> LockResult<&mut T>

The effect of this method is to bypass the lock and grant mutable access to the contents. One could easily write a wrapper for this method like so:

fn access<T>(m: &mut Mutex<T>) -> &T {
    m.get_mut().unwrap()
}

...

However, if we were to adopt this idea that taking in an &mut T and returning an &T implicitly “demotes” that &mut borrow to a shared borrow, that would imply that we could go on using the mutex afterwards.

[cuviper]

That's not safe because 'a is unbound -- you could choose any lifetime you want for it, even 'static, with no connection to whatever it's actually borrowing.

[H2CO3]

@yshui

Is there a way to make the compiler realized T is only being immutably borrowed?

No. It doesn't "realize" that because it is not true.

Unsafe code relies on this guarantee (ie. all borrows upholding a mutable borrow with the same lifetime), and there doesn't seem to be any way to express the same pattern without the current behavior.

Read more here.

[yshui]

Thanks!

Ah this is unfortunate. But although this is not generally safe, it can be safe for specific cases, right? e.g. In my example, I should be able to (using unsafe) forget about the &mut ref to HashMap by downgrading it to a &.

[simonbuchan]

While it wouldn't work in general, you might be able to do something with the signature (&mut 'a Container<'b, T>) -> &'b?

[VorfeedCanal]

@yshui

Ah this is unfortunate. But although this is not generally safe, it can be safe for specific cases, right?

Don't. Just don't. it may work, but it's awfully bad idea.

Technically, on machine code level &𝓣, &mut 𝓣, *const 𝓣 and *mut 𝓣 are the same thing.

All four are pointers that refer some object in memory.

The only difference is the information they pass to the compiler which it then uses to reason about your program and to stop you from doing mistakes.

@yshui

e.g. In my example, I should be able to (using unsafe) forget about the &mut ref to HashMap by downgrading it to a &.

Yes, but if you do that compiler wouldn't be able to reason about your program properly. And that's the only reason to use references in the first place!

If you really need to do that then it's better to just use pointers and wrap the whole thing in the sound wrapper.

Don't lie to the compiler, it'll backfire, sooner or later!

@simonbuchan

While it wouldn't work in general, you might be able to do something with the signature (&mut 'a Container<'b, T>) -> &'b?

It's only safe if you control the whole path between point where mutable borrow is created to a point where it's disappears (and turns into immutable borrow). And in that case it's easy to just replace mutable borrow with mutable pointer.

Trouble starts when some code which you don't control can observe both mutable borrow with “comes in” and immutable borrow which “comes out”.

[jessa0]

@yshui

In my example, I should be able to (using unsafe) forget about the &mut ref to HashMap by downgrading it to a &.

There are a couple problems with this idea:

1. This is still not "safe", in that your function should be marked unsafe, because it's not only the mutable reference to the HashMap itself that's potentially problematic, but also the fact that there's arbitrary T inside the HashMap which could in fact be a Mutex.
2. The downgrading of &mut to & would have to happen on the caller side, which is not something you can even do in Rust. This is what was proposed as the new language feature, which was never added due to many issues like this.

I would think of it like this: Rust references form a permissions system which is leveraged by methods like Mutex::get_mut. Trying to bend that permissions system makes the "safe" ecosystem fall down, which might be ok, but anything outside the safe ecosystem should be marked unsafe so usages will be verified by the programmer.

[quinedot]

There's no sound way to do this as

• if you need &mut, the initial borrow must be &mut
  • as turning & into &mut is UB
• reborrows cannot be valid longer than the source borrow
• lifetimes are a static analysis, not a dynamic shrinkable property

So in order to get a valid shared borrow out, you need an exclusive borrow (&mut) that is at least as long. Anything you do to make the compiler consider the exclusive borrow to be shorter means the shared reborrows are shorter too (or invalid - UB).

You need language support for some sort of chained borrow, akin in spirit to two-phased borrows, to make this work.

If you haven't, I recommend following @H2CO3's link and the discussions and blogs that discussion links to as well.

[simonbuchan]

@VorfeedCanal

It's only safe if you control the whole path between point where mutable borrow is created to a point where it's disappears (and turns into immutable borrow). And in that case it's easy to just replace mutable borrow with mutable pointer.

Trouble starts when some code which you don't control can observe both mutable borrow with “comes in” and immutable borrow which “comes out”.

What I meant was that the (custom) container type has some independent lifetime from the mutable borrow. I haven't checked, but I think things like GhostCell work on this principle.

[foconoco]

As others have pointed out, it is best to find another way to do this. That being said, I had a requirement for this in one of my projects. We created a macro that takes the reference, converts it to a raw pointer, then "rebinds" it's lifetime to the container without the mutable borrow. Obviously this is unsound to use over a generic T, because it could contain a mutex or some other type that that is invalid to use in this way. But if the type is not generic, it could be made sound. Here is the macro we used:

macro_rules! rebind {
    ($value:expr, $cx:ident) => {
        unsafe {
            let bits = $value.into_raw();
            $cx.rebind_raw_ptr(bits)
        }
    };
}

fn sample() {
   ...
    let x = rebind!(x, context);
}