Mutable borrow seems to go on for too long

help
#1

I've recently been trying to wrap my head around how to pass structs/enums containing mutable references through programs but keep getting stuck on problems like the following example.

#[derive(Debug)]
pub enum NumericalTypes<'a> {
    F32(&'a mut f32),
    I32(&'a mut i32),
}

pub struct NumericalCallback<T> {
    callback: fn(&mut T),
}

pub struct Container<'a> {
    callback: NumericalCallback<NumericalTypes<'a>>,
    value: f32
}

fn add_one(data: &mut NumericalTypes) {
    match data {
        NumericalTypes::F32(data) => **data += 1.0,
        NumericalTypes::I32(data) => **data += 1,
    }
}

fn in_a_closure(data: &mut NumericalTypes, mut value: impl FnMut(&mut NumericalTypes)) {
    value(data);
    value(data);
}

fn main() {
   let mut container = Container {
       callback: NumericalCallback {
           callback: add_one
       },
       value: 3.0
   };
   
   dbg!(container.value);
   
   ((container.callback).callback)(&mut NumericalTypes::F32(&mut container.value));
    in_a_closure(&mut NumericalTypes::F32(&mut container.value), |v| {
        add_one(v);
        add_one(v);
    });
   
   dbg!(container.value);
}

Playground

This code won't compile because the first mutable borrow seems to inherit the lifetime of Container which only ends at the end of the program. Even though the value is only actually borrowed by add_one for the length of the function. So, my question is, how can I tell Rust that the callback function will only borrow the data for its length and get the code to compile.

#2

Your playground compiles; I assume you mean it fails to compile if you try to use the callback more than once with the same value:

   (container.callback.callback)(&mut NumericalTypes::F32(&mut value));
   (container.callback.callback)(&mut NumericalTypes::F32(&mut value));

It's not necessarily true that the value is only borrowed by add_one for the length of the function. add_one can borrow the value for any length at least as long as the function. Let's take a closer look at the types involved with your container:

   // Container<'a> for some concrete 'a
   let container = Container {
       // NumericalCallback<T> with T = NumericalTypes<'a>
       callback: NumericalCallback {
           // fn(&mut NumericalTypes<'a>)
           callback: add_one
       }
   };

You end up with a callback that can only be called with NumericalTypes that borrow their values for some single lifetime 'a. That lifetime is stuck behind a &mut and thus invariant -- it cannot soundly be shrunk. Therefore, every value that gets borrowed has to get borrowed for the same lifetime. If you try to borrow the same value twice, the two borrows with the same lifetime will overlap, hence the error.

The actual lifetime in the case of the playground is going to correspond to the two attempted callback calls.

How to make this work? Well, you probably need your function pointer to work with any lifetime of NumericalTypes:

pub struct NumericalTypesCallback {
    callback: fn(&mut NumericalTypes<'_>),
}

pub struct Container {
    callback: NumericalTypesCallback,
}

You can't do this with NumericalCallback<T> as it currently is defined, because T must resolve to a single type, and NumericalTypes<'x> is a different type for every distinct lifetime 'x.

Another option is to make NumericalCallback generic over the function, instead of over the argument:

pub struct NumericalCallback<Func> {
    callback: Func,
}

pub struct Container {
    callback: NumericalCallback<fn(&mut NumericalTypes<'_>)>,
}
1 Like
#3

Thanks for the detailed reply!

The problem that I was seeing makes a lot more sense now.

I've created a slightly different formulation of the same issue. I now understand that I could solve it by making NumericalCallback generic over each function. But is there any way to constrain T to shrink allow rust to shrink it's lifetime to the lifetime of the function?

Playground

#4

There's no direct way. There are two problems:

  • The T in a &mut T must be invariant; if it could shrink, you could do unsound things
    • E.g. push a &local_borrow into a &mut Vec<&'static T>
  • A type parameter T must resolve to a single type
    • ...and &mut NumericalTypes<'long> is not the same type as &mut NumericalTypes<'short>

So if you want to keep fn(&mut T) around and still get the flexibility desired, you're going to need some way to construct multiple types that differ in lifetime from a single resolved T. This is possible with indirection and a GAT or a higher-ranked trait bound (instead of a higher-ranked type like fn(&mut NumericalTypes<'_>), but it can be pretty unergonomic.

Still, let's give it a shot. The heart of the idea is that you need to have some single concrete type X that can represent a NumericalTypes<'y> for any lifetime 'y. We do this on stable by having a GAT-like helper trait:

pub trait Mutably<'mb> {
    type Borrowed: 'mb;
}

pub struct NumericalRepresentative;
impl<'mb> Mutably<'mb> for NumericalRepresentative {
    type Borrowed = NumericalTypes<'mb>;
}

Now we have a single type (NumericalRepresentative) that can construct NumericalTypes<'_> for any lifetime, given the appropriate trait bound U: for<'any> Mutably<'any>. Unfortunately I don't know of any way to avoid this boiler-plate in the general case (like a lifetime generic struct such as NumericalTypes).

Usually I've see this helper-GAT in coordination with a generic callback, at which point trait bounds on the Fn-like traits starts coming into play. Your example is different in that you're wanting to work with some concrete callable types (e.g. fn(&mut NumericalTypes<'_>)) instead. This might not be the best approach, but what I came up with on the fly is to capture both the higher-ranked bound mentioned above and the concrete types of your function pointers in another trait:

pub trait Callback: for<'any> Mutably<'any> {
    type Simple: for<'any> Fn(&mut <Self as Mutably<'any>>::Borrowed);
    type Complex: for<'any> Fn(&mut <Self as Mutably<'any>>::Borrowed, usize);
}

And then implement it:

impl Callback for NumericalRepresentative {
    type Simple = fn(&mut NumericalTypes<'_>);
    type Complex = fn(&mut NumericalTypes<'_>, usize);
}

This is more boiler-plate as I couldn't think of a way to blanket-implement the trait (there's no way I'm aware of to go from the helper-GAT trait to the NumericalTypes<'_> helper). This might be cleaner with built-in GAT support.

Anyway, moving on, we can now use this scaffolding like so:

pub enum NumericalCallback<T: Callback> {
    SimpleCallback(<T as Callback>::Simple),
    ComplexCallback(<T as Callback>::Complex),
}

After that I had to adjust the construction, as coercion didn't apply through the trait projection for whatever reason:

    let mut container = Container {
-       callback: NumericalCallback::SimpleCallback(add_one),
+       callback: NumericalCallback::SimpleCallback(add_one as _),
        value: 3.0
    };

And then it worked.

A macro could help with the boilerplate:

macro_rules! represent_callbacks {
    ($repr:ident, $gentype:ident) => {
        pub struct $repr;
        impl<'mb> Mutably<'mb> for $repr {
            type Borrowed = $gentype<'mb>;
        }

        impl Callback for $repr {
            type Simple = fn(&mut $gentype<'_>);
            type Complex = fn(&mut $gentype<'_>, usize);
        }
    }
}  

represent_callbacks!(NumericalRepresentative, NumericalTypes);

Playground.

#5

Filed #95863.

#6

Ah I clearly need to spend more time looking at what can be done with traits. Thanks again for the help!