`Fn` is not general enough

I am working on a zero sized keypath and facing the following problem when trying to initialize a closure. I get the error: “Fn is not general enough.”

While going through some random solutions, I found a quick fix by calling get_ref and set_ref functions to get rid of the compile-time error. Previously, I used function pointers to create keypaths, but they take 8 bytes, whereas I am aiming to achieve a true zero sized closure.

error: implementation of `Fn` is not general enough
   --> tests/manual_kp_concretetype.rs:129:5
    |
129 | /     Kp{
130 | |         get: |x: &Size| Some(&x.width),
131 | |         set: |x: &mut Size| Some(&mut x.width),
132 | |         _p: std::marker::PhantomData
133 | |     }
    | |_____^ implementation of `Fn` is not general enough
    |
    = note: closure with signature `fn(&'2 Size) -> Option<&u32>` must implement `Fn<(&'1 Size,)>`, for any lifetime `'1`...
    = note: ...but it actually implements `Fn<(&'2 Size,)>`, for some specific lifetime `'2`


Main issue

// Manual keypath: Size -> width
fn size_width_kp() -> Kp<
    Size,
    u32,
    &'static Size,
    &'static u32,
    &'static mut Size,
    &'static mut u32,
    impl Fn(&Size) -> Option<&u32>,
    impl Fn(&mut Size) -> Option<&mut u32>,
> {
    Kp{
        get: |x: &Size| Some(&x.width),
        set: |x: &mut Size| Some(&mut x.width),
        _p: std::marker::PhantomData
    }
}

Full code, i have added quick fix for fn rect_size_kp


#[derive(Clone)]
pub struct Kp<R, V, Root, Value, MutRoot, MutValue, G, S>
where
    Root: std::borrow::Borrow<R>,
    MutRoot: std::borrow::BorrowMut<R>,
    MutValue: std::borrow::BorrowMut<V>,
    G: Fn(Root) -> Option<Value>,
    S: Fn(MutRoot) -> Option<MutValue>,
{
    /// Getter closure: used by [Kp::get] for read-only access.
    pub get: G,
    /// Setter closure: used by [Kp::get_mut] for mutation.
    pub set: S,
    pub _p: std::marker::PhantomData<(R, V, Root, Value, MutRoot, MutValue)>,
}

impl<R, V, Root, Value, MutRoot, MutValue, G, S> Kp<R, V, Root, Value, MutRoot, MutValue, G, S>
where
    Root: std::borrow::Borrow<R>,
    Value: std::borrow::Borrow<V>,
    MutRoot: std::borrow::BorrowMut<R>,
    MutValue: std::borrow::BorrowMut<V>,
    G: Fn(Root) -> Option<Value>,
    S: Fn(MutRoot) -> Option<MutValue>,
{
    pub fn new(get: G, set: S) -> Self {
        Self {
            get: get,
            set: set,
            _p: std::marker::PhantomData,
        }
    }

    #[inline]
    pub fn get(&self, root: Root) -> Option<Value> {
        (self.get)(root)
    }

    #[inline]
    pub fn get_mut(&self, root: MutRoot) -> Option<MutValue> {
        (self.set)(root)
    }

    #[inline]
    pub fn then<SV, SubValue, MutSubValue, G2, S2>(
        self,
        next: Kp<V, SV, Value, SubValue, MutValue, MutSubValue, G2, S2>,
    ) -> Kp<
        R,
        SV,
        Root,
        SubValue,
        MutRoot,
        MutSubValue,
        impl Fn(Root) -> Option<SubValue>,
        impl Fn(MutRoot) -> Option<MutSubValue>,
    >
    where
        SubValue: std::borrow::Borrow<SV>,
        MutSubValue: std::borrow::BorrowMut<SV>,
        G2: Fn(Value) -> Option<SubValue>,
        S2: Fn(MutValue) -> Option<MutSubValue>,
    {
        let first_get = self.get;
        let first_set = self.set;
        let second_get = next.get;
        let second_set = next.set;

        Kp::new(
            move |root: Root| first_get(root).and_then(|value| second_get(value)),
            move |root: MutRoot| first_set(root).and_then(|value| second_set(value)),
        )
    }
}

struct Size {
    width: u32,
    height: u32,
}

struct Rectangle {
    size: Size,
    name: String,
}

// Manual keypath: Rectangle -> Size
fn rect_size_kp() -> Kp<
    Rectangle,
    Size,
    &'static Rectangle,
    &'static Size,
    &'static mut Rectangle,
    &'static mut Size,
    impl Fn(&Rectangle) -> Option<&Size>,
    impl Fn(&mut Rectangle) -> Option<&mut Size>,
> {
    Kp::new(
        get_ref(|x: &Rectangle| Some(&x.size)),
        set_ref(|x: &mut Rectangle| Some(&mut x.size)),
    )
}

fn get_ref<T, U, F>(f: F) -> F
where
    F: Fn(&T) -> Option<&U>,
{
    f
}

fn set_ref<T, U, F>(f: F) -> F
where
    F: Fn(&mut T) -> Option<&mut U>,
{
    f
}

// Manual keypath: Size -> width
fn size_width_kp() -> Kp<
    Size,
    u32,
    &'static Size,
    &'static u32,
    &'static mut Size,
    &'static mut u32,
    impl Fn(&Size) -> Option<&u32>,
    impl Fn(&mut Size) -> Option<&mut u32>,
> {
    Kp{
        get: |x: &Size| Some(&x.width),
        set: |x: &mut Size| Some(&mut x.width),
        _p: std::marker::PhantomData
    }
}

#[test]
fn manual_keypath_then_read_write_works() {
    let mut rect = Rectangle {
        size: Size {
            width: 30,
            height: 50,
        },
        name: "MyRect".to_string(),
    };
    {
        let kp = rect_size_kp().then(size_width_kp());
        if let Some(res) = (kp).get(&rect) {}
        if let Some(res) = (kp).get(&rect) {}
    }

    let mutable_borrowed = &mut rect;
}

I immediately get a "too generic" feel from the code. But ignoring that, here's a workaround for the immediate problem.

+fn funnel<I, O, F: Fn(&I) -> Option<&O>>(f: F) -> F { f }
+fn funnel_mut<I, O, F: Fn(&mut I) -> Option<&mut O>>(f: F) -> F { f }

 // Manual keypath: Size -> width
 fn size_width_kp() -> Kp<
     Size,
     u32,
     &'static Size,
     &'static u32,
     &'static mut Size,
     &'static mut u32,
     impl Fn(&Size) -> Option<&u32>,
     impl Fn(&mut Size) -> Option<&mut u32>,
 > {
     Kp{
-        get: |x: &Size| Some(&x.width),
-        set: |x: &mut Size| Some(&mut x.width),
+        get: funnel(|x: &Size| Some(&x.width)),
+        set: funnel_mut(|x: &mut Size| Some(&mut x.width)),
         _p: std::marker::PhantomData
     }
 }

The compiler has problems inferring the signature of closures which are suppose to take any lifetime as an input and return the same lifetime in the output. If you have a bound that applies directly to the closure definition, like as an argument to funnel or funnel_mut, the bound will drive inference. So that's how the workaround functions.


In various places, I see you have bounds like

    Root: std::borrow::Borrow<R>,
    MutRoot: std::borrow::BorrowMut<R>,
    MutValue: std::borrow::BorrowMut<V>,
    G: Fn(Root) -> Option<Value>,
    S: Fn(MutRoot) -> Option<MutValue>,

If these bounds end up driving inference, they will do the opposite: they'll make the compiler try to make the closure have a signature which can only take and return an input with one[1] specific lifetime. This is because types that differ by lifetime are still distinct types, whereas generic type parameters like Root and Value can only represent one particular type. And thus cannot represent, say, "&'lt Size for any ilfetime 'lt".


  1. or zero ↩︎

Thanks for your help this works.

fn rect_size_kp() -> Kp<
    Rectangle,
    Size,
    &'static Rectangle,
    &'static Size,
    &'static mut Rectangle,
    &'static mut Size,
    impl Fn(&Rectangle) -> Option<&Size>,
    impl Fn(&mut Rectangle) -> Option<&mut Size>,
> {
    Kp::new(
        constrain_get(|x: &Rectangle| Some(&x.size)),
        constrain_set(|x: &mut Rectangle| Some(&mut x.size)),
    )
}

I had some time to poke at your code, so let me illustrate the kinds of issues that can arise from using type parameters instead of explicit references more concretely.

Make this additional change to your test code:

         let kp = rect_size_kp().then(size_width_kp());
-        if let Some(res) = (kp).get(&rect) {}
-        if let Some(res) = (kp).get(&rect) {}
+        if let Some(res) = (kp).get_mut(&mut rect) {}
+        if let Some(res) = (kp).get_mut(&mut rect) {}

And you will get an error:

error[E0499]: cannot borrow `rect` as mutable more than once at a time
   --> src/lib.rs:150:41
    |
149 |         if let Some(res) = (kp).get_mut(&mut rect) {}
    |                                         --------- first mutable borrow occurs here
150 |         if let Some(res) = (kp).get_mut(&mut rect) {}
    |                                         ^^^^^^^^^ second mutable borrow occurs here
151 |     }
    |     - first borrow might be used here, when `kp` is dropped and runs the destructor for type `Kp<Rectangle, u32, &Rectangle, &u32, &mut Rectangle, &mut u32, impl Fn(&Rectangle) -> Option<&u32>, impl Fn(&mut Rectangle) -> Option<&mut u32>>`

And even if we make the possibility of a destructor impossible, the error persists.


error[E0499]: cannot borrow `rect` as mutable more than once at a time
   --> src/lib.rs:150:41
    |
149 |         if let Some(res) = (kp).get_mut(&mut rect) {}
    |                                         --------- first mutable borrow occurs here
150 |         if let Some(res) = (kp).get_mut(&mut rect) {}
    |                                 ------- ^^^^^^^^^ second mutable borrow occurs here
    |                                 |
    |                                 first borrow later used by call

What's the problem? Here we have a

Kp<
    /* R */ Rectangle,
    /* V */ u32,
    /* Root */ &'a Rectangle,
    /* Value */ &'b u32,
    /* MutRoot */ &'c mut Rectangle,
    /* MutValue */ &'d mut u32,
    /* G */ impl Fn(&Rectangle) -> Option<&u32>,
    /* S */ impl Fn(&mut Rectangle) -> Option<&mut u32>,
>

for some inferred lifetimes 'a, 'b, 'c, 'd. They don't have to be 'static due to variance, but there can only be one lifetime per type parameter at a time, and every coercion establishes an outlive relation.

Then you call a

pub fn get_mut(&self, root: MutRoot) -> Option<MutValue> {
// =>
pub fn get_mut(&self, root: &'c mut Rectangle) -> Option<&'d mut u32> {

which can still work once as long as 'c: 'd is inferred. But when you try to call it again, you try to exclusively borrow the Rectangle for the same duration (lifetime), 'c. That would be two exclusive borrows for the same duration, which isn't allowed.[1]

You really want get_mut to look something like

pub fn get_mut<'any>(&self, root: &'any mut Rectangle) -> Option<&'any mut u32> {

where it can support passing through any lifetime. But type variables like MutRoot and MutValue can't represent "this type, but with any lifetime".


IMO, the most likely ways forward which avoid the above problem are one of

For the latter approach you use a trait to represent any sort of type constructor over a single lifetime:

pub trait Cons<'lifetime, Bound = &'lifetime Self> {
    type Ty;
    type MutTy;
}

Use marker types to implement this for the kinds of type constructors you want, like references for example:

pub struct References<T: ?Sized>(std::marker::PhantomData<fn() -> std::marker::PhantomData<T>>);
impl<'lifetime, T: ?Sized> Cons<'lifetime> for References<T> {
    type Ty = &'lifetime T;
    type MutTy = &'lifetime mut T;
}

And now References<Rectangle> can represent "&'lt Rectangle for any lifetime 'lt" in a sense. Due to how the trait solver works, you may also need some indirection around the Fn traits.

// Implement these for everything that meets the bounds
pub trait Convert<'lt, I, O, Bound = (&'lt I, &'lt O)>
where
    I: Cons<'lt> + ?Sized + 'lt,
    O: Cons<'lt> + ?Sized + 'lt,
    Self: Fn(I::Ty) -> Option<O::Ty>,
{}

pub trait ConvertMut<'lt, I, O, Bound = (&'lt I, &'lt O)>
where
    I: Cons<'lt> + ?Sized + 'lt,
    O: Cons<'lt> + ?Sized + 'lt,
    Self: Fn(I::MutTy) -> Option<O::MutTy>,
{}

Then you end up with an API like

impl<Root, Value, G, S> Kp<Root, Value, G, S>
where
    Root: ?Sized + for<'a> Cons<'a>,
    Value: ?Sized + for<'a> Cons<'a>,
    G: for<'a> Convert<'a, Root, Value>,
    S: for<'a> ConvertMut<'a, Root, Value>,
{
    pub fn get_mut<'r>(
        &self,
        root: <Root as Cons<'r>>::MutTy,
    ) -> Option<<Value as Cons<'r>>::MutTy> {
        (self.set)(root)
    }
}

And use Kp parameters like

pub fn rect_size_kp() -> Kp<
    References<Rectangle>,
    References<Size>,
    impl Fn(&Rectangle) -> Option<&Size>,
    impl Fn(&mut Rectangle) -> Option<&mut Size>,
> { ... }

(There may be more ideal ways to distribute the associated types and/or parameters of Kp, etc.)


  1. If you moved kp between the borrows to try to coerce new lifetimes, the first borrow is still forced to be active wherever the second borrow is active due to the outlives relations, and you still get a borrow check error. ↩︎