Implementing FnMut with lifetime

help
1

Hello all.

I use third-party library which has struct Foo constructed from closure:

pub struct Foo<T> { t: T }

impl<'a, T> Foo<T> where T: 'a + FnMut() -> &'a [u8] {
    pub fn from_callback(closure: T) -> Self {
        …
    }
}

If I understand correctly, it means that returned slice lives as long as closure itself.

I can use it in my code:

fn main() {
  let buf = vec![0; 100];
  let foo = Foo::from_callback(|| { &buf });
}

but I want to return it from function. Since abstract return types aren't implemented yet (sadly), I can't use closure, so I have decided to use struct which implements FnMut:

#![feature(unboxed_closures)]

pub struct Func {
  buffer: Vec<u8>,
}

impl FnMut<()> for Func {
  extern "rust-call" fn call_mut(&mut self, _: ()) -> Self::Output {
    &self.buffer
  }
}

fn create_foo() -> Foo<Func> {
  Foo::from_callback(Func { buffer: vec![0; 100] })
}

fn main() {
  let foo = create_foo();
}

this gives me:

error: the trait `core::ops::FnOnce<()>` is not implemented for the type `Func` [E0277]

ok, lets implement FnOnce:

impl FnOnce<()> for Func {
  type Output = &[u8];
                ^~~~  error: missing lifetime specifier [E0106]
  extern "rust-call" fn call_once(self, _: ()) -> Self::Output {
    unimplemented!();
  }
}

Since buffer lifetime must depend on closure lifetime, I expected something like this to work:

impl<'a> FnOnce<()> for Func where Func: 'a {
     ^~~~~ error: the lifetime parameter `'a` is not constrained by the impl trait, self type, or predicates
  type Output = &'a [u8];
  extern "rust-call" fn call_once(self, _: ()) -> Self::Output {
    unimplemented!();
  }
}

but it doesn't.

Please help me to implement FnMut.

2

Func doesn't carry a borrowed slice, it's not really compatible with Foo. Make it capture one:

struct Func<'a> {
    s: &'a [u8],
}
3

Then how can I initialize Func?

fn create_foo() -> Foo<Func<'a>> {
    let buf = vec![0; 100];
    Foo::from_callback(Func { s: &buf })
}
  1. I don't know where to get 'a and
  2. buf will die at the end of function, but Func instance not
4

That's the problem with Foo, isn't it? How is a callback supposed to return a slice? It has to have captured it from somewhere with a guarantee that it outlives 'a.

5

As I understand, T: 'a + FnMut() -> &'a [u8] means «closure itself has lifetime 'a and returned buffer has same lifetime» which is perfectly possible if closure owns buffer and returns reference to it.

6

T: 'a means that no borrow inside T is shorter than that. A value of type T could have an arbitrarily shorter lifetime.

You can try implementing FnMut for &'a mut Func but that doesn't feel like the intention of Foo. It might even work without the mut.

7

If I implement FnMut for &'a Func, then reference to Func will be stored inside Foo, not Func itself, and this reference will be valid until the end of create_foo only.

8

I believe you'd hit the same basic ownership issue with real closures.

9

This code works:

fn use_foo<T>(_foo: Foo<T>) {
}

fn main() {
  let foo = {
    let buffer = vec![0; 100];
    Foo::from_callback(move || &buffer)
  };
  use_foo(foo);
}

Note that buffer is defined in block and lexically not available when use_foo is called, but it is moved to closure which is then passed to Foo.

10

Hmm. I'll have to think about this.. :wink:
BTW did you try the usual anonymous types workaround -- putting the closure in a Box<FnMut() -> &[u8]>?

This code works

Not on the nightly channel. Either it requires more annotations or is actually unsound.

11

But Box<FnMut…> doesn't implement FnMut.

12

If it's possible to alter Foo's API, it could be made more friendly to your use case like this (play):

impl<'a, T> Foo<T> where T: 'a, T: FnOnce() -> Cow<'a, [u8]> {
    pub fn from_callback(closure: T) -> Self {
        Foo { _t: closure }
    }
}

impl FnOnce<()> for Func {
  type Output = Cow<'static, [u8]>;
  extern "rust-call" fn call_once(self, _: ()) -> Self::Output {
    self.buffer.into_cow()
  }
}

fn create_foo() -> Foo<Func> {
  Foo::from_callback(Func { buffer: vec![0; 100] })
}

fn main() {
  let buf = vec![0; 100];
  let _foo1 = Foo::from_callback(|| { (&*buf).into_cow() });
  let _foo2 = create_foo();
}

Now you can choose between returning a captured slice like before (foo1) and a vector (foo2) that doesn't need an external owner.

13

Well, I happen to be the author of the API in question. Seems like I initially got some lifetime annotations wrong :slight_smile:

What I initially wanted to express and what solves this problem is achieved with the following trait and constructor:

trait Source {
    fn read_more(&mut self) -> &[u8];
}

impl<SourceT> Foo<Box<SourceT>>
    where SourceT: Source
{
    pub fn from_source(source: SourceT) -> Self {
        ...
    }
}

What happened, instead of annotating &mut self reference I ended up putting lifetime annotation somewhere it would seem to make sense and compile. That somewhere happened to be closure value. And unsoundness is (likely) hidden by unsafe code for interfacing with a C library.

Any chance to implement this trait for FnMut? This far I haven't found a way to express it, strangely — for reasons mostly syntactic: where do I actually put lifetime annotation?

impl<CallbackT> Source for CallbackT
    where CallbackT: FnMut() -> &[u8] // really, &'a [u8]. But 'a is introduced only later, in fn read_more()
{
    fn read_more(&mut self) -> &[u8] { self() }
}
14

I'm still not sure what the intention is. In my experience readers either return a vector or take a &mut reference to the buffer.

Anyway it doesn't seem possible for a closure to let you borrow a value that it owns.

Of course you can put the buffer and the closure into a struct and implement the whatever trait for that struct:

trait Source {
    fn read_more(&mut self) -> &[u8];
}

struct Func<F: FnMut(&[u8]) -> &[u8]> {
    buffer: Vec<u8>,
    func: F,
}

impl<F: FnMut(&[u8]) -> &[u8]> Func<F> {
    fn new(buffer: Vec<u8>, func: F) -> Self {
        Func { buffer: buffer, func: func }
    }
}

impl<F: FnMut(&[u8]) -> &[u8]> Source for Func<F> {
    fn read_more(&mut self) -> &[u8] {
        (self.func)(&self.buffer)
    }
}

fn main() {
  let mut foo = Func::new(vec![0; 100], |buf| buf);
  foo.read_more();
}
15

It seems it is possible: #![feature(unboxed_closures, core)]pub struct Foo<T> { t: T }impl<'a, T> - Pastebin.com

So the question is how closure in this code may be desugared into struct implementing FnMut.

16

Rust Playground doesn't agree:

<anon>:23:8: 23:22 error: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
<anon>:23       &buffer[j..j+1]
                 ^~~~~~~~~~~~~~

To spell this out for the example from above

impl<CallbackT> Source for CallbackT
where CallbackT: FnMut() -> &[u8] // really, &'a [u8]. But 'a is introduced only later, in fn read_more()
{
    fn read_more<'a>(&'a mut self) -> &'a[u8] { self() }
}

We'd need the return type to reference the parameter 'a from the method signature but it's the trait's associated type defined outside that signature, where there's no 'a, for better or for worse.

Higher-kinded types could probably remove this limitation (i.e. it wouldn't be a hit to ergonomics to let this trait have a lifetime parameter) but they're not available (yet?).

17

I'm still not sure what the intention is. In my experience readers either return a vector or take a &mut reference to the buffer.

That's slightly off-topic, but anyway: a lot of existing reader designs are not so perfect as they hide the existence of the buffer, or require additional copies. Consider (a) a reader that returns an already-existing memory region; (b) a reader that repeatedly fills the same memory buffer from a file descriptor. For your suggested interfaces:

  1. "Return an owned vector": that's an extra allocation per read AND an extra memory copy per read.
  2. "Take a &mut buffer reference": that's an extra memory copy per read and possible mis-coordination of buffer sizes.

TL;DR: this Source trait is actually a stripped-to-the-bone version of std::io::BufRead (without all unnecessary methods) — see fn fill_buf(&mut self) -> Result<&[u8]>;.