Returning data from ephemeral object

help
1

I am trying to return reference data from an ephemeral object (data lives longer).

To simplify code, let's say a function builds an object containing a slice. This object will be used, then at some point I'd like to return the reference while the object will be dropped.
The rust compiler complains that the ephemeral object is borrowed and that I can't return data.

The simplified code looks like:

pub struct Bytes<'a>(&'a [u8]);
pub struct Owned<'a>(&'a [u8]);

fn my_function<'a>(input: &'a[u8]) -> Owned<'a> {
  let any = Any { data: Bytes(input) };

  // this won't work: `any.data` is borrowed
  Owned(any.data.as_bytes())

  // however, this works
  // Owned(any.data.0)
}

Looking at the code above, it seems using any.data.0 is a proper workaround. However, this does not work if there is another intermediate function, for ex:

fn build_owned<'a>(any: &'a Any<'a>) -> Owned<'a> {
  my_function(any.data)
}

This typically happens because I'd like to be able to either call the function on an existing Any object (passed by reference), or build an ephemeral one.

Here is a more complete example: playground

Notes: I cannot add functions like .into_bytes() because of the use case where I need to call the function on an existing &Any object.

I do understand that this may have no solution at the moment, so I am looking for a possible solution, or hints for a better design.
Thank you!

2

Using 'a for both the wrapped reference and the container makes the lifetime shrink to the lifetime of the outer reference to Any. The outer reference is too short-lived as Any is a local variable of your build_owned_consume[2] functions. Use two distinct lifetimes here:

pub struct Bytes<'a>(&'a [u8]);

impl<'a> Bytes<'a> {
    pub fn as_bytes<'slf>(&'slf self) -> &'a [u8] {
        self.0
    }
}

pub struct Any<'a> {
    data: Bytes<'a>,
}

struct Owned<'a>(&'a [u8]);

fn return_content<'any, 'content>(any: &'any Any<'content>) -> &'content [u8] {
    any.data.as_bytes()
}

fn build_owned<'any, 'content>(any: &'any Any<'content>) -> Owned<'content> {
    let s = return_content(any);
    Owned(s)
}

fn build_owned_consume<'a>(input: &'a [u8]) -> Owned<'a> {
    let any = Any { data: Bytes(input) };

    // returning data from object fails:
    // error[E0515]: cannot return value referencing local data `any.data`
    //   --> src/main.rs:32:5
    //    |
    // 37 |     Owned(any.data.as_bytes())
    //    |     ^^^^^^--------^^^^^^^^^^^^
    //    |     |     |
    //    |     |     `any.data` is borrowed here
    //    |     returns a value referencing data owned by the current function
    Owned(any.data.as_bytes())

    // This works
    // Owned(any.data.0)
}

fn build_owned_consume2<'a>(input: &'a [u8]) -> Owned<'a> {
    let any = Any { data: Bytes(input) };
    // for the same reason, this also fails
    // however, here we cannot use `data.0` directly
    build_owned(&any)
}

fn main() {}

Playground.

Elided version.

1 Like
3

Thank your for your quick reply!
It seems so obvious with the explanation. That said, I struggled a bit to understand why this does not work in the production code, because I do use different lifetimes.

It seems the problem comes from the .as_bytes() declaration. In my code, I use the AsBytes trait from nom, where all lifetimes are inferred.
When implementing the trait (even with a similar definition), it seems the lifetime is inferred to be the same, causing the problem.

Playground example: playground

So, I am now trying to understand why similar definitions gives a different result:

  • works with a plain function (tested in production code)
  • does not work when implementing trait with the exact same function declaration
4

The problem is AsBytes' definition:

pub trait AsBytes {
    fn as_bytes(&self) -> &[u8];
}

which can be expanded to:

pub trait AsBytes {
    fn as_bytes<'a>(&'a self) -> &'a [u8];
}

It is missing a generic lifetime parameter to allow as_bytes to return a reference that lives longer than the reference to self:

use std::borrow::Cow;
// use nom::AsBytes;

pub struct Bytes<'a>(&'a [u8]);

// Without using trait, this works:

/*
impl<'a> Bytes<'a> {
    pub fn as_bytes(&self) -> &'a [u8] {
       self.0
    }
}
*/
trait AsBytes<'a> {
    fn as_bytes(&self) -> &'a [u8];
}

// This does not:
impl<'a> AsBytes<'a> for Bytes<'a> {
    fn as_bytes(&self) -> &'a [u8] {
        self.0
    }
}

pub struct Any<'a> {
    data: Bytes<'a>,
}

struct Owned<'a>(Cow<'a, [u8]>);

pub struct Error;

impl<'a> TryFrom<Any<'a>> for Owned<'a> {
    type Error = Error;

    fn try_from(any: Any<'a>) -> Result<Owned<'a>, Self::Error> {
        TryFrom::try_from(&any)
    }
}

impl<'a, 'b> TryFrom<&'b Any<'a>> for Owned<'a> {
    type Error = Error;

    fn try_from(any: &'b Any<'a>) -> Result<Owned<'a>, Self::Error> {
        Ok(Owned(Cow::Borrowed(any.data.as_bytes())))
    }
}

fn main() {}

Playground.

Unfortunately there is no way that you can return a longer lived reference from nom::AsBytes::as_bytes. The only workaround I can think of is returning actually owned data from TryFrom:

impl<'a, 'b> TryFrom<&'b Any<'a>> for Owned<'a> {
    type Error = Error;

    fn try_from(any: &'b Any<'a>) -> Result<Owned<'a>, Self::Error> {
        Ok(Owned(any.data.as_bytes().to_owned().into()))
    }
}

Playground.

1 Like
5

That is also my understanding, I'll have a look at nom to see if I can update AsBytes and send a PR. What is not yet clear to me is if such change can cause problems else in existing code.

note: I also tried the standard AsRef trait, and unfortunately it seems to end up with the same problem

In the meantime, I am using a method where the lifetimes work, which allows me to keep parsing zero-copy.
I marked your first reply as solution. Thanks again!

1 Like
6

In short, this signature:

    fn as_bytes(&self) -> &[u8];
    // aka
    fn as_bytes<'this>(&'this self) -> &'this [u8];

requires that *self remains borrowed so long as the returned borrowed slice is in use, as if the slice of bytes were directly owned by *self (even though this is not the case for your Bytes<'_>). This comes up with a number of std traits too, such as AsRef and Borrow.

Mid-post addendum: As I guess you just discovered, heh.

There's no way to make the trait more flexible that isn't a (SemVer) breaking change AFAIK.

1 Like