Syntax question for implementing a trait that returns a lifetime

#1

I have a trait MessageType that defines a function that returns an enum with an explicit lifetime, as below:

enum Value<'a> {
    String(&'a str),
}

trait MessageType {
    fn get_value(&self) -> Value<'a>;
}

struct FileCapabilities<'a> {
    value: Value<'a>,
}
impl<'a> MessageType for FileCapabilities<'a> {
    fn get_value(&self) -> Value<'a> {
        self.value
    }
}

fn trait_object<'a>() -> Box<dyn MessageType + 'a> {
    Box::new(FileCapabilities {
        value: Value::String("hello world"),
    })
}

fn main() {
    let t = trait_object();
    let Value::String(val) = t.get_value();
    println!("value contained: {}", val);
}

The problem being, I get the obvious:

error[E0261]: use of undeclared lifetime name `'a`
 --> src/main.rs:6:41
  |
6 |     fn get_value(&self) -> Option<Value<'a>>;
  |                                         ^^ undeclared lifetime

error: aborting due to previous error

For more information about this error, try `rustc --explain E0261`.

Link to rust playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e910cea0ea2ba5c5a084437d679d6198

0 Likes

#2

Would this work?

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5ef90951602b64967d5559804bbdc03b


If you don’t want Value: Copy

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c8adb1caa4d5cb2f109ff229635851a5


note: Avoid explicit lifetimes as much as you can. If you are only going to use Value::String with string literals, then you could just define Value like so

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f80cb455467f6eab33feb39dd2852e54

This will make it easier to use.

0 Likes

#3

Thanks for the fast reply!

I’m afraid my actual non-simplified code needs to have an enum variant along the lines of Value::Array(&'a [Value<'a>]) so I don’t think it would work to only use 'static.

I definitely do want Value: Copy so it seems like the first one is the way to go.

Just to be sure: Is the only solution here really to change Box<dyn MessageType<'a>> to Box<dyn MessageType<'static>>?

0 Likes

#4

It is, when you borrow from no input (since in that case the only thing you are allowed to borrow are statics). And that is the case in the simplified example you provided.

Maybe with the code for Value::Array(&'a [Value<'a>]) you do take/borrow an input with a lifetime parameter 'a that maybe you can use in return position (impossible to say before seeing actual code).

0 Likes

#5

Note that you can create slice literals rhat have a static lifetime.

let x: &'static [u32] = &[0, 1, 2];

This should work as long as the slice you are trying to create is a const expr. So only literals and const fns when creating the slice.

0 Likes

#6

Ultimately, I’m writing a crate where I want to return a struct like the following:

pub struct FitFile {
    records: Vec<Box<dyn MessageType>>,
}

So I’m not sure what the downside of introducing some upstream 'static lifetimes would be.

Currently, my Value uses String and Vec like so, heavily inspired by serde-json:

#[derive(Clone)]
pub enum Value {
    String(String),
    U8(u8),
    U16(u16),
    ...
    Array(Vec<Value>),
}

The Values are instantiated in a process that could yield either a single value, i.e. a u8 => Value::U8(u8) or an f64 or a String etc, or several i.e. [u8, u8, u8, u8]. Hence the requirement for Value::Array(Vec<Value>):

if number_of_values == 1 {
    let val = read_value_from_file().map(|val| val.into())
    Some(val)
} else {
    let mut v = Vec::with_capacity(number_of_values);
    read_value_from_file().map(|val| v.push(val.into()))
    Some(v.into())
}

The problem I’ve discovered is that even when only approximately 15 out of 10,000 Values are the ::Array variant (the vecs never being larger than 12 items), the runtime is ~10ms but when none of them are, the runtime is ~50us. I’d guessed that it was related to Value not implementing Copy and tried to get around that, and soon ended up in lifetime hell.

Am I missing something obvious?

0 Likes

#7

That won’t be because Value doesn’t implement Copy. The only thing Copy does is allow you to move a value multiple times. The exact same machine code is produced to move a value regardless of if it is Copy or not.

It may be because you have to go to the heap when using a Vec, and that could slow things down.


does read_value_from_file return an Iterator? If so, then

read_value_from_file().map(|val| v.push(val.into()))

does nothing. Change the map to for_each to fix it.

0 Likes

#8

Whenever you write an explicit lifetime, it’s a good time to stop and ask: what is being borrowed, and from whom?

Your trait has this method:

    fn get_value(&self) -> Value<'a>;

Because 'a is not mentioned anywhere else, this (if it were legal) is essentially saying “I can return a Value containing stuff borrowed from something with an arbitrary lifetime, which you can’t see.” The caller can’t do anything useful with that, because (for all it knows) the lifetime could be ending immediately.

Your implementation of the trait suggests that Value will contain data borrowed from self. In that case, you probably want

    fn get_value(&self) -> Value;

which, by lifetime inference, is equivalent to

    fn get_value<'s>(&'s self) -> Value<'s>;

…meaning, “The Value I return may contain data borrowed from self and so cannot outlive self.”

0 Likes

#9

Because we now have '_ t signal inferred lifetimes, it would be preferable to start teaching that syntax. I will make code more clear as there are no longer any hidden lifetimes.

fn get_value(&self) -> Value<'_>;
0 Likes

#10

I use Vecs elsewhere in the code so that probably isn’t the problem.

Apologies, I missed a line when I copied the code, it should be

for _ in (0..number_of_values) {
    read_value_from_file().map(|val| v.push(val.into()))
}

The interesting thing is if I replace that with the following which still reads the sequence but just doesn’t instantiate the vec, then the runtime is ~2000x faster

(0..number_of_values)
     .filter_map(|_| read_value_from_file())
     .nth(0)
     .map(|v| v.into())

What I was ultimately trying to express with the lifetimes was that Value should contain data borrowed from the scope where the Value was instantiated, although now I come to write this out I realise it’s impossible.

So, if lifetimes aren’t the answer, I guess I’m back to square one. Is there some special mechanics for enums that contain vecs that might explain why the difference would be so huge?

0 Likes

#11

You didn’t answer my question. I asked because Rust iterators are lazy, so they won’t do anything on their own.

for example, this code won’t print anything.

fn main() {
    (0..100).map(|x| println!(x));
}

In order to start iterating, you need to use one of the many functions in the Iterator trait (or others like it) that actually go through and iterate over the elements.

for example, this will print out all of the numbers from 0 to 99 inclusive.

fn main() {
    (0..100).for_each(|x| println!(x));
}

When reading the docs for the Iterator trait, if you see “creates an iterator” in the function docs, then that function does no iteration. It will not go through any of the elements of the input iterator.


Now looking at the code, looks like read_value_from_file returns an Option, so you could make it more clear by writing the following

for _ in (0..number_of_values) {
    if let Some(val) = read_value_from_file() {
        v.push(val.into())
    }
}

Or better yet, use the functionality in std

v.extend(
    (0..number_of_values)
        .filter_map(|_| read_value_from_file())
        .map(Into::into)
);

This is because you aren’t doing anything with the vec in this example. Your code here is the same as the following.

read_value_from_file().map(|v| v.into())
0 Likes

#12

No, there aren’t any special mechanics for enums that for Vec. There is a space saving layout optimization that uses the fact that a Vec's ptr is never null, but that won’t change your performance, and you can’t turn it off.

0 Likes

#13

Sorry, I thought I was answering your question by explaining that since it takes place in a for loop, there was no iterator. I used read_value_from_file as a placeholder, but the real function is a closure like so:

    let mut read_single_value = || {
        read_raw_u8()
            .ok()
            .map(|v| v.into())
            .filter(|v| v != &typ.invalidvalue)
    };

Where typ.invalidvalue is something like Value::U8(0xFF)

I’m quite definitely actually performing the reads.

0 Likes

#14

I’m quite willing to accept you’re right and that I’m not actually doing anything in my last “#nth(0)” example, but isn’t that still significant? Skipping ~15 instantiations of Value::Array(Vec) out of ~10000 instantiations of Value::anything_else speeds runtime up 2000x.

0 Likes

#15

For what it’s worth, I tried to do a little sandboxed benchmark. For an enum

enum Value {
    Array(Vec<Value>),
    String(String),
    U8(u8),
}

Using the same trait object in my original post, gives these approximate numbers:

pub fn run() {
    let i = 10000;
    let mut val_v: Vec<Value> = Vec::with_capacity(i);
    let mut box_v: Vec<Box<dyn MessageType>> = Vec::with_capacity(i);
    for _ in 0..i {
        // create raw enums
        // let val = Value::U8(5); // 32ns
        // let val = Value::String("hello world".to_owned()); // 29ns
        // let val = Value::Array(vec![Value::String("hello world".to_owned())]); // 302us
        //
        // create boxed trait objects
        // let val = prim_object(); // 31ns
        // let val = str_object(); // 195us
        // let val = arr_object(); // 415us
        //
        // push raw enums to vec
        // val_v.push(Value::U8(5)); // 77ns
        // val_v.push(Value::String("hello world".to_owned())); // 519us
        // val_v.push(Value::Array(vec![Value::String("hello world".to_owned())])); // 1ms
        //
        // push boxed trait objects to vec
        // box_v.push(prim_object()); // 507us
        // box_v.push(str_object()); // 1ms
        // box_v.push(arr_object()); // 1.5ms
    }
}
0 Likes

#16

I very sheepishly have to own up to my mistake and admit that the explanation for the ~2000x speedup was in fact because the program was panicking early and criterion was happily looping onwards regardless.

Sorry for wasting everyone’s time! :slight_smile:

0 Likes