Lifetime of opaque data structures?

Hello, I am trying to rewrite a RIFF crate just as an exercise. The entire code is here: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=893be8172f87b23f74002cb5235b256a

Am I doing something too crazy here? I think this kind of thing is normal when writing file formats. The problem I am having is the implementation of the From trait for ChunkContents. I mean, it should be possible to create an opaque type that must exist within the lifetime of its data, yes?

edit: updated the code

In methods taking &self, any unspecified lifetime in the return type is considered to be the lifetime of the self reference. In your case, that is incorrect, so you need to explicitly tag the result with the data’s lifetime ’a:

    pub fn child_iter(&self) -> ChunkIter<'a> {
        ChunkIter {
            cursor: self.pos + 12,
            end: self.len,
            data: self.data,
        }
    }

( see playground for full code)

2 Likes

I ended up using std::ptr altogether lol
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=3b461538d2125b40dbd58cdcec9635d5
It compiles^(tm) but I am not sure if it werks, writing some tests now...

@2e71828 but that's exactly what I want tho? There is only a single underlying owning Vec so I want all of them to have the same lifetime? Obviously Rust is correct here (it compiles) so I guess I don't understand my own program?

There’s the Vec that contains the audio data, but also the Chunk struct that contains metadata. Your original code unintentionally said that the returned ChunkIter contained a reference to the Chunk struct itself, and not just the underlying Vec.

The compiler then complained that you dropped the original struct when you still had somehing referencing it.

It doesn't look like you're doing anything here that should need unsafe code, so I'd suggest avoiding doing so if your goal is to learn rust. There are several issues, but I think you highlighted the key above when you pointed to the From implementation.

Right here is a bug, in the From implementation:

impl<'a> From<Chunk> for ChunkContents<'a> {

You can't create something with a lifetime without borrowing from something else, and the from method consumes its argument. So either you need to add a lifetime to Chunk (so it doesn't own its own data), or you need to make ChunkContents own its own data, or you need to not implement From like this.

/// A chunk, also known as a form.
/// Note that `Chunk` is an opaque type.
/// To obtain the actual chunk.data, chunk.convert this into a `ChunkContents`.
#[derive(Debug, Clone, PartialEq)]
pub struct Chunk {
    pos: u32,
    len: u32,
    data: *const u8,
}

This looks an awful lot like a slice. Why not use a slice?

1 Like

@droundy Indeed, I knew that I am not being explicit/wrong about my lifetime and the compiler can't infer it. After following @2e71828's suggestions, I was able to finally implement it without a single unsafe!

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=9b8375287f31a8921d072f244aa2f7a4

Yep, lol that's what I thought too. I was basically trying to circumvent through the borrow-checker xD

Excellent! :smiley:

Hello everyone, still don't fully grasp lifetime yet...
I have come across a different problem now.
Instead of reading the entire content of file into resident memory, I want this to instead lazily read into the file as required by the user.
In particular, its complaining that

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'b in function call due to conflicting requirements
  --> src/riff_disk.rs:201:25
   |
201 |             let chunk = Chunk::from_reader(self.reader, self.payload_cursor);
   |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 197:5...
  --> src/riff_disk.rs:197:5
   |
197 | /     fn next(&mut self) -> Option<Self::Item> {
198 | |         if self.payload_cursor >= self.payload_end {
199 | |             None
200 | |         } else {
...   |
205 | |         }
206 | |     }
   | |_____^
note: ...so that reference does not outlive borrowed content
  --> src/riff_disk.rs:201:44
   |
201 |             let chunk = Chunk::from_reader(self.reader, self.payload_cursor);
   |                                            ^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 191:6...
  --> src/riff_disk.rs:191:6
   |
191 | impl<'a, R> Iterator for ChunkIterNoType<'a, R>
   |      ^^
note: ...so that the types are compatible
  --> src/riff_disk.rs:197:46
   |
197 |       fn next(&mut self) -> Option<Self::Item> {
   |  ______________________________________________^
198 | |         if self.payload_cursor >= self.payload_end {
199 | |             None
200 | |         } else {
...   |
205 | |         }
206 | |     }
   | |_____^
   = note: expected `std::iter::Iterator`
              found `std::iter::Iterator`

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0495`.
error: could not compile `riff`.

To learn more, run the command again with --verbose.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=78c0f8ce7c9d633dc95d3366ca280bb8

I think the clue is the anonymous lifetime...as far as I understood it, my calling to Chunk::from_reader actually uses the anonymous lifetime instead of 'a?

I can't test it right now, but I think the problem is that the lifetime originates from the &mut self parameter, which does not have anything to do with 'a. Annotating it like &'a mut self may clear the error, by requiring that self and the return value are both valid for 'a, instead of just the return value as it is now.

Hmm, you cannot annotate borrow expressions

error: borrow expressions cannot be annotated with lifetimes
   --> src/riff_disk.rs:201:43
    |
201 |             let chunk= Chunk::from_reader(&'a mut self.reader, self.payload_cursor);
    |                                           ^--^^^^^^^^^^^^^^^^
    |                                            |
    |                                            annotated with lifetime here
    |                                            help: remove the lifetime annotation

Ended up wrapping the Read + Seek inside Rc.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0ad095efce9f9ce9c0d3d796fa393760

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.