Need help storing a value with lifetime in a struct

One thing that have stuck me for a while. I know two values have a same lifetime, but how can I put them in one struct.

Let me explain using the following example. ZipFile and ZipArchive have a same lifetime, I wish to keep them in a single struct, but rust doesn't allow me to do such thing.

use zip::ZipArchive;
use std::fs;

struct Test<'z, R> {
    zip: ZipArchive<R>,
    file: zip::read::ZipFile<'z>
}

impl Test<'_, fs::File> {
    pub fn new(path: &str) -> Self {
        let f = fs::File::open(path).unwrap();
        let mut zip = ZipArchive::new(f).unwrap();
        let file = zip.by_index(0).unwrap();

        // I'm only interested in `file`,
        // but I have to keep `zip` in a struct
        // otherwise it will be cleaned by rust
        Test { zip, file }
        // cannot return value referencing local variable `zip`
        // returns a value referencing data owned by the current function
    }
}

Are there any proper ways to do it? any suggestion is appreciated.

I'm on mobile right now, so the formatting might be sub optimal.

I assume you have experience in garbage collected languages like python or java?

As in all programming languages objects or instances of structs must be located somewhere in memory. In garbage collected languages this is usually on the heap and the decision when the memory is not needed anymore is made when the program runs (also called "runtime"). This has the benefit to be "easier" as in many cases the programmer does not need to worry about releasing the memory.
In non garbage collected languages (like rust, c, etc) instances of structs are not allocated on the heap but on the stack (by default). Allocations on the stack have the benefit of being incredibly cheap but come with the cost of only being valid until the end of the function (each function has a "stackframe" which contains the allocations for variables in the function).
What the rust compiler is telling you, is that because zip is allocated on the stack -> it will only live until the end of the function -> not long enough for file which contains references to it.

For starters I would recommend cloning file if performance is not super crucial for you. Especially since your current solution (both fields inside one struct) will also not work (easily) because of the self referential nature (if this interests you I recommend searching for one of the many other threads about self referential structs on this forum :wink: ).

Edit: ZipFile doesn't seem to support cloning, but you should be able to read the (decompressed) contents into an owned Vec<u8> or String depending on your requirements.

Thanks @raidwas, after some searches about self referential posts, I realized it might not be the optimal design in rust.

But out of curiosity, I tried to mimic a working example but I still get error, I am guessing the example only used immutable ref but file requires &mut zip, is it correct?

I see someone said unsafe block can achieve my goal but there is not much of example code that I learn from...

// this example works
struct Look<'a> {
    n: i32,
    r: Option<&'a i32>,
}

impl<'a> Look<'a> {
    fn new(n: i32) -> Self {
        Look { n, r: None }
    }
    fn init(&'a mut self) -> &'a Self {
        self.r = Some(&self.n);
        self
    }
}

// but mine just desn't work at all
struct ZipBundle<'z, R> {
    zip: ZipArchive<R>,
    file: Option<zip::read::ZipFile<'z>>
}

impl<'z, R: BufRead + Seek> ZipBundle<'z, R> {
    pub fn new(stream: R) -> Self {
        ZipBundle { zip: ZipArchive::new(stream).unwrap(), file: None }
    }
    pub fn init(&'z mut self) -> &'z Self {
        self.file = Some(self.zip.by_index(0).unwrap());
        self
        // cannot borrow `*self` as immutable because it is also borrowed as mutable
        // immutable borrow occurs here
    }
}

Hey, very interesting thing in your first example.
I'm still on mobile, as such I can't try code out right now. I'm not too certain as to why the first one works and the second doesn't, but if I had to guess I would assume the second creates some temporary value (on the stack) which would still not live long enough?

A totally different approach you could take to keep going forward is to have a method that accepts a closure and the closure takes the ZipFile as a parameter.
The method would then first create the ZipFile and then call the closure with it. This should at least solve the issue you are facing.
I will definitely take a deeper look into this as well, but cannot promise that I have time for that soon.

Edit:
On second thought, i don't think it is due to a temporary on the stack, the error message of the compiler should look different in that case (I think).
Out of curiosity: did you actually manage to use the first version? I would imagine it is impossible to call the method since it would require a reference to a strict with the exact same lifetime as the strict itself, in other words: Look<'a> cannot exist outside of 'a, and the reference has to exist for at least 'a. I'm not sure if the compiler is able to merge "creating the strict" and "referencing it" into the same "instant" (the moment where 'a starts).

The problem here is that you're trying to create a self-referential struct, and doing so (with actual references) doesn't really work out in Rust [1]. For example here:

#[derive(Debug)]
struct Look<'a> {
    n: i32,
    r: Option<&'a i32>,
}

impl<'a> Look<'a> {
    fn new(n: i32) -> Self {
        Look { n, r: None }
    }
    fn init(&'a mut self) -> &'a Self {
        self.r = Some(&self.n);
        self
    }
}

You can manage to create the self-referential struct...

    // Works...
    let mut look = Look::new(0);
    // Works...
    look.init();

...but then you can't actually use it!

    // error[E0502]: cannot borrow `look` as immutable because it is also borrowed as mutable
    println!("{look:?}");

Why not? Because in order to initialize the r field, you needed a reference with a lifetime of length 'a; to get such a borrow out of self, you needed a &'a mut Self. So in order to call look.init(), you had to mutably -- or more accurately put, exclusively -- borrow look for the entirety of its remaining validity. Once you create that exclusive borrow, it is impossible to use look again, except through that borrow.

You can't print it. You can't move it, so you can't return it. You can't even run a destructor if you have one.


Why does it work this way? Well, consider what would happen if you could move look after you created it: you would move the thing that r is referencing, and r would dangle. (Rust does not have move constructors.) Something has to prevent that from happening.

You may say, well in my case the referenced value is on the heap, but Rust the language has no distinction between references to the heap or to the stack.

Not only that, but Rust's aliasing rules dictate that nothing can have exclusive access to the n field so long as there's a live reference that can observe it. So something has to prevent another &mut look from being created, no matter where the referenced value is in memory.

And there are probably other, similar reasons as well.


ZipFile has a Drop desctructor, so this is probably why you can't even construct your Test. If you don't need to gradually read the ZipFile, you could just store its index in the ZipArchive. If you do need to gradually read, what you probably want is Seek support, but apparently the plan is to have that for raw (compressed) data, and not for ZipFile. I didn't look into it, but maybe seeking requires uncompressing everything anyway.

This issue looks to mirror your use-case.


  1. outside of some extremely niche circumstances ↩ī¸Ž

4 Likes

@quinedot Thanks for such detailed explanation! I think I will drop zip support for my crate :joy:...

@raidwas As for the first example, it is not runnable, sorry for the incorrect information. I saw it from this post and tried only on rust playground.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.