Pointer wrapper and callback

1

I'm trying to implement a Rust binding for a C library but now fall into a problem of pointer wrapping with callbacks.

First, to wrap the existing pointer of the library, I've used:

struct PtrWrapper<'a> {
    ptr: u64,  // raw pointer
    callback: Callback<'a>, // callback which will use the raw pointer
}

where Callback is parametrized by lifetime 'a

struct Callback<'a> {
    pw: &'a PtrWrapper<'a>,
    callback: Box<Fn(&'a PtrWrapper<'a>)>,
}

for which I want to mean the callback inside would be called only by passing a pointer with lifetime the same as pw (I think this design for safety: one can only pass a living pointer to the callback).

Until now, there isn't any problem yet. But when I implement a function update_callback for PtrWrapper as:

impl<'a> PtrWrapper<'a> {
    pub fn update_callback(&mut self, F: impl Fn(&'a PtrWrapper<'a>)) {
        let cb = Callback {
            pw: self,
            callback: Box::new(F),
        };

        self.callback = cb;
    }
}

the compiler is not happy, I stuck with the error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
  --> src/main.rs:13:27
   |
13 |         let cb = Callback {
   |                  ^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 12:5...
  --> src/main.rs:12:5
   |
12 | /     pub fn update_callback(&mut self, F: impl Fn(&'a PtrWrapper<'a>, u64)) {
13 | |         let cb = Callback {
14 | |             pw: self,
15 | |             callback: Box::new(F),
...  |
18 | |         self.callback = cb;
19 | |     }
   | |_____^
note: ...so that reference does not outlive borrowed content
  --> src/main.rs:14:17
   |
14 |             pw: self,
   |                 ^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 11:6...
  --> src/main.rs:11:6
   |
11 | impl<'a> PtrWrapper<'a> {
   |      ^^
   = note: ...so that the expression is assignable:
           expected &PtrWrapper<'_>
              found &PtrWrapper<'a>

error: aborting due to previous error

For more information about this error, try `rustc --explain E0495`.

I guess that there is a conflict between the lifetime 'a used in pub fn update_callback(&mut self, F: impl Fn(&'a PtrWrapper<'a>)) and the lifetime of self, but still do not know how to fix.

Moreover,

rustc --explain E0495
error: no extended information for E0495

Many thanks for any help.

2

That pw: self looks suspicious. In Rust it's not possible to have a struct that borrows a reference to itself.

edit:

It is actually unsafe. I'll explain it on simplified cases:

struct Person {
   full_name: String,
   last_name: &str,
}

The borrow checker performs checking on fields individually. If it allowed full_name to be replaced or modified, it'd leave last_name pointing to the old value.

If these were two separate variables, it'd be solved by marking full_name as immutable for as long as references to it exist. But for a struct that creates a problematic situation where the entire struct has to be immutable and umovable.

struct Foo {
    num: u32,
    num_borrow: &u32,
}

Instance of the Foo struct is going to live at some address in memory (e.g. on stack when it's assigned to a local variable). When you take a reference to one of its fields, that field is relative to that address, too. However, when you return the whole struct from the function, or insert it into a Vec, Box, etc. it will be copied/moved to the new location. The address of the whole struct will change, and the borrow inside of that struct would end up pointing to the old location.

Because references are trivially copyable, they can end up in lots of places, and keeping track of all of them and updating them when their target changes is out of the question. Rust deliberately chose to have ability to move values in memory without having to run code to fix after the fact. It simplifies some things, but makes self-referential structs unsafe.

2 Likes
3

Ah, thank you. I didn't know that

But I still do not understand why, could you explain this?.

BTW, I come to a simpler implementation

struct PtrWrapper<'a> {
    ptr: u64,
    callback: Box<Fn(&'a PtrWrapper<'a>, u64)>,
}

impl<'a> PtrWrapper<'a> {
    pub fn update_callback(&mut self, F: impl Fn(&'a PtrWrapper<'a>, u64) + 'static) {
        self.callback = Box::new(F);
    }
}
4

When PtrWrapper is moved in memory (e.g. passed into a function or returned from one), the reference inside of it would still be pointing to the old, now invalid, location.

2 Likes
5

Somewhat related: rust - Why can't I store a value and a reference to that value in the same struct? - Stack Overflow

2 Likes
6

Thanks you all for the responses.

I understand better now why a field of a struct cannot borrow a reference to the struct itself. But now I fall into another weird behaviour.

In fact, I've reimplemented PtrWrapper without borrowing itself:

struct PtrWrapper<'a> {
    ptr: u64,
    callback: Box<Fn(&'a PtrWrapper<'a>)>,
}

impl<'a> PtrWrapper<'a> {
    pub fn from(ptr: u64) -> Self {
        let cb = move |pw: &PtrWrapper| {
            println!("raw pointer: 0x{:x}", pw.ptr);
        };
        PtrWrapper {
            ptr: ptr,
            callback: Box::new(cb),
        }
    }

    pub fn run_callback(&'a mut self) {
        (self.callback)(self)
    }
}

fn main() {
    let mut pw = PtrWrapper::from(0);
    // pw.run_callback();
}

While doing

pub fn run_callback(&'a mut self) {
    (self.callback)(self)
}

I would like to set that the run_callback is valid only if the lifetime of the reference used in the callback is at least self.

The code compiles fine, but when the last line pw.run_callback() is decommented, the compiler complains:

error[E0597]: `pw` does not live long enough
  --> src/main.rs:29:5
   |
29 |     pw.run_callback();
   |     ^^ borrowed value does not live long enough
30 | }
   | -
   | |
   | `pw` dropped here while still borrowed
   | borrow might be used here, when `pw` is dropped and runs the destructor for type `PtrWrapper<'_>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0597`.

I still don't understand the line:

30 | }
   | -
   | |
   | `pw` dropped here while still borrowed
   | borrow might be used here, when `pw` is dropped and runs the destructor for type `PtrWrapper<'_>`

pw is dropped, yes, but what does the next line mean? Why is borrow might be used here? Which borrows pw at this line?

Many thanks for any help.

7

Hi, I got it working like that, if I understand the error correctly, you tied PtrWrapper and PtrWrapper.callback lifetimes together, the issue is that rust drops the fields of a struct first, and you can use them in the drop implementation (that's the last sentence). I don't know if my explanation is really good, someone will probably explain it more clearly.

1 Like
8

@leudz Thanks for the explanation, but I still don't understand (sorry).

If this relates to the order of drop of fields and the struct itself, then why the line pw.run_callback() is commented, the code compiles just fine?

9

If you don't call pw.run_callback(), there is just no borrow so the compiler doesn't complain. I tried to explain it in more detail (playground). I don't know if it's 100% accurate, lifetimes can be tricky sometimes.
Also, you can remove the "mut" in run_callback, I just saw it.

1 Like
10

Wow, thanks you for the response.

I think I may understand the problem better. If I understand you correctly, that's because of in

(pw.callback)(&pw);

there is always one thing that will be dropped earlier (either pw.callback or pw itself, we just don't care), but the code will not compile except they have exactly the same lifetime.

So in general, it's exactly the problem: a field of struct cannot borrow the struct itself.