[solved] Difficulties with correct lifetime in library bindings


#1

Hi,

I am trying to create some bindings for the ExprTk library. This seems to be working fine, however I’m not sure about the best way to make the API “Rust compatible”. I have to ensure that values registered to the symbol table will outlive the table they are referenced by. This is a simplified example of what I did (Playground, leaving out all the ffi stuff):

use std::marker::PhantomData;
use std::cell::Cell;

struct SymbolTable<'a>(PhantomData<&'a ()>);

impl<'a> SymbolTable<'a> {
    fn new() -> SymbolTable<'a> {
        SymbolTable(PhantomData)
    }
    
    fn add_variable(&mut self, name: &str, value: &'a Cell<f64>) {
        // some unsafe code
    }
}


struct Expression<'a>(PhantomData<&'a ()>);

impl<'a> Expression<'a> {
    fn new() -> Expression<'a> {
        Expression(PhantomData)
    }
    
    fn register_symbol_table(&mut self, t: &'a SymbolTable<'a>) {
        // some unsafe code
    }
    
    fn value(&self) -> f64 {
        // calculate...
        0.
    }
}


fn main() {
    let mut values = vec![];
    let mut t = SymbolTable::new();
    
    values.push(Cell::new(5.));
    
    t.add_variable("x", &values[0]);
    
    let mut e1 = Expression::new();
    e1.register_symbol_table(&t);
    
    let mut e2 = Expression::new();
    e2.register_symbol_table(&t);
    
    println!("{:?}", e1.value());

}

Code like this will not compile:

    let mut t = SymbolTable::new();
    {
        let v = Cell::new(5.);
        t.add_variable("x", &v);
    }

(actually, the additional scope is not needed because the value is created after the table, it would fail anyway)

However, in my application using the ExprTk library, I would like to add variables to a symbol table AFTER it was registered to some expression. I assume that this should be perfectly safe (in contrast to removing variables), as adding them does not interfere with the existing ones. However, this only works if add_variable does not borrow self as mutable. So, I turned &mut self into &self (Playpen).

    fn add_variable(&self, name: &str, value: &'a Cell<f64>) {
        // some unsafe code
    }

However with this modification, the above “errorneous” code suddenly compiles (Playpen).

Is there a way to enforce the correct lifetime, if self is borrowed immutably, or do I have to step back from adding variables to the symbol table after it was linked to some expression?

In general, I hope that this approach makes sense, I am of course open to better suggestions.


#2

Yeah, that’s weird. I don’t understand why it compiles. Even if I change it to:

fn add_variable<'x>(&'x self, name: &'static str, value: &'a Cell<f64>) where 'a:'x, 'x:'a {

then it looks like borrow checker thinks that scope of SymbolTable and Cell is the same (but in MIR output they clearly are not the same). Maybe it’s a bug?


#3

What you’re running into is variance. Here’s a couple links that might be useful:

https://doc.rust-lang.org/nomicon/subtyping.html

Basically, if you have an &'a T, you can pass it in place of an &'b T as long as 'a outlives 'b. Why not? It lives more than long enough, so the callee is guaranteed not to get a dangling pointer. However, if you have an &'a mut &'b T, you cannot pass it to a parameter v: &'c mut &'d T unless 'b and 'd are identical and 'a outlives 'c. If 'b doesn’t live as long as 'd, the callee could store a copy of *v somewhere and assume that copy lives as long as 'd, and that would become a dangling pointer. And if 'd doesn’t live as long as 'b, the callee could assign an &'d T to *v which the caller would then assume lives as long as 'b, again creating a dangling pointer. So, once you introduce mutability, the compiler enforces invariance.

The problem is that you’re using unsafe code to create interior mutability. The compiler can’t reason about this unless you tell it you’re doing this via an UnsafeCell. Here’s an example which fails because the compiler now knows your code needs to be invariant over the lifetime parameter of SymbolTable.

https://play.rust-lang.org/?gist=f64aee2a66688da831cd66bc9f4b5e18&version=stable&backtrace=0


#4

@stevenblenkinsop Thanks a lot for your explanation and the link to this nice talk. UnsafeCell indeed fixes the situation and makes the code work as expected, so I marked the thread as solved.

I’m still not sure whether this can be considered idiomatic Rust code. There is actually some modification happening when adding a variable to SymbolTable. Creating those library bindings seems to require more carful consideration than I thought in the beginning…