Enforcing Lifetimes in an FFI Situation

help
1

Consider the following C function:

void foreign_func(S1* ptr1, S2* d2) {
   ....
}

where S1 and S2 are some C structs. The contract of this function requires that ptr2 lives at least as long as long as ptr1. Let's say I've declared a safe wrapper function over this function like this:


fn my_wrapper(ref1: &S1, ref2: &S2) {
    foreign_func(mem::transmute(ref1), mem::transmute(ref2));
}

(Assuming I've properly declared structs S1 and S2 in my Rust code with#[repr(C)],) is there a way to enforce the above-stated contract using lifetime annotations on my_wrapper?

2 
fn my_wrapper<'a, 'b:'a>(ref1: &'a S1, ref2: &'b S2) {
    foreign_func(mem::transmute(ref1), mem::transmute(ref2));
}
3

Sure.

fn my_wrapper(ref1: &'a S1, ref2: &'b S2) where 'a : 'b {
    foreign_func(mem::transmute(ref1), mem::transmute(ref2));

This reads:

ref1 has some lifetime 'a and type &S1. ref2 has lifetime 'b and type &S2 where lifetime 'a lives at least as long as lifetime 'b (mnemonic a "extends" b FOR THE DURATION OF THE BORROW (which ends when the function returns).

EDIT: To clarify, adding this "where" restriction accomplishes nothing though (see comments below) because it is a tautology that for any pair of input parameters' lifetimes it is a tautology that 'a : 'b and 'b : 'a because by the definition of "Borrow" all borrows within a function have a lifetime that "at least" begins before the function is called and end no sooner than after the function returns so, withing the function the constraint between any 2 parameters' borrow lifetimes will always be 'a : 'b and 'b : 'a (they are equal).

4

I had considered that but when I tried the following code on Playground it compiled just fine:

fn func_x<'a, 'b: 'a>(r1: &'a i32, r2: &'b i32) {
}

fn func_y(r: &i32) {
}

fn main() {
    let d1 = 32;
    let r1 = &d1;
    {
        let d2 = 23;
        let r2 = &d2;
        func_x(r1, r2);
    }
    
    func_y(r1);
}

Playground link

I had expected it to fail because, inside main, r1 is clearly outliving r2 yet the call to func_x does not fail to compile despite the subtyping annotation.

(The only reason func_y exists in this example is to make it completely sure that the lifetime of r1 is longer than that of r2.)

5

Ok, I see what you actually want. There’s no way to do that with lifetime bounds alone, AFAIK. You can put an outlives bound (ie 'b:'a) on the function but the compiler is allowed to squeeze and shrink (immutable reference) lifetimes to make the call to it. So in this case it “shrinks” the lifetime of r1 down.

There might be a way to accomplish this by putting the references into a struct with manual variance control using PhantomData.

6

func_x borrows r1 and r2. At that point the life-time of r2 extends from the let until the end of the block. r1 lives from it's let untll the end of main. During the borrow, lifetime 'b is definitely as long as 'a (which is what 'b : 'a means - 'b is at least as long as 'a). When the func_x returns, both borrows end and everything is A-OK.

7

Got it. I had suspected I'm gonna have to resort to gymnastics with structs for this. Interesting to know about the lifetimes being adjusted by the compiler to the minimum needed by the function.

@gbutler69 I don't think that statement is true for this example without taking into account the shrinking business mentioned by @vitalyd.

Edit:

Oh. On second reading I think you're saying the same thing as @vitalyd

8

The only thing relevant is how long 'a and 'b are during the borrow. They are both equally long (at least). All borrows to a function have a lifetime that "at least" starts when the function is called and "at least" ends when the function returns. Therefore for any borrows to a function call, not matter how many borrows and different lifetimes on input parameters, give lifetime l1 and lifetime ln this would always hold:

  • 'l1 : 'ln
  • 'ln : 'l1

Seems wrong to say that both are "at least as long as the other", but, all it cares about is "at least as long as the other for the duration of the borrow". Borrows, by definition, MUST end when the function returns.

Basically, specifying, where 'a : 'b as part of a function call is redundant because all borrows of parameters have the property that they all have lifetimes for the duration of the borrow at least as long as one another.

The 'a : 'b lifetime notations is only useful for specifying relationships between input and output parameters and/or between parts of structures. It's not useful for specifying relative lifetimes between parameters because its a tautology that 'a : 'b and 'b : 'a by the definition of borrowing.

9

Yup. That's correct. Sorry for misreading you earlier (i.e. missing the crucial word "borrow")

Thank you both for the help guys.

10

I think what you are trying to do is guarantee that the pointers you passed to the c-function, if stored somewhere, 'b is guaranteed to outlive 'a. However, a borrow always ends when the function returns. Storing the pointers that came from borrows and using them at some undefined point in the future would definitely be undefined behavior (UB). This is why FFI calls are unsafe. Rust doesn't know what is being done with those pointers. If the pointers are used in the FFI call (but not stored anywhere) and then the C-func returns, everything is good-to-go, but, if the C-Func stores those pointers and tries to use them later somehow (or hands them off to a background thread or something like that) you would have UB.

EDIT (@gurry): To further clarify, if you want to give pointers to the C code that it can store, you need to do one of two things:

  • Box the values, get pointer to the boxed value, and mem::forget it (so it won't be deallocated by Rust) then pass those pointers to C (now, if C is using a different allocator than Rust, which it likely is unless you are using nightly and using system allocator for Rust), then, if C attempts to "free" the pointer, it will be UB, but, C can deref as much as it wants and use the value without UB because Rust will never deallocate it (due to the mem::forget)
  • make the lifetimes of the borrows 'static - this will only result in UB if the c-code attempts to do something with threading where it mutates the value pointed to (of curse, this will require that the things you are borrowing do, in fact, have a 'static life-time)

EDIT: To even further elaborate....

From the above it should be obvious that passing pointers to an FFI call is fraught with danger. You have to think very carefully about what you are doing on the Rust side as well as knowing that the C-Code is actually going to do with those pointers. An easy rule of thumb is:

  • If the C-function is guaranteed to only use the pointers for the duration of the call to the C-Funciton (and not store them anywhere, pass them to another thread, or in any way use them after the c-func returns), then it is OK to simply take a borrow (ref) and turn it into a pointer and pass to the C-func; however:
  • If the C-function plans to store those pointers and use them in some fashion after the c-func returns, then, you have 3 possible alternatives:
    • 'static lifetimes of the borrows (with the Caveat that you can still have UB if the C-Code multi-threads)
    • Rust allocate using Box and mem::forget after getting pointer and before calling the C-Func (with the Caveat that if the C-code attempts to "free" the pointer it will likely be UB; otherwise, it is a "Memory Leak" in the sense that nothing (Rust nor C) will ever deallocate it
    • Only pass pointers to C-Funcs where the pointer originated from calling a C-Func that allocated the memory so that any allocation that took place was done by C and C is responsible for freeing it

The last is the typical pattern in C of:

// Call the allocator to allocate a "foo" and store the pointer
someType *foo = foo_allocator( ... );

// Sometime later, call a function that operates on foo
foo_user( foo );

// Sometime later (or even elsewhere) deallocate the foo
free (foo);

11

Thanks @gbutler69 for the detailed explanation and advice.

In my case the the C code is indeed going to store the pointers. To be specific, foreign_func sort of "registers" -- which basically implies stores -- the pointer to S2 inside S1. I'm sort of going for the idiom you mentioned at the end. I'll enhance my S2 struct so that it keeps a reference to S1 and implement Drop for S2 which will unregister it from S1 (there's another C function available which does the inverse of foreign_func). This way my lifetime constraint should be enforced :slight_smile:

12

Some microoptimizations:

  • you don't need transmute to convert between pointers and references. reference as *const _ gives you a pointer. pointer.as_ref().unwrap() gives you a reference.
  • you don't need mem::forget hack. There's Box::into_raw() for FFI.
2 Likes
13

If you want to free a pointer you've passed to C, you need to get the pointer back and then reconstruct the box, letting it be dropped at the end of its scope. This will free the memory.

14

I’m re-reading portions of this thread and realize that it’s worth saying that lifetime bounds/relations that @gurry was looking at specifying are typically used in the “reverse” direction: when unsafe code returns pointers that you want to add memory safety to on the Rust side of the fence. Giving Rust-allocated pointers to C doesn’t require this and instead requires getting the pointer back at some point so it can be freed; in addition, the Rust allocation needs to be forgotten so that Rust doesn’t attempt to drop it prematurely; that’s basically where this thread has focused on anyway.