Issue with lifetime subtypes

help
1

I'm not quite sure I understand lifetime subtypes fully yet. I have code similar to the simplified example below, that when I try to compile gives me the error "cannot borrow batch as immutable because it is also borrowed as mutable".

fn batch_process<'a, 'b: 'a>(batch: &'b mut [Request<'a>]) -> Result<(), String> {
  // ...
}

fn main() {
  let a = "foo";
  let mut batch = vec![Request::new(a)];
  {
      let slice = &mut batch;
      batch_process(slice).unwrap();
  }
  assert_eq!(batch[0].result, Some(3));
}

The error goes away if I remove the 'b: 'a lifetime requirement on the batch_process method, i.e. this works (as expected):

fn batch_process<'a>(batch: &mut [Request<'a>]) -> Result<(), String> {
  // ...
}

fn main() {
  let a = "foo";
  let mut batch = vec![Request::new(a)];
  batch_process(&mut batch).unwrap();
  assert_eq!(batch[0].result, Some(3));
}

A more complete but still simplified example can be found here: Rust Playground.

I don't understand how the additional lifetime requirement on batch_process affects how I can use the batch data in the main method. How does the &mut borrow "escape" the scope of the enclosing { ... } to affect the following non-mutable reference in the first example? Any way to avoid this?

2

It is a bit weird, but it's what your lifetime requirements specify :slight_smile:. The 'a lifetime becomes 'static because you're giving Request a string literal. The mutable borrow is specified to outlive that due to the 'b: 'a requirement, and that's why you get the error.

Is there a reason you need to specify it like that in your real code?

3

Isn't the error message misleading here? As far as I can see, the variable slice goes out of scope before the assert..., no matter the signature of batch_process. It's just that by your reasoning, it can't possibly work when you have a request containing a 'static item (unless you somehow managed to get a 'static reference to it, which I'm not sure how to do). So the error message should probably say something like batch_process: can't satisfy lifetime condition 'b: 'a where 'a is 'static and batch comes from slice, I think.

4

slice is passed to batch_process - how do you "know" it didn't stash it somewhere? It doesn't return it back, yes, but the borrowck works at the fn signature level.

5

slice is passed to batch_process - how do you "know" it didn't stash it somewhere?

Not sure what you mean. The variable slice defined at line 4 of main() is defined inside of {}, so it must go out of scope at }, those are just the scoping rules, right? You can't "stash" it somewhere because the lifetime analysis will know its scope and check accordingly.

6

slice is a borrow of/reference to batch, it's not an owned value. The slice binding goes out of scope at the end of {}, but not the value behind the reference.

7

Right, but isn't that exactly what 'b in the signature of batch_process denotes? I.e. &'b mut denotes that batch is a mutable reference of lifetime 'b, and when batch_process is called, slice is inserted as batch (sorry if my english is a bit rocky here), so 'b is the lifetime of slice, not the lifetime of the value referenced by slice.

8

Yes, but now that you insert the lifetime of slice for 'b, the signature also says that 'b outlives 'a. In this case, 'a becomes 'static but you can just as well change the code to be and get the same error:

let a = "foo".to_string();
let mut batch = vec![Request::new(&a)];

The borrow scope is extended because of the 'b: 'a requirement, even though slice binding itself goes out scope.

9

I'd say the same reasoning applies: 'b is the lifetime of slice, 'a the lifetime of &a (which is owned by batch, so it's invalidated when batch goes out of scope), and you can't make slice outlive batch, which is what 'b: 'a would imply. Sot it rightfully errors out, but I still think the error message is not what it should be.

(Maybe I should clarify, I don't think I'm right, I don't have a lot of rust experience, I just want to find out what's wrong with my thoughts. If you don't want to argue with me, that's fine for sure :))

10

fn batch_process<'a, 'b: 'a>(batch: &'b mut [Request<'a>]) -> Result<(), String>

The above says that batch is a slice of Request values, where each Request must not outlive 'a (e.g. it's holding a reference to something of lifetime 'a, and must therefore not outlive that - but, that's a requirement on Request purely on Request<'a> signature).

So, slice can't outlive batch because batch is the owner, yes, but that's not what the 'b:'a is implying. The 'b:'a is relating the slice to the Request value's inner references, independent of batch.

11

(e) Ignore this for now, we posted in parallel :slight_smile:

But note that in addition to what the OP said, the example runs if you only change 'b: 'a to 'b in the signature of batch_process. Looking at the first sentence of The Rust Programming Language, that does not change how long any of the references live (in particular, slice willl still go out of scope after }).

12

Right, the error goes away because that requirement/constraint is lifted since 'b and 'a are independent and borrowck fills them in with its own concrete lifetimes when the function is used :slight_smile:. It's kind of like defining a generic function on type T and requiring that T impl, say, Hash but then not using Hash in the function body - it's a needless constraint (in that circumstance).

13

I think we've fallen into the trap of same-named-variables, there's a batch as the first argument of batch_process, but there's also a batch as a variable of main. I've made a small change at Rust Playground, so it can be clearer.

What I meant is that slice can not outlive batch1. But if 'b: 'a would hold true, that would mean that slice outlives the references contained in the slice of Request values, which in turn live longer than batch1, so slice would outlive batch1, contradiction.

So as far as I can (still) see, the problem is not that at the time of assert the mutable borrow from batch1 still exists (it doesn't), but that we don't fulfill the signature of batch_process. We can change the signature so we fullfill it, but note that by The Rust Programming Language this does not change how long any of the references lives.

14

Well, if the constraint is needless is dependent on the body of batch_process, so we can't tell right now. But we know that if batch_process can be written without requiring 'b: 'a, we can drop that constraint and the sample code compiles :slight_smile:

15

Correct, the issue is precisely that we don't fulfill the requirements of the batch_process signature. Generic lifetime parameters never change how long things live, they simply state constraints. Similarly how type parameter constraints (bounds) don't change concrete values passed in, just put bounds on what can be passed in.

16

Ok, gotcha :slight_smile: So my original point still stands, I think the error message doesn't reflect the problem. I'd be thinking about opening an issue, if you agree.

@jhecking: Sorry for derailing your thread!

17

Well, I'm not sure the error message is a problem (once you understand what's happening, of course). The signature of batch_process is extending the borrow. Now, to your earlier point, no actual liveness changes of the real values. But, the borrow (which can be thought of as a logical concept, although most often used interchangeably with a concrete reference notion) is extended. So, since the compiler fills in concrete lifetimes, it's inferring that the borrow must outlive the lifetime used in Request, which ends up being much longer than we expect here. It then tells you it cannot borrow immutably because the lifetime it inferred earlier does not allow it.

I'm not sure it's practical for the compiler to attempt reverse engineering what you really meant here. It does indicate where the borrow starts and ends, although perhaps it could be unclear to someone unfamiliar with this as to why the mutable borrow seems extended (maybe such an extension could be marked somehow in the help output?).

18

I understand what you mean, but I think the message is quite confusing and would lead people down the wrong path. I guess it might at least say " Can't borrow.... BECAUSE OF THE LIFETIMES I INFERED FROM THE SIGNATURE"? I kinda feel that while lifetimes and borrowchk are intertwined, this is not a problem of the borrow checker, but of lifetime analysis... Maybe @jhecking wants to chime in, since he had (has?) the problem in the first place :slight_smile:

19

Not at all! This discussion has been very helpful to me to clarify my thinking around these issues. I understand now why the compiler is raising the error though I do agree that the error message could be more helpful. The error message and detailed explanation for the error E0502 eventually lead me to the section on &mut references in the book. I thought the example given there exactly matched my use case. I did not even consider that the method signature of the batch_process method might be relevant in determining the lifetime of the &mut batch reference in main().

You are getting at the heart of the problem here: I do not fully understand why (or rather if) the 'b: 'a constraint is even required. I arrived at it more through trial and error than through careful consideration but haven't found a way to remove it. Unfortunately, in the playground example I posted at Rust Playground it is not actually required. But in my real code it is and I am not sure of the reason. The actual version of the batch_process function can be found here: https://github.com/aerospike/aerospike-client-rust/blob/batch-reads/src/batch/batch_executor.rs#L46. If I change that method signature from

pub fn execute_batch_read<'a, 'b: 'a>(&self,
                                      policy: &BatchPolicy,
                                      batch_reads: &'b mut [BatchRead<'a>])
                                      -> Result<()>

to simply

pub fn execute_batch_read<'a, 'b>(&self,
                                      policy: &BatchPolicy,
                                      batch_reads: &'b mut [BatchRead<'a>])
                                      -> Result<()>

then I get the following compile error which I do not understand:

   Compiling aerospike v0.0.1 (file:///Users/jhecking/aerospike/aerospike-client-rust)
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'b` due to conflicting requirements
   --> src/client.rs:196:18
    |
196 |         executor.execute_batch_read(policy, batch_reads)
    |                  ^^^^^^^^^^^^^^^^^^
    |
note: first, the lifetime cannot outlive the lifetime 'a as defined on the body at 194:47...
   --> src/client.rs:194:48
    |
194 |                                    -> Result<()> {
    |  ________________________________________________^ starting here...
195 | |         let executor = BatchExecutor::new(self.cluster.clone(), self.thread_pool.clone());
196 | |         executor.execute_batch_read(policy, batch_reads)
197 | |     }
    | |_____^ ...ending here
note: ...so that reference does not outlive borrowed content
   --> src/client.rs:196:45
    |
196 |         executor.execute_batch_read(policy, batch_reads)
    |                                             ^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'b as defined on the body at 194:47...
   --> src/client.rs:194:48
    |
194 |                                    -> Result<()> {
    |  ________________________________________________^ starting here...
195 | |         let executor = BatchExecutor::new(self.cluster.clone(), self.thread_pool.clone());
196 | |         executor.execute_batch_read(policy, batch_reads)
197 | |     }
    | |_____^ ...ending here
note: ...so that expression is assignable (expected &mut [batch::batch_read::BatchRead<'_>], found &mut [batch::batch_read::BatchRead<'b>])
   --> src/client.rs:196:45
    |
196 |         executor.execute_batch_read(policy, batch_reads)
    |                                             ^^^^^^^^^^^

error: aborting due to previous error

If I remove the lifetime declaration 'b entirely I am getting this somewhat similar and to me equally cryptical error:

   Compiling aerospike v0.0.1 (file:///Users/jhecking/aerospike/aerospike-client-rust)
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
  --> src/batch/batch_executor.rs:51:27
   |
51 |         let batch_reads = SharedSlice::new(batch_reads);
   |                           ^^^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #3 defined on the body at 49:48...
  --> src/batch/batch_executor.rs:49:49
   |
49 |                                   -> Result<()> {
   |                                                 ^
note: ...so that reference does not outlive borrowed content
  --> src/batch/batch_executor.rs:51:44
   |
51 |         let batch_reads = SharedSlice::new(batch_reads);
   |                                            ^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the body at 49:48...
  --> src/batch/batch_executor.rs:49:49
   |
49 |                                   -> Result<()> {
   |                                                 ^
note: ...so that expression is assignable (expected batch::batch_executor::SharedSlice<'_, batch::batch_read::BatchRead<'_>>, found batch::batch_executor::SharedSlice<'_, batch::batch_read::BatchRead<'a>>)
  --> src/batch/batch_executor.rs:51:27
   |
51 |         let batch_reads = SharedSlice::new(batch_reads);
   |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

In the simplified batch_process method I can remove the 'b lifetime entirely and the code still compiles.

20

This is with removing the 'b:'a bound here as well: https://github.com/aerospike/aerospike-client-rust/blob/batch-reads/src/client.rs#L191?