I have a mutable reference to a slice, I want to write a function which puts data into a slice and then advances a mutable reference by the amount of data written.
This was my first version which didn't compile.
fn extend<'a: 'p, 'p>(out_buf: &'p mut &'a mut [u8], data: impl AsRef<[u8]>) {
let bytes = data.as_ref();
let len = bytes.len();
{
let buf: &'a mut [u8] = &mut out_buf[0..len];
buf.copy_from_slice(bytes);
}
*out_buf = &mut out_buf[len..];
}
At first I tried to change the lifetime bounds to 'p: 'a, but this made it impossible to call this function multiple times.
fn extend(out_buf: &mut &mut [u8], data: impl AsRef<[u8]>) {
let bytes = data.as_ref();
let len = bytes.len();
let buf: &mut [u8] = std::mem::take(out_buf);
buf[0..len].copy_from_slice(bytes);
*out_buf = &mut buf[len..];
}
But I don't understand exactly why this works.
My current reasoning is that getting a mutable reference from a mutable reference forces the lifetime to be narrowed, i.e:
let pp: &'small mut &'big mut u8 = ...;
let p = &'small mut u8 = *pp; /* cannot be &'big */
/* so once I write it back ... */
*pp = p; /* requires 'small: 'big */
But I don't understand why exactly this limitation exists.
It will work, thanks. It will also work if I create my own struct like UdpPacket and will keep the length there. But I want to understand how to make my example work also.
Yes, it will. Then after I am done. I can take the remaining length, sub it from the original length and get how many characters were written into the buffer.
It's generally easier to explain why something doesn't pass borrow check than to explain why something else does pass borrow check, but I'll give it a shot.
In the working version, out_buf: &'a mut [u8] is reborrowed twice in extend:
// A short reborrow that expires immediately
out_buf[0..len].copy_from_slice(bytes);
// A reborrow for all of `'a`
&mut out_buf[len..]
As for the returned slice: when you have a t_ref: &'a mut T, and you reborrow *t_ref for all of 'a, you generally can think of it as moving t_ref instead -- you can't use t_ref directly again,[1] because in some sense you have given away all the access t_ref had. You don't use out_buf after you return it, so there's no borrow checker concerns in extend.
Then from the perspective of main: You have buff: &'buff mut [u8] = elements.as_mut(), then you pass a &'buff mut [u8] to extend -- that is you pass a reborrow with the entire original lifetime -- and then overwrite buff with the return value, which is also a &'buff mut [u8].
The borrow checker is okay with this as it understands that overwriting a reference doesen't read the referent data, so overwriting buff doesn't conflict with *buff being reborrowed. It's a particular special case that the borrow checker was designed to handle. So buff is unusable when you pass it in to extend but becomes reusable after the assignment -- and you don't try to use it during that period, so it's fine.
A &mut _ implies exclusive access, and violating that is UB. It's a fundamental rule of Rust. Your start became invalid once you started using buffer directly again.
A good rule of thumb is that once you go into raw pointer land, do everything via the raw pointer until you're ready to come out of raw pointer land. So for example:
fn fill_buffer<'a>(buffer: &'a mut [u8], f: impl Fn(&mut &mut [u8])) -> &'a [u8] {
// Get the length first
let buf_len = buffer.len();
// Go into raw pointer land
let start = buffer.as_mut_ptr();
// Recreate the buffer from that
let p = &mut unsafe { std::slice::from_raw_parts_mut(start, buf_len) };
// Transiently use the new `&mut _`
let msg_len = {
f(p);
buf_len - p.len()
};
// Leave raw pointer land
unsafe { std::slice::from_raw_parts(start, msg_len) }
}
Of course, an even better rule of thumb is "don't resort to unsafe unless you absolutely must" .
Edit: Keep reading this thread to see why (the above playground isn't sound either).
When you use slicing there's an implicit function call happening which picks a self.
Exclusive references are not copyable. They can either be moved or reborrowed, like uniquely owned objects.
It's not simply &mut buf[..], it's IndexMut::index(buf, range).
When you make a function call, there's automatic adjustment of references happening. You can't index &mut &mut buf, because that's a reference to a reference, not a slice. In the call it automagically becomes &mut **buf, and the compiler has to figure out a lifetime for the newly dereferenced and reborrowed object.
When you call on the original object, with its original lifetime, this autoderef has to reborrow to give you a shorter temporary lifetime for the call. Otherwise it wouldn't uphold the exclusivity and invariance promises, because the full-length reference still exists where it was before. It's seems silly when it's &mut, but think what it would do for Vec. You can't have the same non-empty Vec in two places! You can't have two &'exclusive Vec either. There can only exist Vec and a temporary &mut Vec.
However, when you use take, you do create a new object! It returns you exclusive ownership of the non-empty slice reference, and creates a new independent empty slice reference in its place.
So now that you have exclusive ownership of the slice, &mut **buf doesn't have to reborrow, it can move it instead.
The same as if you took ownership of a Vec, leaving vec![] in its original location.
In spirit the example is trying to "unsplit" a split borrow, and there's no purely safe way to do that, though it's possible someone has safely encapsulated the logic in a crate or such.
Maybe they "returned" the start of the slice... ↩︎
.