Lifetime names? Confused

[duaneellissd]

I am not getting anywhere reading the various 'lifetime' things.
I've tried the RUST BY EXAMPLE, and the Brown UNIV RUST Course
And things here. so I'll try here:

First background: Yes, I understand what a variable lifetime means, I understand that if something is allocated from the heap and if something points into that memory blob, if it is reallocated - that blob is no longer valid because the heap moved your buffer. Lifetime is also a lot like what I know as the "gen" and "kill" of a variable a long time ago. The GEN (or generation) of a variable is when it is first assigned, and the KILL is the last place it is used within the control flow.

But as much as I read - I do not see/understand the 'a, seems common, and other times it is 'a, 'b, 'c - these names seem arbitary- but - there are others like 'static - which sometimes is illegal to use because it is a reserved word..

So my problem is this:

a) I need to have a rust module that manages some compile time pre-allocated buffers in C, think of these as memory regions in "DMA SAFE" memory space. I have one called the ETH_INP_BUF, and the ETH_OUT_BUF, and the SPI_RD_BUF and the SPI_WR_BUFFER. - on my platform I must allocate these by way of tricks in the linker because the FPGA can only access certain memory regions. You might guess, I am doing low level OS stuff, and in my case it is all NO_STD, and I cannot use any allocation.

I want to have a create called a "ByteBuffer" - I would know it as a C struct, with three elements: (A) A pointer to memory (B) a capacity, and (C) a cursor - where I am writing or reading. In RUST - both (A) and (B) are effectively a reference to an array.

I created something in the playground here:

Rust Playground

Some confusion: RUSTC says I need to add 'a everywhere -but I do and then I get other errors.

The lifetime of the ByteBuffer2 will always be less then the buffer reference.
Each buffer is generally a global variable in DMA safe memory.

I can't allocate it - RUST does not understand the idea/concept of multiple heaps and allocating data from different heaps [Yes, I looked over the slab allocator it does not help]

Suggestions would be most helpful - I'm now 48 hours into trying numerous things and I am not making progress and seem to be going in circles.

[robofinch]

Here's my attempt to clean up your code: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=087e7efe8997e8685062d5717ebfec2a

I did some manual linting, fixed bugs, and left some comments explaining lifetimes.

Four main notes:

• Wherever you have &mut T, the lifetime is "elided" but still very much present. What the lifetime is inferred to be depends on context to some extent. E.g., const MSG: &str = "whatever"; infers that &str is &'static str. When you have a method like fn foo(&self, other: &[u8]) -> &[u8]), this is desugared to something like fn foo<'a, 'b>(self: &'a Self, other: &'b [u8]) -> &'a [u8]. That is, by default, the output is inferred to have the same lifetime as self and the other input is permitted to have an arbitrary lifetime.
• Also, as you noticed, 'static is special. All other lifetime parameters are generic lifetime parameters, and generic parameters like 'a or T can be given arbitrary names. (There's also '_, which desugars into one of the first two cases, in the same way as discussed above for references not explicitly given a lifetime parameter at all.)
• Since static mut is very hard to use correctly, Rust adds quite a bit of friction to it. At the very least, to make its usage sound, it requires unsafe (this likely includes making your own functions unsafe, to pass on soundness requirements to the functions' callers).
• fn bar(self: &'a mut ByteBuffer2<'a>) says "I, as a value of type ByteBuffer2<'a>, can only exist for lifetime 'a. Also, I need to be mutably/exclusively borrowed for the rest of my existence for my bar method to be called, so you can never use me again." It is not what you want here. fn baz(self: &mut ByteBuffer2<'a>) desugars to fn baz<'b>(self: &'b mut ByteBuffer2<'a>), which says "I need to be mutably/exclusively borrowed while my baz method is called". Much easier to call baz than bar.

@duaneellissd

I can't allocate it - RUST does not understand the idea/concept of multiple heaps and allocating data from different heaps

Rust (as in, the core language, excluding alloc and std) doesn't inherently have a conception of any heap (or stack, even; the stack is an implementation detail which doesn't affect language semantics, though it's of course useful for understanding why some of the language semantics are what they are).

However, I do know that Rust isn't very friendly to runtime-loaded and unloaded libraries (e.g. for dynamic plugins or hot reloading and whatnot... I'm not very familiar with that area), since the 'static lifetime means "lasts for (or could last for) the entirety that the program is loaded"... there's not a notion of different crates having different 'statics, or of some parts of the binary being loaded for different lengths. That's a niche problem, though, and likely doesn't affect you.

[robofinch]

@duaneellissd

First background: Yes, I understand what a variable lifetime means, I understand that if something is allocated from the heap and if something points into that memory blob, if it is reallocated - that blob is no longer valid because the heap moved your buffer.

Lifetimes are somewhat different than that in Rust, at least in that you can do interesting type system shenanigans with PhantomData<&'a ()> or implement "generativity" regardless of whether real backing data is involved.

Even when lifetimes are used normally, they really denote "borrow durations" more than the liveness of values. Others can probably explain this far better than I can: Explain this program. just for understanding purpose - #6 by quinedot

[quinedot]

This comment is a high-level response, not about the particulars of your code.

@duaneellissd

First background: Yes, I understand what a variable lifetime means, I understand that if something is allocated from the heap and if something points into that memory blob, if it is reallocated - that blob is no longer valid because the heap moved your buffer.

As was noted above, Rust lifetimes -- those 'a things -- are generally about the duration of borrows. The don't directly correspond to the validity^[1] of a variable. The overlap in terminology is unfortunate, and the fact that most learning material conflates the two concepts (along with lexical scopes, to boot) is very unfortunate.

You create shared borrows with & and exclusive borrows with &mut, and these borrows usually have a much shorter duration than the validity of the referent.

It is true that it is illegitimate to borrow something beyond its validity,^[2] and (so long as you don't obfuscate things via unsafe) the compiler will enforce this.^[3] It's illegitimate to do other things while borrowed too -- for example you cannot move something which is borrowed, and you can not take a fresh &mut _ to something which is borrowed. Additionally, you can't take a fresh &_ to something which is exclusively borrowed -- borrowed by a &mut _.

If you want to do things the compiler can't prove are legitimate -- like creating &mut _ to global memory without some synchronizing abstraction (a Mutex or whatever) -- you'll need unsafe in some form. That may be working with *mut instead of &mut to forgo borrow checking, or it may be creating &mut _ to static muts like in the playground. When you do so, it's up to you to uphold the language invariants, or the results are undefined behavior.

For example, it's UB to create two independent &mut _ to the same place (such as a static mut) simultaneously. And not only in a multi-threaded application! For example, it is UB to create two such &mut _ at different levels in the call stack simultaneously, which you can do in a single-threaded application.^[4]

Because you're working at such a low level, you'll have to use unsafe to some extent. In that case, there will probably be a learning curve for doing so correctly (what is defined or UB is different between C and Rust). Refer to the Nomicon and use Miri to test your unsafe. Ideally you can find a way to encapsulate most of the unsafe. You might also want to read the embedded book. There are some embedded-knowledgeable people active on this forum, but there may be more specific forums too (I'm not sure).

If Miri says you're triggering UB and you don't understand why, feel free to open another post asking about it.

---

1. due to reallocation or deallocation, etc ↩︎

2. say, when the destructor is called, or when a variable goes out of scope and becomes uninitialized ↩︎

3. More accurately, if it can't prove this is impossible, an error is thrown. ↩︎

4. Rust doesn't really have a concept of a "non-multithreaded application" anyway. But things like this come up when people think such a concept would save them from borrow checker headaches and the like. ↩︎