I am trying to understand how thread local storage manages to be safe in Rust. If I can safely (not writing any unsafe blocks of my own) get a reference to something in thread-local storage, what prevents that reference from getting out of that thread and surviving longer than that thread's local storage (and dangling)?
That's what the Send auto trait is for. In Rust it's used as a marker trait to signify that something is safe to be sent between threads or, in other words, if something does not implement Send, it cannot be safely (as in without using unsafe) sent between threads.
Two things.
First, the thread-local storage object itself has a lifetime, and any reference to it has a lifetime which is bounded above by the lifetime of the object. If you try to pass that reference to something that doesn't guarantee it will only be used while that lifetime is still valid, then the compiler will catch that and point out the lifetime problem. The only references for which this isn't true, in safe Rust, are 'static references, which can only be derived from objects guaranteed to exist for the rest of the program's execution anyways.
Second, if the referent is not Sync, then a reference to it is not Send, and inter-thread communication almost always requires Send. If, on the other hand, the referent is Sync, then there's no problem with passing out a reference to another thread (assuming the lifetime of the referent allows it).
What if the thread local var is something that is Sync because its POD, like an i32. So the ref &i32 is Send. What's to prevent that reference from leaking out of that thread into another? It can even be a &'static i32. My understanding is that thread-local requires 'static lifetimes just like global data - is that wrong? If I can get a &'static i32 from a thread-local i32 value and pass that out of the thread, how does that not end up dangling when the thread terminates?
Thread locals are accessed via the LocalKey type. If you look at that, you'll notice it doesn't allow you to directly obtain a reference to the thread local. The main access is via the with() method, which takes a closure. A reference to the value contained in the thread local is only available in the closure and Rust's borrow checker ensures it cannot escape the closure. Other methods on LocalKey either offer specialised versions of with() or move the value out in some way.
@firebits.io LocalKey does implement Send, because accesses from different threads access different locals, so there is no particular problem sending the key between threads because it will still access the appropriate local.
6 Likes
@jameseb7 , sorry, I didn't check if it implemented Send. I just wanted to let the OP know about the Send and Sync traits in Rust.
1 Like
The main access is via the
with()method, which takes a closure.
The with() method takes a FnOnce(&T)->R closure. Is the reason this works because the &T arg isn't &'static T, so it cannot be captured into any external global state that can be accessed by other threads, or even returned as part of R? Even if R is itself &T?
Yes, the param and return value will have different lifetimes.
use std::thread;
thread_local!(static FOO: u32 = 1);
let x: &u32 = FOO.with(|r| r);
error: lifetime may not live long enough
--> src/lib.rs:17:36
|
17 | let x: &u32 = FOO.with(|r| r);
| -- ^ returning this value requires that `'1` must outlive `'2`
| ||
| |return type of closure is &'2 u32
| has type `&'1 u32`
Thanks! What really threw me was that thread-local values are expected to have 'static lifetime. Which meant to me that something else had to be preventing reference escape from the thread. It's the fact that the lifetime of a closure's arg may not be long enough to last outside the closure (unless the signature of the closure specifies it can by linking that lifetime to something else).
I agree, the use of 'static here is confusing. Rust usually doesn't have "magic" behavior, but this seems to have some magic. I suspect it's because the native OS thread locals are used underneath, so the unsafe code to make this work is tricky.
I think the post from @jameseb7 explains it best.
If you mean that it says T: 'static, that means that the type T cannot contain references to anything with a shorter lifetime than 'static. That includes all types that don't contain references. This is so it doesn't hold a reference to anything that could be dropped before the program ends. That means it could be stored until the end of the program, but doesn't have to be. It doesn't mean that it's automatically accessible from other threads.
1 Like
By the way, to be clear (I don’t think it has been mentioned in this discussion yet), there’s nothing preventing just the first part (the “reference from getting out of that thread”), only the second part (“surviving longer than that thread's local storage”) is prevented by the closure-expecting API, but also only that second part matters.
Other threads can obtain a reference to the local data, provided that data is thread-safe (Sync) and you use API such as std::thread::scope).
Also, note that the closure in the LocalKey::with method features an implicit HRTB, where F: FnOnce(&T) -> R is a shorthand for F: for<'a> FnOnce(&'a T) -> R. This mechanism (of using a HRTB) is what powers the restriction that the &T reference is not allowed to escape the closure.
2 Likes
That's a good point. Those threads are within the scope (and lifetime) of the thread-local owning thread. If I really do need to prevent something leaking to any other thread, it still needs to be !Sync, not just thread-local.
This brings up another topic I've been thinking about: how hard it is to make sure unsafe code turns out to be safe. It seems that in some cases, the developer of unsafe code has to know virtually everything about Rust in order to determine that there are no edge cases that remain unsafe. The analogy that comes to mind is of adding a single new axiom to an existing formal mathematical system and proving that the result remains consistent.
If you hadn't mentioned thread::scope,I might have relied on thread-local storage keeping some unsafe code safe by preventing (without assistance from !Sync) some reference from being accessible to any other thread.
1 Like
It is quite difficult IMO, and difficult enough that one should run their tests with Miri to identify unsoundness, and should ask for a code review here by the experts (who we appreciate very much!)
It is true that a thread-local can be accessed by a scoped thread, but there is no particular safety issue. Accessing the thread-local in the scoped thread, whether with safe or unsafe code, is no different (more or less safe) than other situations. Edit: What I mean is that when writing unsafe you always have to assume the worst (e.g., that violating ownership rules will cause UB).
But I don't think there is a succinct answer to your question, since the difficulty depends greatly on the situation. I only have experience in 3 situations writing unsafe code, and they were all very different from each other, and I did it incorrectly in 2 out of 3 (which I found with Miri).
My personal goal is not to learn to write unsafe code in every situation, but rather to learn how to avoid writing unsafe code in every situation. 
1 Like
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.