How is locking a Mutex not unsafe?

You can meaningfully analyze it. The important part is that if you try to enter that function in a reentrant fashion, it's not going to result in undefined behavior, and control flow isn't coming back out. That's as much as most applications need. If you dig deeper, you can look at the various implementations (they're all scattered about in here somewhere) and figure out exactly what it's going to do on any supported platform. If it truly did have undefined behavior, you would be unable to glean any information about what the program might decide to do. In particular, Mutex::lock would be unable to guarantee that it would not return normal control flow.

Where?

No. In addition to that it also says that a domain error or range error may occur.

It doesn't say how that domain range would be handled. It may trigger exception or send letter to grandma, there are no limitation for how that situation. None whatsoever. It's very explicit permission to do anything after error is detected. Including sending messages to grandma.

In practice it may raise signals and since C standard doesn't include such notion (even if POSIX standard does) it literally can not describe what happens and what limitations are there. It's, basically: here something entirely crazy happens which portable program couldn't even hope to handle… despair.

On MS-DOS that code may call an FP interrupt handler (which you can not, of course, handle in portable program, but can easily handle with specific MS-DOS C extensions).

It's exactly the same that Rust says about Mutex — and for the same reason (actually no, Rust gives you more: it explicitly promises that recursive mutex where such call just succeeds is not an option).

Wow. So if lrint jumps to interrupt handler (and yes, on some systems it's precisely what it does when domain error happens) then it's fine. When a /= b does the exact same thing (if b is zero) then it's fine, too. But if lock does that… unsafe, danger, Rust is no longer usable?

What makes Mutex and lock special???

No. When you call Do in your program you are triggering UB. Anything may happen — and in this particular case anything is call to rm -rf.

Why? Because in any program which doesn't trigger UB (and such programs do exist) an attempt to call Do would call that same function anyway. This makes such optimization valid: we only care about behavior of programs which don't trigger UB and if some other programs (with the exact same set of functions) actually trigger UB… well… too bad. Don't do that.

This call is not a guaranteed UB, that's the trick. You can add another module to your program (which would call NeverCalled before main) and if your would do that your program would become well-defined and everything would be 100% defined by standard.

For that, correct and well-defined program code does what is was supposed to do, which makes that optimization valid.

I am sorry but you switched the subject. We were talking about the word "unspecified" there, which refers to the return value. Now you're talking about domain errors, i.e. floating point exceptions, a completely separate thing from the unspecified return value.

Floating point exceptions also don't allow any arbitrary behavior. They can either be set to set a flag, or they can trap. Nothing else is allowed. There is a specification of this in the C standard. It's not that anything can happen. If it was "anything can happen", it would be called undefined behavior (like 1 / 0 is).

Yes, you can set up your system so that after your C program exits because of a trap then it will send an email to grandma. But that's a different thing from undefined behavior. You can also send an email after successful finish from your program.

Yeah, that's a bad example. I tried to search or make up a better example, but it's actually not that easy to cause on-purpose miscompilation with a clear specific cause. I'll post if I find a better case.

But the example I'm thinking about is something like this. You have a loop, you have a branch inside. Depending on the dynamic values, none of them may be executed. But if the compiler does a loop invariant code motion and does the check first, the code crashes. That's the example of UB triggering even though you don't execute the affected lines under normal conditions.

I'm sure that I saw some similar examples where instead of a loop we have a branch with a condition, which is known to be false at compile time. Depending on the order the compiler does constant propagation, dead code elimination and branch switching, the inner check may either be entirely removed, or hoisted to the top level and thus executed.

There is actually an extra requirement on UB: it must refine observable behaviour. If the first sequential part of the program does something observable (e.g. I/O), and then the second part has UB, the side effects of the first part must still happen, in the same order. But it's tricky to define and not very relevant to the specific topic discussed. It's something which can mitigate the blast range of UB in practice, though.

PR 97316. And I agree that this thread is basically going the same as that one did.

It will always be somewhat informal/wishy-washy because it's generally not possible to simultaneously enumerate all acceptable behaviors and give the API flexibility required to change the implementation (including supporting more platforms) in unforeseen future ways. The entire point of being unspecified is to leave wiggle room.

Mutex is actually better off in that it specifies it won't return (that would be unsound), but I'm sure it could gain an "encapsulated to the Mutex" clarification too.


For whomever mentioned I/O or side-effects, the default panic hook prints.

2 Likes

I feel like this is the exchange that just happened:

Me: "I don't know what this is going to do. Therefore it must be undefined behavior."
You: "It's going to do something other than causing undefined behavior."

But I still don't know what's going to happen, so that's not very convincing.

But you know what wouldn't happen and that's precisely what distinguishes unspecified behavior from undefined behavior.

Because unspecified behavior have some bounds. Enough to meaningfully talk about what would happen when it's triggered.

Undefined behavior have no bounds. If it happens in your program then it's not longer feasible to talk about what may or may not happen. Anything is possible.

Specifically WRT Mutex's lock what is know is that it may never succeed. You would have a deadlock, or maybe panic, or maybe program would just crash… and that's enough to meaningfully reason about behavior of your program after calling that lock.

It may not be precise enough for your needs, but it's precise enough to guarantee, e.g., memory safety.

3 Likes

That's literally undefined behavior in C

If the only restriction on behavior is that that it won't return, then that doesn't help me. I still don't know what's going to happen.

What will happen can vary significantly depending on the target. Currently, Windows and UNIX mutexes will deadlock. Wasm will panic. Other platforms may or may not do other things, depending on what tools the hardware and OS provide. You shouldn't rely on any behavior beyond what's said in the docs. Suffice it to say, your program will cease doing useful work, unless you get lucky, and are able to catch a panic and resume work. If that guarantee isn't strong enough, you'll need to roll your solution, using better-specified platform-specific APIs.

1 Like

Maybe, but it does help anyone who wants to ensure that lock wouldn't cause memory corruption.

Sure, but so what? You can write code which would do something unpredictable in many other ways. If you would just use 1 (one) array of u8, rand and nothing else you can easily put yourself into situation where you would be unable to predict what will happen to your program.

This doesn't, somehow, turn array of u8 into something unsafe!

Yes. The same is true of dividing by zero. That's why it's undefined behavior in C. For that one, Rust opts to panic, so that the behavior will be consistent.

To be clear, the docs don't actually say that memory corruption won't happen. That's part of the problem. If they did, that still wouldn't help. With rand, I know that my grandma will receive zero messages from me. I don't get that with lock. The problem is that it can do anything besides return.

The form that undefined behavior takes is also limited. It can't teleport my brother, nor can it sprout wings from my friend's back. There are still infinitely many possible things it can do, and the same is true for lock.

Yes, but division by zero being undefined behavior has far more consequences than just exposing platform-specific quirks. It means that if your code ever reaches a branch where division by zero occurs, your program is allowed to do literally anything from that point on. Zero division causes arbitrary parts of code to do arbitrary things in an unpredictable manner, which is very not good.

lock not having UB prevents that. Regardless of exactly what happens, a call to lock recursively lets you reason about things after the fact. Since it doesn't return, that means it aborts, panics, or enters an infinite loop, and potentially does other cleanup work in the process. You can reason about what happens to your program in each of those cases, and go from there.

Why would it have to say that? It's safe function, it have to ensure that memory can not be corrupted. If it would be possible to corrupt memory using lock with just a safe code it would be a soundness bug.

You wouldn't know that. Or, rather, you would be unable to prove that she wouldn't get them given the convoluted program (said program would just have to emulate x86 CPU and then Linux or Windows with rust program on top of that… not that hard to do if, maybe, not that fast).

You claimed that half-dozen of times yet offered zero explanations about why is that a problem.

It probably wouldn't do that. Today. But it's permitted to do that. And may do precisely that one day, who knows?

That's not true in both cases, but that's besides the point. The point is: undefined behavior can do anything, unspecified behavior is limited. In particular soundness is guaranteed.

You may not like these limits but that's not a reason to throw a temper tantrum and start claiming that this function is unsafe, contains UB and so on.

Yes, it's behavior is not fully specified. Deal with it.

It's all true but that doesn't explain why division behavior is defined like this: this operation will panic if other == 0 or the division results in overflow.

If it would have followed Mutex footsteps then it would have been defined as this operation will panic if other == 0, result on overflow is unspecified or, maybe, even when other == 0 or the division results in overflow behavior is unspecified.

This would have allowed to define -2147483648 / -1 as -2147483648 which would have produced less code and would have been faster than what Rust does today.

Yet, somehow, it was defined differently. Why?

To be entirely honest, I'm not entirely sure, but I can make a few guesses as to why.

  1. Hide the underlying platform. Integer overflow on most of the other integer operations is unspecified. Rust can do this because it allows checking for overflow errors on debug mode, while allowing a sane, fast, and portable default (in this case wrapping) for release mode. However, going to bitshifts, a shift greater than the integer's bit width does something different on basically every major platform. It's a similar case for integer division by zero- arm processors return zero, and x86 and wasm both trap. In both of those instances, ensuring code panics makes it much easier to write platform-independent code.

  2. Integer division by zero is almost certainly a bug. There isn't really any reason to compute an integer division by zero, the check is trivial, and on many machines, the hardware has to perform that check already. Additionally, it's much easier to track the origin of a panic than something like a floating point exception, especially if you're unfamiliar with the fact that x86 can throw those during integer division.

  3. Mutexes are hard. There's a reason why rust delegates to the OS for mutexes on platforms that support threads. Determining if an attempted lock is reentrant can't be done without OS support, and that support could very well come with a performance hit, which is bad since mutexes are often performance bottlenecks, and rust's stdlib has had plenty of trouble getting good performance out of the mutexes already.

  4. Finally, it could just be poor design on the mutex's part. It might be the case that porting all current mutex implementations to a design that enables always panicking on reentrant locks but it, for whatever reason, just hasn't. It wouldn't have been the first time that poor design has hampered stdlib's mutexes- poor design is the reason Mutex::new wasn't const until 1.63, third party mutexes have been able to significantly outperform the stdlib ones (at least until 1.63 anyway), and poisoning by default has also been considered a mistake by a not insignificant portion of rustaceans.

Of course, take everything I've just said with a grain of salt, as I was probably in middle school when these decisions were being made, and I haven't looked to find any official motivations for them.

Sending rude messages to the grandma of either one of us still doesn't corrupt memory (with the possible exception of her memories about humans being nice to each other).

3 Likes

This is basically the same rationale that (at least originally) led to C adopting the idea that int overflow and division by 0 is undefined behavior -- there are different platforms, what if we're on a platform where int overflow kills the program, etc.

I think Rust should just commit to some set of specific behaviors (e.g. deadlock or panic or abort, or something like that), just like it has committed with int overflow and division by 0.

If Rust is ever ported to a platform where the default OS mutex behavior does something crazy, Mutex can always be implemented on that platform in a way that explicitly checks for double locking and does a sane thing in that case, like panic.

Just like Rust already inserts explicit checks for division by 0 on x86, for example, so that it can panic, rather than saying that division by 0 may lead to unspecified unpredictable behavior.

7 Likes

This is a common misconception. Programming languages don't use "undefined behavior" in its English meaning, but rather as a label for (paradoxically) very precisely defined rules of what is forbidden in a language.

1 Like