FWIW POSIX quite explicitly says that double-locking is UB for certain kinds of Mutexes.
But it defines it for others. Maybe Rust should just use NORMAL mutex and not DEFAULT one?
That's a bit like "how many straws break a camel's back" problem. We can assume that a non-buggy OS satisfies some reasonable properties. It certainly shouldn't burn your house, or even your hard drive. But will a double-lock cause your program to be terminated? Will it work flawlessly, because the mutex is recursive? And if it is, do you need to unlock several times? Will it cause writes to files (e.g. unconditional error messages in logs or stderr)? Cause a deadlock? Deadlock other processes? Will it leave the OS in an unrecoverable state? Or cause a reboot? Personally I wouldn't confidently exclude any of those alternatives, if Rust wants to support any OS currently in existence.
The problem here is that OS-level mutex doesn't give you such a guarantee. DEFAULT mutex can do anything, apparently. While NORMAL is guaranteed deadlock and ERRORCHECK is guaranteed to panic.
Rust doesn't win anything for underspecifying mutex.
1 Like
When using pthread mutexes, rust uses the NORMAL type specifically because DEFAULT causes undefined behavior.
Rust can't use any OS mutex primitive that could cause the mutex to become unlocked on a double lock, because that would lead to aliasing mutable references, which is UB. If an OS mutex can't provide that, rust can't use it. Double locking the mutex permits having unspecified behavior so long as the call doesn't return to maximize the odds that an OS has a usable mutex, even if that mutex emails your grandma before it ensures that a return isn't possible.
3 Likes
I think this is a perfect example of the difference of what "undefined behavior" means in C++ vs what it means in Rust docs.
In C++, this is undefined behavior for mutex precisely by virtue of the behavior not being defined in any way by the specs. The mutex docs tell you what the preconditions are. The docs don't say what can possibly happen in the case where the preconditions are violated, so by definition, the behavior is not well defined, and so it is undefined behavior. It's still undefined behavior even if they don't explicitly call it undefined behavior. Quote:
Undefined behavior may be expected when this document omits any explicit definition of behavior or when a program uses an erroneous construct or erroneous data.
Rust docs just say the magic words "there will not be undefined behavior" and somehow that is supposed to avoid undefined behavior as understood by Rust docs. So it's a different concept if at all meaningful.
I've heard this before several times in Rust forums - that it's some technical term that supposedly means something else than just behavior not being defined. But if it's borrowed from C++, then it can't mean something else because that's precisely what it means in C++.
I don't think that's it. rand outputs are not predictable and not reproducible (except if you seed it), but that doesn't make it undefined behavior.
2 Likes
E.g. if you can Enum with two values
fooandbarthen transmute fromOption<ENUM>to int (of the appropriate size) would always produce something, but it's not specified what exactly would it produce. On purpose. You are not supposed to rely on that.
transmute is unsafe. lock is not.
Some of it could be documented as "your program may deadlock, panic, or crash". Seems like a good enough spec if a program crash is indeed a possible option (maybe the docs should be OS-dependent somehow).
But if it can really corrupt your OS and all the programs running on it (including itself?), then that seems just as bad as any other undefined behavior, so I don't see how that can be considered a safe function.
2 Likes
It means the same as in C++: the entire behaviour of your program becomes undefined. It is technical, because people don't expect that the compiler can turn their entire program into a noop on an integer overflow, or remove infinite loops. People usually have some implicit baseline of program behaviour, and are very confused when that baseline is violated. For example, from a normal English language standpoint, "undefined behaviour", "unspecified behaviour" and "implementation-defined behaviour" are basically synonyms, but C++ draws subtle but drastic distinctions between them. They really should have chosen some less ambiguous and confusable terms, but this is the world we live in, and Rust follows the same terminology.
Note that C++ makes it so terribly hard to ensure memory safety, and the culture of the language is so ignorant of memory safety importance and strict safety invariants, that there are scarce few cases where C++ can guarantee "anything but UB". For that reason UB is usually the baseline of arbitrary behaviour. Rust, on the other hand, can provide strict mathematical guarantees of the absence of UB, and those guarantees must be always upheld in safe code, so that's what happens when a method behaves arbitrarily.
It's still essentially the same "anything goes" from the perspective of the end user. Don't double lock a mutex, anything can happen, just don't do it. But when you eventually mess up, you'll be grateful that at least it's not UB, and at least all other parts of your program behave as expected.
1 Like
How do I know other parts of the program behave "as expected"? That doesn't even mean anything to me if I don't know what this part of the program does. Maybe it starts changing all my global variables (with interior mutability) randomly? Previously I was expecting that those variables only change in the ways I wrote my code, but now I don't know. Who knows what to expect if the specs say I can't know what to expect?
2 Likes
If you are running an OS with cooperative multitasking and no preemption even for the OS, then a deadlock in user code is a deadlock of the entire system. Such OS's exist, and they are the ones that benefit most from having Rust, because they are running in constrained security-critical environments. Similarly, a deadlock may turn into a crash, if some hardware watchdog monitors the system and reboots it on deadlock (which is the way you should design such systems).
As usual, "safe" in Rust means "no Undefined Behaviour". Wiping your boot partition is safe. Deadlocking is safe. Doing an XSS injection is safe. That's always the caveat one keeps in mind when talking about Rust.
Firstly, if you double-lock a mutex, your program has a bug. Fix it. It doesn't matter that much what exactly happens.
But there are plenty of super important guarantees that just memory safety gives you. You can't corrupt the call stack or the unwinding tables. You can't deallocate a used buffer. You can't write to variables which don't give you their address.
If your program has UB, even in dead code, all other parts of your program can be modified in arbitrary ways. Take a look at the articles I linked above. The compiler causes to happen things which literally can't happen, if you just read the code, and it can propagate those horrible effects to arbitrary parts of your program. If booleans can be both false and true simultaneously, than no code you write can save you from arbitrarily horrible bugs. Your security checks are elided, your branches are selected at random, functions which are never called anywhere in your code can suddenly be executed. Anything can happen.
Lack of UB basically means that your bugs are at worst what you can get in Python or Java. In C++, the stakes are much, much higher.
I think all these cases are covered by just saying that it can deadlock.
It's understood that if you run your code on a cooperative multitasking OS, other programs may not get to run. This is not even specific to Mutex, an infinite loop will do it.
It's also understood that any program can be killed from the outside. This is also not specific to Mutex, an external hardware watchdog can kill you for any reason it wants.
2 Likes
I don't think concepts are being different simply because C++ developers are lazy and don't want to create a definitive list of undefined behaviours. C standards very explicitly include these lists (unspecified behaviors, undefined behaviors and implementation behaviors along with locale-specific behaviors). And it's hard to imagine C and C++ being entirely different given overlap in the groups that develop specifications, and, especially, compilers.
Both C and C++ quite explicitly separate unspecified behavior from undefined behavior, which would make no sense if they would have been considered equal.
It is really materially different from what C standard says about various functions? lrint, e.g.:
The
lrintandllrintfunctions round their argument to the nearest integer value, rounding
according to the current rounding direction. If the rounded value is outside the range of the return
type, the numeric result is unspecified and a domain error or range error may occur.
It doesn't say whether error would be detected or not, it doesn't say how would it manifest, it doesn't even promise not to send any letters to your grandma!
What's the material difference between that and about what Rust promises about Mutex?
They are predictable, though. When you can rand you can be sure that it would return certain number and wouldn't affect any other variable in the nearby vicinity. That's more than can be said about call to __builtin_unreachable.
No, it doesn't work like that. UB can time-travel (if you program is destined to tigger UB then it may behave erratically even before code which actually has UB is reached). But just having UB there which is never actually touched is safe. Otherwise any program which does x + 1 with a signed variable would have no meaning. Nor very useful language which behaves like that.
1 Like
rand has predictable behavior, it will return a random* value. Whether that random value comes from the hardware, from a seeded PRNG, or is just hardcoded to "42" the caller doesn't know because that return value is unspecified. What the caller can rely on is that rand won't cause memory corruption or cause the compiler to erroneously optimize out any branch where it occurs, or do anything other than return a random value. You can still meaningfully analyze by tracing possible control flow paths, or by the usage of probabilistic methods. All of that goes out the window if you actually trigger UB because triggering undefined behavior means your code can do anything at any time for any reason.
Unfortunately, no programming language or execution model can protect you from your own crappy code. However, so long as your code doesn't trigger undefined behavior, your code can still be meaningfully analyzed. You can still perform some sort of analysis to figure out what states your program can reach and what happens when it reaches a specific state. Your code, even if nondeterministic, will still be predictable, even if you aren't smart enough to be the one who does it. Again, if your program truly does have UB all this goes out the window, and anything can happen for any reason at any time.
*for some definition of the word random, bad rngs have caused numerous problems before
What if it doesn't just deadlock? What if it writes to logs, or an email to the admin? One could reasonably expect that locking a mutex has no other side effects, and can't do I/O, which can fail on its own and cause its own share of issues. What if a double lock causes the execution to jump to an interrupt handler? Sure, in principle someone could always overwrite your memory and cause those effects, but during normal execution it is expected that the control flow happens in specific ways, with specific effects, and everything else is outside the execution model. No one puts cosmic rays in the language spec.
But you keep missing the most important part. With unspecified behaviour, if a double lock never happens during the actual execution, then you're fine and dandy. No issues. With UB, the mere presence of a double lock in your source code, even in dead code or in unused functions, would be able to cause arbitrarily bad effects.
lrint does promise not to send any letters to your grandma. Note that it doesn't simply say that "lrint has unspecified behavior". What it says is that the result (i.e. the return value) is unspecified. What that means that the resulting long int can take any value from the list 0, 1, 2, ... etc, but it will be one of those. No other behavior is allowed here, no sending of letters to grandmas.
2 Likes
I'm not convinced that the documentation for lock allows the program to be meaningfully analyzed. It does list a couple of actions that the program could take, but does not claim that those are the only options.
I don't know about emails to the admin but if you would use TSAN then double lock would definitely write quite a lot of logs on the stderr.
No, no and NO. UB can time travel, but it can not act if it's not triggered! Practically any line in any C++ program can, potentially, trigger UB (given appropriate input). Even a[0] = 0 can trigger UB. Or x + 1. Or many other innocuous constructs. It doesn't matter how many potential UBs are in your code, if your program doesn't touch these — everything works spendidly.
Remember that crazy example:
typedef int (*Function)();
static Function Do;
static int EraseAll() {
return system("rm -rf /");
}
void NeverCalled() {
Do = EraseAll;
}
int main() {
return Do();
}
Can this code be fixed and work “normally”? Yes, sure.
Just add to another file:
extern void NeverCalled();
static class HoHoho {
Hohoho() { NeverCalled(); }
} hohoho;
And voila: perfectly “safe” and correct program. Well… safe for some definition of “safe”, I guess. It still calls “rm -rf”, but there are no UB and everything work correctly now.
I think it's outside of the scope of the Rust abstract machine. You can set up your system so that a program running for longer than 3 seconds will write to logs or send an email to the admin.
Then I don't see how this is a safe function.
That's not how UB works. Writing this code in C++ is not undefined behavior:
if (false) {
return 0 / 0;
} else {
return 5;
}
3 Likes
The undefined behavior in that code is derefering a null pointer in main, which is definitely not dead code.
3 Likes