Let x be some C library that cannot safely be called from multiple threads (due to globals and/or use of functions like strtok).
Suppose I make unopinionated x-sys and/or x-src crates to link the library and directly expose the C API.
I also make an x wrapper crate with a more opinionated API. This crate hides the functionality behind a static mutex to make it safe.
Now suppose somebody else has different opinions and makes a crate called ecks which independently wraps x-sys. Under the hood, this crate uses it's own method of synchronizing calls to the API to be safe.
Somebody inadvertently depends on both x and ecks, and it ends up getting used by two threads simultaneously. Oops.
This makes it seem wrong for the x wrapper crate to label its API as safe, because I cannot prove that a crate like ecks does not exist. It seems that I am doomed to have to label virtually all code that uses x as unsafe, all the way up to an unsafe block somewhere in main.rs.
How is this solved? Perhaps x-sys should provide it's own "official" API for synchronization to allow x to coordinate with other unknown wrapper crates?
Even if all users use your opinionated wrapper, a separate crate, y, that uses strtok could cause a lot of problems for you.
I think you would be justified and calling your wrapper crate safe. No matter what you do, it'll probably be possible to use it incorrectly if I try hard enough. Even if x-sys provided some sort of synchronization, there's nothing stopping me from including my own handwritten FFI bindings - maybe I only need one or two functions and I decide that it's not worth the trouble of adding a dependency on x-sys.
A non-thread-safe C API can never be "Safe" in the sense you've mentioned, but, your wrapper API can be "Safe" in the sense that all the unsafety has been isolated to the boundaries of the FFI and that you've prevented the object as wrapped from being "Send" (so it can't be sent to another thread in Rust). That being said, if some other crate can still use that C-API behind your back directly in another thread, it has the potential to create all kinds of UB that could lead to security issues. At the end-of-the-day, what this means is that any Rust code that ulitmately includes C code that is "unsafe/ub" in the truest sense, can never be "Safe" truly. But, the unsafety will be isolated to the wrapper boundary and the C-API so you'll know where your issues could arise. That's the best you can do I think.
What this points out (to me) is that spending too much time in the Rust community wrapping C-API's is at best a stop-gap measure and ideally more work should be put into pure Rust solutions in order to achieve a "Safe" ecosystem.
Hmm... forgive me for stepping into the land of unicorns, but: What if there was a way to guarantee that a piece of code was only run in a single-threaded context?
If it were possible, I think it would allow some limited subset of thread-unsafe C-APIs to be safely wrapped; namely, those where all thread unsafety is encapsulated within a call to the API. So we still couldn't safely wrap a library like ARPACK which remembers global state across calls, and we'd have to be really careful around APIs that can take callbacks; but it's better than nothing.
Granted, I can't even begin to imagine how such a feature could be backwards-compatibly retrofitted into the standard library. Even if there were theoretically a function
// Returns Some only when there is a single thread of execution,
// and prevents the creation of new threads until all guards are dropped.
// ...somehow.
fn ensure_single_threaded() -> Option<SingleThreadedGuard> { ... }
...then it's not clear how it would work, because there is nothing in the type system to prevent people from simply spawning a thread after calling the function. (and go figure, std::thread:spawn doesn't even return a Result, so the only way to prevent calling it at runtime is to make it panic!).
My inner unicorn thinks the portability lint could accomplish this... if the lint supports a kind of negative reasoning I'm not sure anyone's even suggested before. Which doesn't seem completely insane for a lint. Maybe.
My usual solution to this is to have a 'static AtomicBool which will be set to true in my library wrapper's constructor (which is !Send) and then set back to false in the destructor. Here's an example of wrapping libmagic as part of my FFI guide. This way you can at least ensure only one thread can use your library at a time.
You can then combine this with the links = "..." field in your Cargo.toml to ensure your crate is the only one to link to that specific native library. cargo should then make sure it's impossible for your x-sys and ecks to be used in the same binary.
I think this is about the only way to prevent the issue you're talking about. Other than that, like most unsafe code, a lot of the responsibility lies on the developer to ensure their code is correct.
But, if C lib a (your lib that you're wrapping) and C lib b both use a non-thread-safe C API under the hood (like strtok), you're still up sh-t crick despite that as far as potential UB/unsafety is concerned. Wrapping C libraries with Rust can never be truly safe. The entirety of the C code (and all the C code it calls) needs to be treated as one humongous "unsafe" block. How do you make sure that what is in your "unsafe" blocks in Rust is actually safe/doesn't create UB? You audit it thoroughly. Without such a thorough and ongoing audit (every time any of the C code changes), you have to consider the whole program "unsafe". That is why, to truly get the benefits of Rust you can't wrap C-libs. That is why I said that it really isn't a way forward for the Rust ecosystem to spend a lot of capital on wrapping existing C (and other non-safe language) libraries.
EDIT: This reminds me of OS/2. In my opinion, it failed because it spent so much "capital" on trying to be 100% compatible for existing DOS/Win3.11 that enough effort wasn't put into the actual OS/2 ecosystem and so when Win95 arrived on the scene, which could also run the existing DOS/Win3.11 stuff, OS/2 had no competing already existing ecosystem of software and developers and so had no compelling use case.
I would be hesitant to make such sweeping generalisations. Just because a language may call a function which may cause data races, doesn't mean we should boycott the entire language. Instead you try to recommend/use libraries which are thread-safe by default and over time the ecosystem will evolve and start doing things more safely. Indeed, even the man page for strtok() has big warnings saying it's not thread-safe and you should use strtok_r() if thread-safety matters to you.
I like the idea of RIIR as much as the next guy, but if there are already loads of really high quality C libraries in public use it seems silly to not take advantage of all those thousands of man-years of work...
You just use a common locking mechanism. Like @Michael-F-Bryan mentions. (So maybe -lock crate needs to go with any such -sys)
The safety problem is when a wrapper (with there being more than one) does not make use of it. Finding what is unsafe in the lib whole different game.
strtok can also fail even in single threaded use (more race condition rather than data race.)
I don't see how that would work or be helpful. The c-code of both libs is capable of doing whatever it wants and using whatever C-apis it wants, including spawning threads and using thread-unsafe API's (the OP's original point). It is impossible to block that from happening in Rust. You can never have a safe Rust program that relies on unaudited (for safety/lack of UB) C-code. Static analysis tools for C can help with this, perhaps looking for known "unsafe" C-apis and such, but, at the end-of-the-day, you cannot have a safe Rust lib/binary that relies on unaudited C-Code and call it safe. The best you can do, is consider all the C-code a huge "unsafe" block and treat it as such. The rest of your Rust program can be "safe", but, you still have a huge block of unaudited "unsafe" code (the C-code) unless you explicitly audit it for safety/lack of UB. There really is no other option (other than just punting and relying on the fact that the c-code "has many eyes on it" - just like OpenSSL did for years - oops, "Heartbleed").
I don't think this is what the topic OP is covering. I don't want to argue over the potential of a C lib failing to meet safety internally. It is a matter of trust that the API supplies a contract of how to be used safely. (e.g. not to be called from multiple threads, specific functions to only get called when in specific state.)
Individually the wrappers meet the contract so supply the safety. The issue is maintaining the contract even when the unknown (3rd party) has API access.
Keeping a wrapper as unsafe isn't really practical. It all comes down to trust. (In the extreme you ban all foreign code but still also have to audit rust for any unsafe; When your coding to such extreme even safe code needs audit.)
I wasn't suggesting that. In fact, I said that the "wrapper" can and should be "safe" (in the sense that it doesn't need to propagate unsafety throughout the Rust code), but, as the OP states, it can never truly be "safe". "unsafe" keyword denotes blocks of code where unsafe things can occur that can result in UB. It is up to the "safe" Rust function wrapping the unsafe block (and C-FFI is just one huge "unsafe" block) to ensure whatever contract the "unsafe" code relies upon is upheld. Whether the "unsafe" code itself actually is safe or not (as long as the contract is upheld) is only known through auditing it.
Sure it is. He specifically mentions thread-unsafe API's (like strtok) in C being called by unrelated libraries having actual UB/unsafe effects on one another despite any efforts one can take in Rust to wrap a particular C lib in a "safe" wrapper.
So... I don't think either of you are wrong per se. These are solutions to two different problems:
A library which is thread-unsafe, but would be 100% safe if the calls were synchronized.
A library which is thread-unsafe, and is still unsafe even with synchronization (due to thread-unsafe C standard library functions that could interfere with other C libs).
My own personal problem is of the second variety. However when I wrote the OP, the issue of other libraries using cstdlib didn't occur to me, which is why I initially described it as a problem of the first variety.
Just as an aside, regarding this:
It is a matter of trust that the API supplies a contract of how to be used safely.
...I wish mine did.
A lot of mathematical and scientific code is poorly documented with regards to things that we consider fundamental basics, such as who allocates this and whose job is it to free this pointer? (or what function must I use to free the pointer?) A couple of weeks ago I was trying to wrap SuperLU and eventually gave up all due to just this.
I think @gbutler69 has an interesting perspective on the matter, and I think it's the only perspective I've seen so far that can really justify the (desirable) outcome of labeling my wrapper crate as safe. Though, taking this perspective even further, it seems to suggest that the wrapper crate does not even need to provide its own synchronization. (I guess this is true as the synchronization ultimately doesn't solve the problem, but I still feel like it ought to make the best effort that it can!)
But there's some fuzzy bits: Suppose for instance that libx is dynamically linked; then different versions may be used on different systems. In this case, there is not one single body of source code to be audited. I'm not sure what to make of this, just food for thought I guess...
I came to a sort of a conclusion about this while writing a PM to somebody. FFI has nothing to do with it.
We already know that unsafe abstractions do not compose in general. This is to say that, if you have two crates that each provide a safe wrapper around unsafe functionality, and each one is correct in isolation, it may be the case that using them together can create UB in safe code.
If every unsafe abstraction had to worry about the theoretical existence of another unsafe abstraction which violates it's assumptions, crates like rayon and rental couldn't exist. So until there is a way to solve the greater problem of crate incompatibility, it seems that the most that a crate must be obligated to do is to guarantee that it's API is safe in isolation. (with the standard library, it's dependencies, and those it is advertised to work with)
