We have some amount of rust code in the project that is all accessible through C API. There are some possible panics in the code, that can only happen due to logic errors in rust code, so not that we need to refactor the code so that it returns Result instead.
In case of panic, I want to gracefully shut down the application showing a window with an error message. We already have the infrastructure for this: all of our C++ code is performed inside a try/catch block, and we show an error window if we caught an exception.
I see two possible solutions:
Wrap every exposed rust function in std::panic::catch_unwind and then return the error code to the calling C++ code in case of panic. That is what is the recommended way as I understand. But this involves modifying more code: every function should be processed separately both on C++ and on the Rust side.
Use std::panic::set_hook to catch any panic in rust code, and from this hook call C++ function that will raise the exception. This does not feel really safe, because C++ stack unwinding would have to unwind the rust stack, but from my experiments, it works. But this is one centralized point to handle errors, which is better than the previous solution.
I'm not offering a technical solution, but I can say that the kind of error handling you're doing is rather complex, and if it's an important part of your application, it deserves having the rust code expose Result even in case of logic errors.
Alternatively, if these logic error are quite infrequent, and you can test/detect them early, it should be OK to just let them panic/crash the program.
Problem is, as I've already mentioned, panicking over the FFI boundary is UB:
Rust's unwinding strategy is not specified to be fundamentally compatible with any other language's unwinding. As such, unwinding into Rust from another language, or unwinding into another language from Rust is Undefined Behavior. You must absolutely catch any panics at the FFI boundary! What you do at that point is up to you, but something must be done. If you fail to do this, at best, your application will crash and burn. At worst, your application won't crash and burn, and will proceed with completely clobbered state.
That is, if the function currently executed was called by C++, one must catch panic before returning from it, or it may corrupt the caller's state.
That is, if the function currently executed was called by C++, one must catch panic before returning from it, or it may corrupt the caller's state.
Actually, that is what is going on in my case. I set up a panic hook and from that panic hook, I raise a C++ exception preventing rust from actually panicking.
Not that I say that this is not a UB.
On the rust side, writing an appropriate attribute macro might easily avoid all the boilerplate. I'm not sufficiently familiar with C++ to determine what do to in the C++ side.
One thing that just came to mind: be careful when using catch_unwind in case you are dropping the panic payload; if you want full soundness, you need to consider the possibility that dropping the payload can panic again.