Some things I learnt in the last few weeks

As a rust beginner, I thought I would post some things I have learnt recently, which I think are useful, but they are not covered in "the book" ( as far as I know ).

(1) traits can have a supertrait.

(2) mem::replace for extracting a value from a struct accessed by a mut reference.

(3) The crate cap for checking for memory usage and leaks.

(4) Panics can be caught, and may be worth considering as an alternative to Result.

(5) self can be declared in many different ways, for example as self: &Rc<Self>

Is there anything useful you have learnt about Rust recently ( or not so recently )?

7 Likes

Cool idea for a post. I hope Jon (and his publishers) won't get too mad about me sharing a small quote from the first chapter of their book, but here it goes nevertheless:

Newer Rust developers are often taught to think of lifetimes as corresponding to scopes: a lifetime begins when you take a reference to some variable and ends when that variable is moved or goes out of scope. That’s often correct, and usually useful, but the reality is a little more complex. A lifetime is really a name for a region of code that some reference must be valid for. While a lifetime will frequently coincide with a scope, it does not have to.

A 'static lifetime, thus, doesn't represent a lifetime of a reference to a value that has to live for the entire program (as many people think, and you can't really blame them - as the static keyword is used to create static variables that are valid for the whole length of the program), but a lifetime of a reference that has to be valid for an arbitrary length of time.

This is especially important to understand when dealing with threads: whenever you try to pass a usual reference to a new thread that you're creating, the compiler will scream at you that it doesn't live for a 'static length of time. But that doesn't mean that you can only pass to it values that exist for the entire span of the program - it only means that they have to live long enough for the new thread to deal with whatever values are behind that reference.

Which is why you're usually required to pass an Arc to a new thread - it's a smart pointer that keeps track of how many references (shareable between threads, thus the name - atomic) were shared so far, and when all of them get dropped, the original reference gets dropped as well.

That was quite an eye opener.

Also, there's an awesome crate called tap which can be used to chain several methods, applicable to the same object. Quite a useful thing to keep all of your manipulations of the same data to just one piece of code, instead of going for o.do_a(); o.do_b(stuff); o.do_another_c(); and so on.

7 Likes

When it comes to the 'static lifetime, it is worth to distinguish between &'static T and T: 'static. The former really does mean that the value lives forever, but the second one doesn't. The meaning of the second one is that it would not be invalid if it lived forever (but it doesn't have to).

13 Likes

I wasn't aware of this crate. Thanks!

Yes they can be, but not necessarily reliably.
So they're not a true alternative IMO.

2 Likes

I don't think there is anything unreliable about it. It does preclude turning off unwinding ( "abort on panic") if you want the panics (errors/exceptions) to be caught and processed by the program ( rather than the invoker of the program ). It was discussed here.

1 Like

It's precisely the fact that it can be turned off that makes it unreliable as an error recovery mechanism.

In addition, if there's anything that Java's exceptions have taught me, it's that control flow is simply the wrong mechanism to leverage for error recovery.
While in theory it could work fine, in practice pretty much every place I've seen it used it just turns into a catch-all mess. No precision to be found whatsoever.

If I ever saw panics used for error recovery rather than fatal errors in a Rust project I cared about, I'd immediately refactor that to use Result values. It's quite literally what they're there for.

10 Likes

It's what you do when a fatal error occurs that is the issue. If you are running a web server ( for example ) you may want to do various things : if the logged on remote user is an engineer ( who has just made some silly mistake ) you tell them what the mistake was, if it's an end user you want to report an error occurred, log the error, maybe send an email to notify an administrator there is a problem. You probably don't want to stop and restart the web server entirely.

Using a fatal error for such use cases is indicative of a behavioral modeling issue.
Fatal errors are, by their very definition, (supposed to be) fatal i.e. the process halts immediately in a controlled fashion.
If any other actions are (potentially) desired or required in response to an error occurring, then that leads right back to the Result type.

This is also the reason that libraries in particular should always favor Result over fatal errors: consumers of the library are stuck when such a panic occurs. I had that issue once with the zmq crate. The only solution at the time to keep things workable was forking the project and changing the code myself until the issue in zmq was resolved.

Maybe you could explain to me why you'd combine fatal errors (i.e. panicking) with attempted error recovery in the first place, when Result is available?

2 Likes

Except for cases where it's baked directly into the compiler.

How many newcomers are aware of the fact that println! will panic without a second thought if there's something doesn't go right? And that the safer alternative is write!() / writeln!()?

There are even more subtle examples - one of which was brought up by Torvalds back in April.

This was another thing I discovered about Rust in the last week, by the way - despite all the safety baked in the use of Option and Result some parts of Rust as the language seem to disregard quite important, albeit rather uncommon scenarios, as `let's just panic here at this point' instead of a proper option/result-oriented feedback from an underlying function.

Arguably even there.
As the issue mentions, it is very surprising behavior, and as Torvalds mentions, it can be quite unacceptable (e.g. how do you deal with OOM in Rust 2021? answer: you can't, not really). I know I was surprised to learn that the allocator can fail, yet the API suggests no such thing. I also understand the tradeoff - Result is infectious - but the argument can easily be made that such a fallible API is merely reflective of the complexity of the problem being solved. When viewed through that lens, using panic was a mistake even in stdlib.

Given the 2021 status quo, the only way to deal with that is catching such baked-in panics, so an exception has to be made there, for the sake of pragmatism.

However I was aiming mainly at user-defined library code, and to a lesser extent at binaries, but not really at stdlib.

Yes, this is what I was talking about earlier in this post.
If they had to be redesigned (which they might to some degree, in order to facilitate integration into the Linux kernel), my opinion would be that the API would have to reflect such fallibility, even if it means that all the collection types have to have their APIs updated as well.
I personally think it was a mistake to just panic at various places in stdlib.

In a library, you would usually not want to raise an exception, unless there is no prospect of continuing execution. For example, a function to compute the inverse of a matrix, if it's not possible, that would be an Err result, as the caller may well want to handle the error ( without catching a panic ). On the other hand, if a simple contract is broken ( an index out of range in an array access for example ), I would say that it is quite reasonable to make that an exception ( if the caller wants to avoid the possibility of the exception he has a simple recourse - it's a matter of convenience ).

But anyway, why do you think Panics can be caught in Rust? Do you think every program handling panics has it all wrong? I don't think so!

1 Like

Well, exceptions in Rust don't exist, and panics are not exceptions. The thing they have in common is that they use control flow, but that alone doesn't make them the same.

Aside from that, I agree. In fact that was exactly the point I was making with the zmq example: there was a match expr in one of the functions that defaulted to a panic. And since it didn't properly enumerate all possible cases, my code hit exactly that panic, hence the necessity of the fork.

Because given the stability guarantees of Rust, someone got backed into a corner, and that became a valid use case given that context.

For each usage where modeling it with Result instead is a viable option, I'd say yes, that's wrongly modeled. But I also think that won't hold for 100% of the code out there.
Still doesn't mean one should reach for panics when Result values will do though.

Here's a rough analogy: Rust has both functions and methods. So when one defines a new type, they have a choice: do they model its behavior with methods, or with free-standing functions? I'd argue one should first reach for methods because they are not only somewhat syntactically more convenient, but also make it clear that there is an inherent connection between the type and the method. Only when that doesn't work should one reach for a free-standing function in such a case.

What difference between them did you have in mind?

1 Like

Exceptions have an associated object that can be manipulated when they're caught.
For example:

class Foo {
    void foo() {
        throw Exception("whoops");
    }

    void bar() {
        try {
            this.foo();
        } catch (Exception e) {
            // stuff can be done with `e`
        }
    }
}

Haven't seen panics allow that thus far, not that I'm advocating for that.

fn panicky() {
    std::panic::panic_any("hello!".to_owned())
}

fn main() {
    if let Err(payload) = std::panic::catch_unwind(panicky) {
        println!("{}", payload.downcast::<String>().expect("wrong payload"));
    }
}

The type of payload is Box<dyn Any + Send + 'static>, which is not too different from an opaque Exception.

5 Likes

Then it seems that, when used as a recoverable fatal error (still a contradictio in terminis if you ask me), panics really do almost behave like exceptions. The almost comes from the switch allowing the recoverability to be turned off, which I still contend makes them the less reliable option.

I guess I would say that "panic = fatal error" is a (strong) convention, not something built into the language. You can certainly argue that because of various properties of panics they are best suited to error conditions that are not expected to be handled further up the call stack.

FWIW, GCC also supports the equivalent of panic = "abort" for C++ exceptions. Well, it's not exactly equivalent—passing -fno-exceptions causes try { ... } catch and throw to be compile errors.

I think there are valid use cases. If you have a thread-pool for example, you can catch panics in order to avoid losing (and needing to re-create) any worker threads. If you have a long-running application, you can add a recovery system, catching panics if any occur, to avoid shutting down the service – even if panics are never actually expected to happen. I.e. if your program happens to have a logic bug somewhere triggering a panic in certain situations, catching a panic allows you to avoid a small bug taking down the whole application.

Also, threads. If you spawn a thread, a panic from this thread will just kill that single thread. You can observe this from a different thread via the JoinHandle. Hence, even without catch_unwind, one could use threads for a (less efficient) way to catch panics. In fact, thread::spawn uses catch_unwind internally (here). There’s not really any good alternative option since unwinding can’t work across threads for technical reasons, so there always has to be a way to catch panics in Rust, if you want Rust to support unwinding. And unwinding itself is useful to free up resources properly with destructors.

That’s another big use-case for catching panics: Free up some resources, then re-throw the panic. Admitted, it’s nicer to use destructors for this usually, but – again – for multi-threading, using .join().unwrap() is the way to get parent threads fail together with their child threads in a way that cleans up resources, and – as mentioned – catch_unwind is used internally in order to achieve this.

I suppose, a similar use-case is (manually) propagating unwinding across FFI-boundaries. Maybe – again – spawning threads qualifies as a special-case of this since it’s using e.g. pthread_create on Linux, and you wouldn’t want the callback passed to pthread_create to unwind across FFI-boundaries. I can imagine other FFI-bindings involving callbacks would need to do a similar thing and use catch_unwind inside of the callback, and then perhaps even re-throw the panic at a different place.

5 Likes

I think a lot of these valid uses can be simply restated:

When a library calls into long-running user code, it should catch_unwind at the transition point.

For Java developers, think about when you would } catch (Throwable t) { and actually have a recovery plan: exactly the same use cases.

Well, in terms of implementation they're using exactly the same stuff as C++ exceptions, so...