Is there an elegant way to handle None with a type that cannot be inferred?

code review
1

I'm trying to write a mildly flexible library function API that accepts an optional event handler parameter (currently an rxrust Observer, but I'm not married to that) -- so far without much luck.
Behold her glory:

pub async fn my_fantabulous_fn(
    opt_progress_observer: &mut Option<impl Observer<f64, std::convert::Infallible>>,
) -> anyhow::Result<()> {...}

The Observer expects floats that represent percentage progress events towards some goal. The calling client goes something like:

let task_progress_observer = Shared::subject();
let task_progress_subscription =
    task_progress_observer.clone().subscribe(move |v: f64| {
        tracing::info!(
            "Progress: {:.2}",
            v
        );
    });
let result = match my_fantabulous_fn(
    &mut Some(task_progress_observer),
)
.await {...}

Which works great! BUT what about a calling client who doesn't care about getting progress reports? Well...

let result = match my_fantabulous_fn(
    &mut None,
)

Seems like it should work, but fails with the error:

let result = my_fantabulous_fn(
     |                          -------------------- required by a bound introduced by this call
...
1759 |                 &mut None
     |                      ^^^^ cannot infer type of the type parameter `T` declared on the enum `Option`
     |
     = note: cannot satisfy `_: rxrust::Observer<f64, Infallible>`
     = help: ...
note: required by a bound in `my_fantabulous_fn`
    --> src\...
     |
1698 | ...pub async fn my_fantabulous_fn(
     |                 -------------------- required by a bound in this function
1699 | ...opt_progress_observer: &mut Option<impl Observer<f64, std::convert::Infallible>>) -> anyhow::Resu...
     |                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `my_fantabulous_fn`
help: consider specifying the generic argument
     |
1759 |                &mut None::<T>
                               +++++

I tried shoving the exact impl trait type signature into None as None::<impl Observer<f64, std::convert::Infallible>>, but that syntax doesn't compile. Trying dyn trait complains that my_fantabulous_fn doesn't want dyn anything, which brings us back to the dyn trait problem for the same API where I don't want to impose an exact type signature on callers.

The best 'solution' I can think of is to make a 'null Observer' type in the library that clients can use to satisfy Option::None's generic:

pub struct NoneObserver {}
impl Observer<f64, std::convert::Infallible> for NoneObserver {
    fn next(&mut self, _: f64) {}
    fn error(self, _: std::convert::Infallible) {}
    fn complete(self) {}
    fn is_closed(&self) -> bool {
        true
    }
}

and then the calling client can do:

let result = match my_fantabulous_fn(
    &mut None::<NoneObserver>,
)

However, that is unintuitive and awful. None should be synonymous with nullptr in my mind; it shouldn't need to know what it isn't! It also should be size 0 imo, but I gather it winds up with a size derived from the inferred type it could be but isn't. Is there any better way to make an Option wrap a 'placeholder' concrete type such as impl trait?

I can't have a concrete type with fully implemented behavior as the my_fantabulous_fn() param since that's a library function and has no idea what the calling client will need to do with progress report data, but the library function obviously needs to know what the input type will be so any concrete type would need to be implemented in the library crate... and we go 'round in design circles trying not to look at impl trait because of the None:: problem or dyn trait because of the strict type signature problem. Meanwhile, the Orphan Rule prevents the client from overriding the implementation of a function in a type owned by the library crate, so we can't get cute with behavior injection on the client side.

I could maybe do a concrete wrapper type that has a function that accepts a closure or something and otherwise does nothing, and use that to satisfy None's type inference, but that feels both ugly AND overcomplicated.

Am I missing any options for a better API design? I don't think I've come across any stdlib APIs that required me to manually specify the type of a None value, so I feel like I'm in antipattern territory. Maybe not, though?

2

to some extent, an Option argument type is similar to a boolean flag argument, sometimes it's more ergonamic to create a wrapper api (or several wrappers) on top of the actual implementation, and the caller can choose to call the more specific apis instead of passing an argument to affect behavior of a single api.

// the actual implementation, can be exposed to the user too, don't have to be private
fn my_fantabulous_fn_inner(opt_progress_observer: ...) {
    todo!()
}

// specialized wrapper apis
fn my_fantabulous_fn() {
    struct DummyObserver;
    impl Observer for DummyObserver {...}
    my_fantabulous_fn_inner(&mut None::<DummyObserver>)
}

fn my_fantabulous_fn_with_progress(observer: impl Observer) {
    my_fantabulous_fn_inner(&mut Some(observer))
}

btw, I just copied your original &mut Option<impl Observer>, but personally I would probably choose Option<&mut impl Observer> instead.

3

sigh me too, me too. But then this happens:

...
loop {
  if let Some(progress_observer) = opt_progress_observer {
      progress_observer.next(
          progress
      );
  }
  // or alternatively
  match opt_progress_observer {
      Some(observer) => {
          observer.next(
              progress
          );
      }
      None => {}
  }
}

gives me the error:

error[E0382]: use of moved value
    --> ....rs:2201:22
     |
2201 |                 Some(observer) => {
     |                      ^^^^^^^^ value moved here, in previous iteration of loop
     |
     = note: move occurs because value has type `&mut impl Observer<f64, std::convert::Infallible>`, which does not implement the `Copy` trait
help: borrow this binding in the pattern to avoid moving the value
     |
2201 |                 Some(ref observer) => {
     |                      +++

I'm not actually looping forever, but I do need to make use of the optional value or reference inside a loop. It seems that the syntax necessary to even examine an Option requires creating a new Option value and possibly moving the contained reference? Even the notion that a reference is moveable still gives me a headache, but I know Rust treats 'em like smart pointers to allow the borrow checker to work etc. etc. But anyway, how would one check for Noneness of the option without moving the Observer ref into an Option container destined for oblivion at the end of the loop iteration? Using ref as the compiler suggests doesn't work because I need a mutable ref, which is what the moved type already was and afaics there is no mut ref syntax for these semantics... why, Rust, why?

4

the mut reference is moved by default, you need to reborrow it in the loop. try:

if let Some(progress_observer) = opt_progress_observer.as_deref_mut() {
    //...
}

alternatively, use ref mut binding mode:

if let Some(ref mut progress_observer) = opt_progress_observer {
    //...
}

another alternative is to make use of the implicit binding mode of the match ergonomic mechanism:

if let Some(progress_observer) = &mut opt_progress_observer {
    //...
}
5

you need ref mut here, mut ref does exists as an unstable feature, but it does a different thing.

6

As a general rule, Option is often not a good way to express “optional parameters”. Alternatives include:

  • Have two functions, one with and one without the parameter. (For example, Vec::new() and Vec::with_capacity().)

  • In a case like this, where the argument could be of many types, pass a type which actually implements the trait but does nothing. (This is the “null object pattern” in design patterns.)

I see later in your post that you did that, writing a NoneObserver. But the trick is, if you do that, you don't need the Option at all!

pub async fn my_fantabulous_fn(
    opt_progress_observer: &mut impl Observer<f64, std::convert::Infallible>,
) -> anyhow::Result<()> {...}

...

my_fantabulous_fn(&mut NoneObserver)

Some traits have implementations like this already provided. Keep an eye out for traits being implemented for the () type, and for types called something like Null or Sink. (It doesn’t look like rxrust::Observer has such an implementation, though.)

It’s actually impossible for your case to be zero-sized, because &mut Option<T> would allow the callee to write a new Some(value_of_type_t) into that place. In order to make it actually zero-sized, you need to provide an uninhabited type parameter, so that Some cannot exist; if you wanted to do that with your existing code, you could do it by changing struct NoneObserver {} to enum NoneObserver {}. Then, None is the only value of type Option<NoneObserver> that can ever exist, so it will be zero-sized.

That said, the null object approach is, in my opinion, better in this case.

In this case, you want Option::as_mut(), not Option::as_deref_mut(). as_deref_mut() is for cases where the value in the Option implements Deref, e.g. going from &mut Option<Vec<T>> to Option<&mut [T]>, but it will fail to compile in this case since the observer does not implement Dereef.

7

Good alternatives have been suggested above, and this reply is mostly observations about how Rust works or why.

Note that impl Trait in arguments[1] is basically a[2] generic, and generics on functions have to resolve to concrete types in order to call the function.

Turbofishing dyn can work sometimes if the trait bounds can be met, but the other replies give better solutions so I won't explore that unless you really want me to.

(If Rust ever fixes defaulted generics on functions, that would be another possibility, but sadly they're largely useless and deprecated currently.)

Enums are more akin to (tagged) unions than (type agnostic) indirections. Which variant an enum is is a value consideration, not a type system consideration.

This just can't work with inline variants. Consider values in a slice or Vec (which has consistent stride), or *ref_mut_var = Some(thing) if the referred place was zero sized.

(Implicit in this section and the last: Rust doesn't box everything up behind an indirection, unlike some languages.)

It's ref mut to because you write & then mut for &mut _.

  1. but not in returns ↩︎

  2. hamstrung ↩︎

8

i would do this instead :

pub use core::convert::Infallible;

pub fn none() -> Option<Infallible> { None }

impl<T,U> Observer<T, U> for Infallible {
    fn next(&mut self, _: T) { match *self {} }
    fn error(self, _: U) { match self {} }
    fn complete(self) { match self {} }
    fn is_closed(&self) -> bool { match *self {} }
}

thhat way Option<Infallible> is indeed zero-sized

9

I wouldn't pass an Option<impl Observer>. I would create a FakeObserver or NullObserver which does indeed impl Observer, be zero-sized and do nothing.
And then simply pass an impl Observer.

10

I agree with previous posters that a NoneObserver (or NonObserver) implementation is a good fit in combination with not turning it into an Option at all. That latter part is important, as it gives you - basically - most of your property that it should be "like a null pointer"; if you’re passing a &mut NoneObserver, then that will be a pointer to some specific zero-sized object, and steps fo observing will still be no-ops.

Well… they’re no-ops unless you need some setup work that can be skipped if observation is completely impossible. If that really matters, you could work around this with some defaulted const IS_NON_OBSERVER: bool = false; in the trait, let the NonObserver make that true, and do a pre-check on this to avoid overhead.

Why I don’t like the Option in the first place: 2 reasons

  • the use-case where you do provide an observer requires you to always Some-wrap the thing
  • what is the point of being able to call this method with None::<SomeObserverType> vs. None::<OtherObserverType> at all?

If you do like to keep the Option anyway, others have already mentioned: empty enums, or Infallible can . Things will become slightly nicer in the future; when we get the never type ! stabilized, then the syntactic overhead for writing out the None::<!> is really getting quite minimal :ferris_happy:

I can additionally imagine that we could add a help message to such ambiguity errors could heuristically notices when an implementation of ! exists and seems reasonable to choose (especially for a None this may be the true). Maybe something like

help: the never-type `!` implements this trait, consider specifying `!` if you never need a value here
     |
1759 |                &mut None::<!>
                               +++++

and you'd be able to apply this easily through an IDE, too then :slight_smile:

11

What's the difference between an empty enum and an empty struct? Is there a difference between the following two declarations?

pub enum NoneObserver {}
impl Observer<f64, std::convert::Infallible> for NoneObserver {
    fn next(&mut self, _: f64) {}
    fn error(self, _: std::convert::Infallible) {}
    fn complete(self) {}
    fn is_closed(&self) -> bool {
        true
    }
}
...
pub struct NoneObserver {}
impl Observer<f64, std::convert::Infallible> for NoneObserver {
    fn next(&mut self, _: f64) {}
    fn error(self, _: std::convert::Infallible) {}
    fn complete(self) {}
    fn is_closed(&self) -> bool {
        true
    }
}

My plan is to change the API to something like what nerditation recommended above:

fn my_fantabulous_fn_inner(opt_progress_observer: Option<&mut Observer>) {
    // actual behavior that unpacks the optional observer
}

// specialized wrapper apis
fn my_fantabulous_fn() {
    struct NoneObserver;
    impl Observer for NoneObserver {...}
    my_fantabulous_fn_inner(None::<&mut NoneObserver>)
}

fn my_fantabulous_fn_with_progress(observer: &mut impl Observer) {
    my_fantabulous_fn_inner(Some(observer))
}

Could also just eliminate the Option always have the inner function interact with the input impl Observer, relying on NoneObserver to be a no-op.

The more important question is whether NoneObserver and/or None::<NoneObserver> would be size 0? What would calling e.g. opt_progress_observer.next() on a NoneObserver presumably of size zero do at runtime? Hopefully nothing, but based on the discussions above it seems like we're in weird territory where there's potentially nothing being allocated, yet there can still be interaction with an instance of something...?

12

This is where things get a little weird and unintuitive (even if they are internally consistent), so let me approach it from a couple angles. One way to look at a struct is that it is a lot like an enum with exactly one variant. That is, if you have:

struct Foo { x: bool }

and

enum Fooish { Foo { x: bool } }

then they are identical in many ways; in particular, they can be constructed and destructured in the same ways, and they have the same number of possible values (two). From this, we can say something about what an empty struct is like: it's like an enum without fields.

struct EmptyStruct {}
enum EmptyStructish { Empty {} }

This type has exactly one possible value (which takes up zero bytes of memory). An empty enum goes further: it has zero possible values. This means that an empty enum cannot be constructed, because in order to construct it, you would have to pick a value to construct, but there are no values for you to pick.

Getting back to the original use case, we can say something simple about how many values Option<T> has: it always has one more than T (the None value). Therefore, if you put an empty enum in Option, there are 0 + 1 = 1 possible values: only None. Thus, if you have a generic Option<T> that, in a particular case, should always be None, using an empty enum is a way to accomplish that.

On the other hand, if you don’t use an Option (as we have recommended), then you need the empty struct, because you need to have one possible value to pass.

In the first case, the methods of the Observer implementation cannot be called, because there is no possible self value to pass in. In the second case, they can be called, and it is up to the implementation to actually do nothing.

You should not use both Option and the empty struct, because then you have two possible values: None and Some(NoneObserver {}). There should be only one possible value, and the most straightforward way to do that is to not use Option. That is, you should “just eliminate the Option”.

  • An empty struct has size 0.
  • An empty enum has size 0.
  • An Option of an empty enum has size 0.
  • An Option of an empty struct has size 1, because it could be either None or Some(MyStruct) and so we need a byte to store that distinction.
  • A reference to any of these things is nonzero size, because every reference points to some location in memory (even if zero bytes of relevant data are stored there).

The way you should think about this is that values can require any number of bytes of memory to store them, even zero. A value having size zero does not mean that its existence is less real than otherwise; only that it is cheaper to keep around.

(It’s a little weird that zero-sized values still have specific locations in memory, but doing something else would create more problems than it solves.)

13

Fascinating analysis, thank you! One more deep dive question: if I have e.g.

#[derive(Default)]
pub struct NoneObserver {}
impl Observer<f64, std::convert::Infallible> for NoneObserver {
    fn next(&mut self, _: f64) {}
    fn error(self, _: std::convert::Infallible) {}
    fn complete(self) {}
    fn is_closed(&self) -> bool {
        true
    }
}

an instance of NoneObserver constructed by NoneObserver::default() would be size zero (if I understand correctly). Where does the no-op implementation code go in this case? I guess the larger question is how do member functions or implemented interfaces affect the size of a given type? Going by the size_of() docs I suppose they don't, but seeing an empty struct with potentially non-empty/non-no-op behavior confuses me way more than good ol' nullptr :sweat_smile:.

14

Yeah, they don't. Non-empty implementations don't change the size of structs, either. Rust structs don't carry around all their methods in a vtable,[1] if that's what you were thinking. Methods are invoked directly, not through a vtable (except in the case of dyn Trait, which the compiler implements).[2][3] The (empty) function code does exist in the executable somewhere, at least notionally. (In this case one expects/hopes that they are optimized away completely).

  1. much less a copy of all the code ↩︎

  2. even with dyn Trait, the vtables are in static memory and just pointed to, so adding methods doesn't make it more expensive to move around a Box<dyn Trait> or whatever ↩︎

  3. Your Observer trait is not dyn-compatible as-is, so there will be no related vtables at all. ↩︎

15

Note the ironic duality: for me an idea that there may be an abstract None entity not attached to any type is equally strange and unsettling…