Struct containing a generic Box<dyn Fn> as static

[arturo]

Is there any way or workaround to do something like the following:

pub trait StaticTrait: 'static{}

pub struct Func<T> {
    f: Box<dyn FnMut(&T)>
}

impl<T> StaticTrait for Func<T>{}

That code fails cause in order to be static Func<T> needs T to also be static but since T is only used as a parameter to the dyn FnMut, T doesn't really need to be static for the struct to be static and indeed this equivalent code works:

pub struct FuncU8 {
    f: Box<dyn FnMut(&&u8)>
}

impl StaticTrait for FuncU8{}

Is there any way to implement a trait for a generic struct that would allow this?

Rust Playground

[alice]

No. When you write T: 'a, you are saying that T has no lifetime annotations shorter than 'a. Your definition of the trait implies that anything that implements it is 'static, so it has no lifetime annotations shorter than 'static. However, Func<&'a T> has a lifetime annotation, so it cannot implement the trait.

[jbe]

@alice

However, Func<&'a T> has a lifetime annotation, so it cannot implement the trait.

I don't think the 'a in &'a T is the problem. It's T itself:

pub trait StaticTrait: 'static{}

pub struct Func<T: 'static> {
    f: Box<dyn FnMut(&T)>
}

impl<T> StaticTrait for Func<T>{}

(Playground) works, even though the lifetime of &T can be very short.

But the following fails:

pub trait StaticTrait: 'static{}

pub struct Func<T> {
    f: Box<dyn FnMut(T)>
}

impl<T> StaticTrait for Func<T>{}

(Playground)

So it's not the lifetime (annotation) of the reference. It's the requirement that T must be a valid type for the runtime of the entire program.

[alice]

The problem is that if Func<SomeTypeGoesHere> has any lifetime annotations when you write out SomeTypeGoesHere, then Func<SomeTypeGoesHere> is not 'static — this fact is true regardless of the fields inside the struct. When I mentioned &'a T, that was just an example type with a lifetime annotation.

When you mention the reference inside the Box directly, its different because the lifetime on the reference does not become part of Func itself. To see this, note that

pub struct Func {
    f: Box<dyn FnMut(&u32)>
}

is short-hand for

pub struct Func {
    f: Box<dyn for<'a> FnMut(&'a u32)>
}

In particular, it is different from this:

pub struct Func<'a> {
    f: Box<dyn FnMut(&'a u32)>
}

This would not compile either. However, when you try to put the reference in the generic, you get something equivalent to my last snippet.

[quinedot]

@arturo

That code fails cause in order to be static Func<T> needs T to also be static but since T is only used as a parameter to the dyn FnMut, T doesn't really need to be static for the struct to be static and indeed this equivalent code works:

It's not equivalent. Here:

pub struct Func<T> {
    f: Box<dyn FnMut(&T)>
}

T represents a single, concrete type.

But here:

pub struct FuncU8 {
    f: Box<dyn FnMut(&&u8)>
}

The inner &u8 is not a single, concrete type. It's a type constructor that needs a concrete lifetime in order to resolve to a single, concrete type (&'a u8, ..., &'static u8). Types that differ only in lifetime are still distinct types.

---

dyn FnMut(&&u8) can still be 'static because it is higher-ranked over its lifetimes. The Fn traits (and fn pointers) use syntactic sugar to elide the lifetimes here, but spelling it out the field actually has the type:

Box<dyn for<'a, 'b> FnMut(&'a &'b u8) + 'static>

The for<'a, 'b> is the higher-ranked part. This type as a whole is concrete -- it doesn't take any "outside" lifetime or type parameters. But the references inside are still type constructors and not concrete types, due to the for<'a, 'b> binder.

---

In contrast, the type of the field in Func<T> is

Box<dyn for<'a> FnMut(&'a T) + 'static>

This is still a type constructor (as a whole) that takes a type parameter T. Its region of validity is restricted to areas where all of its parameters are also valid, so Box<dyn FnMut(&T)>: 'static only when T: 'static.

Because type parameters can only take on "values" of concrete types, there's no way to assign something to T to get from this Box<dyn FnMut(&T)> to Box<dyn FnMut(&&u8)>. Or to put it another way, Rust does not have generic type-constructor parameters.

There is also no for<'any> &'any u8 higher-ranked reference type in Rust you could assign to T.

So, they are not equivalent.

[H2CO3]

What I guess OP doesn't understand (and to be frank neither do I completely) is why T in the function argument type influences the validity of the function type itself.

It is entirely conceivable to create a function that lives (and is valid to store) for the 'static lifetime, even though references to its argument don't satisfy 'static. After all, only storing the function doesn't by itself require having access to a witness (value) of its argument type – that's only needed for actually calling it.

I would think this property is universal in that i doesn't depend or whether a function is generic or if it desugars to a HRTB, because it's true no matter what the specific types and lifetimes involved are. It seems to me that if this were the case, then creating global (and thus statically-living) fn items that take short-living reference arguments would be impossible.

Accordingly, I think it is a reasonable expectation that dyn FnMut(WildTypeWithFunkyRefs) also be 'static, provided that the underlying function object doesn't capture anything non-'static (i.e., it's either an fn item or a closure that doesn't capture short-living references).

It seems to me like the requirement you mentioned is of purely "syntactic" nature – not in the sense of source text syntax per se, but in the sense of the mathematical structure describing a type. I.e. it is imposed only because it is easier/better to implement a general, uniform rule for checking well-formedness of generic type constructors, one of the form "F<T>: 'a only if T: 'a". This doesn't seem to be necessary unless F<T> contains a T.

This situation is similar to #[derive] macros that aren't sufficiently smart, and impose a T: Trait bound on the generated impl even if the implementing type doesn't contain a T and thus doesn't need it. However, it is perfectly possible to impl Clone for F<T>: even though T: !Clone, as long as F<T> doesn't contain instances of T. It feels like the same should also apply to lifetime bounds.

I'd be happy to be proven wrong, because this is an interesting aspect of the behavior of function types and generics.

[quinedot]

@H2CO3

It seems to me like the requirement you mentioned is of purely "syntactic" nature – not in the sense of source text syntax per se, but in the sense of the mathematical structure describing a type. I.e. it is imposed only because it is easier/better to implement a general, uniform rule for checking well-formedness of generic type constructors, one of the form "F<T>: 'a only if T: 'a". This doesn't seem to be necessary unless F<T> contains a T.

Yes, that's correct. The syntactic rules were laid out in RFC 1214; you can also read there how the rules used to allow things like fn(&'non_static u8): 'static (due to some sort of reasoning about "reachability"). There is occasionally a push to get that expressiveness back for at least function pointers.

If that was extended to structs it would probably be some sort of per-field property, and would probably make their validity region leaky like variance. ^[1] For dyn Trait then... you can't really tell if the parameters are "reachable" in the type-erased base. But perhaps the lifetime of applicability is sufficient? I'd have to think it over and look into those past soundness issues that helped motivate RFC 1214.

---

1. It's already a little leaky due to inferred outlives. ↩︎

[VorfeedCanal]

@H2CO3

What I guess OP doesn't understand (and to be frank neither do I completely) is why T in the function argument type influences the validity of the function type itself.

The short version: because rustc doesn't use chalk.

The story here is the following. If T: static then this:

pub struct Func<T: 'static> {
    f: Box<dyn FnMut(&T)>
}

desugars to this:

pub struct Func<u32> {
    f: Box<dyn for<'a> FnMut(&'a u32)>
}

But what would happen if T is not : 'static ? Suddenly we need to desugar it to something like this:

pub struct Func<&'a u32> {
    f: Box<dyn for<'a: 'b> FnMut(&'b &'a u32)>
}

But that's not valid Rust and. more importantly, I suspect that it cannot be expressed in the rustc! Thus we need T: 'static and all that silliness.

The same thing which plagues GATs (and makes them much less useful than it feels on the first glance), more or less.

@H2CO3

After all, only storing the function doesn't by itself require having access to a witness (value) of its argument type – that's only needed for actually calling it.

Yes, but storing function for longer than it's type can exist is not, really, useful. You cannot call it when it's type would stop being valid, anyway. And I'm pretty sure when rustc does various checks it's just not ready to process something which includes type which is no longer valid. What would happen if that Func<T> would implement drop and then would try to use type which is no longer valid from it?

What you really want are bounds on HRBTs… but these don't exist, they can not be expressed in Rust (and probably can not be expressed in rustc).

@H2CO3

It seems to me that if this were the case, then creating global (and thus statically-living) fn items that take short-living reference arguments would be impossible.

These can exist wiithout complex HRBTs, though: all types are available to reason about when you see declaration of such function.

@H2CO3

I'd be happy to be proven wrong, because this is an interesting aspect of the behavior of function types and generics.

I think you are correct: of course if you say that your StaticTrait is 'static then you cannot implement it for Func<T> if T is not 'static.

But in many cases you have to mark StaticTrait as 'static not because it have to be static, but because aforementioned HRBTs limitations make it impossible to express it's requirements otherwise.

[quinedot]

@arturo

Is there any way to implement a trait for a generic struct that would allow this?

Your example is a little too simplified for me to be sure about what you really want, but following up on what I wrote before: if you wish you had something like a type constructor parameter, you can sometimes use a bound on a GAT-carrying trait (or emulated GAT on stable for now) in place of a generic type constructor parameter. However it is verbose, otherwise unergonomic, and tends to completely wreck inference... so may or may not be worth it (if it is even something that applies to your use case).

[H2CO3]

@VorfeedCanal

Yes, but storing function for longer than it's type can exist is not, really, useful. You cannot call it when it's type would stop being valid, anyway. And I'm pretty sure when rustc does various checks it's just not ready to process something which includes type which is no longer valid. What would happen if that Func<T> would implement drop and then would try to use type which is no longer valid from it?

To be clear, that's not what I meant. I have no intention to hold on to functions with an invalidated type. What I meant was that in general a (non-capturing) function's full type itself is logically valid indefinitely (and thus satisfies 'static), because it doesn't itself store or refer to any data that is non-'static`. Eg. an fn item of the form

fn greet<T: Display>(name: &T) {
    println!("Hello, {}", name):
}

has no reason not to be 'static regardless of T, because the fn item itself doesn't store any instances of T in itself (even if the body of the function does create or refer to such values when invoked). And it is definitely useful: whenever you have a value: T of arbitrarily short lifetime, you can call greet(&value). This is in fact how the majority of functions is used currently.

I'm imagining the situation w.r.t. Drop that you described should be something like the following?

struct Foo<T: Default>(PhantomData<fn() -> T>);

impl<T: Default> FnMut(&T) for Foo<T> {
    fn call_mut(&mut self, arg: &T) {}
}

impl<T: Default> Drop for Foo<T> {
    fn drop(&mut self) {
        let value = T::default(); // BOOM
    }
}

This can only happen with funky user-defined types, and it won't ever happen with compiler-generated fn items and non-capturing closures. The problem then is to differentiate between a generic callable coming from a compiler-generated value and a generic callable coming from an arbitrary untrusted UDT. I think unsafe marker traits could technically be used for this purpose, but that's probably way more complication and special-casing than warranted.

[H2CO3]

@quinedot

There is occasionally a push to get that expressiveness back for at least function pointers.

Nice, although I have to admit in the 6 or so years I've been seriously using Rust, I never needed this (it hasn't even come up to make me even think about the issue so far). So it's probably not going to be high priority even if agreed on.

[arturo]

Yes that was the kind of workaround I was expecting, as you said it's really unergonomic but I imagined any workaround would be something like this. Thanks!

[VorfeedCanal]

@H2CO3

This is in fact how the majority of functions is used currently.

You are mixing implementation details and the language model.

@H2CO3

What I meant was that in general a (non-capturing) function's full type itself is logically valid indefinitely (and thus satisfies 'static), because it doesn't itself store or refer to any data that is non- 'static`.

No. It's not logically valid indefinitely and it's not static. Let's expand your example:

fn greet<T: Debug>(name: &T) {
    println!("Hello, {:?}", name);
}

fn greet_greet<T: Debug, Greet: Fn(&T) -> ()>(name: &T, greet: Greet) {
    greet(name);
    greet(name);
}

fn test<'a, 'b>(x: Wrap<'a, i32>) {
    greet_greet(&x, greet)
}

#[derive(Debug)]
struct Wrap<'a, T> (&'a T);

Here we pass the address of the function into another function and everything seems to work. But what if we would add 'static requirement?

error[E0521]: borrowed data escapes outside of function
  --> src/lib.rs:14:5
   |
13 | fn test<'a, 'b>(x: Wrap<'a, i32>) {
   |         --      - `x` is a reference that is only valid in the function body
   |         |
   |         lifetime `'a` defined here
14 |     greet_greet(&x, greet)
   |     ^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     `x` escapes the function body here
   |     argument requires that `'a` must outlive `'static`

Code no longer compiles.

What happens here? Function with type which doesn't exist for a'static lifetime doesn't have one, single, 'static type! Instead it represents an infinite amount of types!

Logically it's a bunch of functions with a bunch of different types. The fact that they all are satisfied by the exact same machine code is an implementation detail. If you would ask the compiler to make a type for that function 'static… it cannot do that.

[H2CO3]

@VorfeedCanal

You are mixing implementation details and the language model.

Lifetimes are definitely an "implementation detail" in this regard. They matter and they need to be annotated because of how much low-level control the language is willing to give the programmer. There would be no need for lifetimes if the language managed all memory by e.g. garbage collecting everything. In this regard, lifetimes are a leaky abstraction (albeit a very useful and desirable one).

@VorfeedCanal

Code no longer compiles.

That is not a valid argument, because we are trying to establish whether it should compile. Besides, if you replace T with the more concrete Wrap<T> in the signature of greet() (note that this doesn't change anything because the T parameter was instantiated with Wrap<_> in your original code, too!), then it still compiles, even with the 'static bound added (probably because i32: 'static is a concrete type, so it now transparently satisfies the 'static requirement).

@VorfeedCanal

What happens here? Function with type which doesn't exist for a'static lifetime doesn't have one, single, 'static type! Instead it represents an infinite amount of types!

Logically it's a bunch of functions with a bunch of different types. The fact that they all are satisfied by the exact same machine code is an implementation detail. If you would ask the compiler to make a type for that function 'static… it cannot do that.

It's not about the machine code; you don't need to think about the machine code at all in order to see why this works. The "bunch of different types" is a red herring, too: values can only be instantiated from types, not type constructors, so if you have an fn<'a>(&'a T), you have to substitute something for 'a and T anyway before being able to refer to it as a type (on which to apply a generic bound) and declare values of that type.

This in turn means that an fn item with a reference-typed argument is actually a bunch of different fn items, and if the compiler wants to reason about what lifetime (or other) bounds the type of such an item can satisfy, it has to pretend it's got all type and lifetime (and const…) arguments bound anyway.

IOW it's simply good old-fashioned universal quantification: for every lifetime 'a and every type T, the concrete, instantiated fn item greet::<'a, T> satisfies typeof(greet::<'a, T>): 'static, even though it might be hard to express using the current well-formedness check of the compiler.

---

FYI, There's a much more convincing but completely unrelated reason why this might be impossible to relax: Any::downcast_ref() relying on 'static-fulfilling types being a single, concrete, and thus unique type. As demonstrated in the linked comment, this assumption is too simplistic, because function types violate it; however, Any and downcasting already relies on it for soundness, so it looks like we are stuck with it.

[VorfeedCanal]

Thanks for writing that. At least now I understand why it's so damn hard in Rust to do many things which are very simple and obvious (note: no sarcasm and no quotes).

Rust essentially repeated the story of C… I just don't know whether to laugh or to cry.

@H2CO3

Lifetimes are definitely an "implementation detail" in this regard.

That's what C developers traditionally have been feeling about types. And if you consider types “an implementation detail” then it's really hard to understand what's the issue with type punning.

And, in fact, in languages like Forth83 or B (which, after long evolution, have become C and then C++) type punning is nothing special, types there are, indeed, just “an implementation detail”.

But of course it's almost impossible to create modern, optimizing, compiler if you consider types an implementation detail, const an implementation detail and so on.

On the contrary, modern compilers consider the fact that both int and float are mapped to the 32bit memory quantity (machine word on 32-bit architectures) as merely “an implementation detail”!

Language deals with types and not with machine words!

@H2CO3

There would be no need for lifetimes if the language managed all memory by e.g. garbage collecting everything.

Yes, but that would be entirely different language. It wouldn't have RAII, e.g.

It fact it's RAII that needs lifetimes, not memory management. The fact that Rust uses RAII for memory management, too may be considered an implementation detail: if drop wouldn't free memory, but would leave it to GC then you wouldn't even know it… but without lifetimes you cannot, e.g., guarantee that you wouldn't be able to access variable before it would be initialized:

    let x;
    println!("Before: {x}");
    if … {
      x = 42
    } else {
       x = 0
    }
    println!("After: {x}");

How would you make that first println! illegal without lifetimes? While keeping 2nd one legal?

@H2CO3

In this regard, lifetimes are a leaky abstraction (albeit a very useful and desirable one).

I suspect that at least some developers think like you do. That's why we have trouble with specialization, trouble with GATs, trouble with traits which we are discussing here…

I was always thinking about this rule (liefetimes can not affect runtime behavior) similarly to problems monomorphization is causing: very annoying implementation detail which makes nice things impossible.

But you are right: because that rule does exist we can treat lifetimes as annoying markup on top of a nice type system.

Frankly, I wouldn't want that, but I can see why people may want it to be treated that way.

@H2CO3

Besides, if you replace T with the more concrete Wrap<T> in the signature of greet() (note that this doesn't change anything because the T parameter was instantiated with Wrap<_> in your original code, too!), then it still compiles, even with the 'static bound added (probably because i32: 'static is a concrete type, so it now transparently satisfies the 'static requirement).

Oooh. Nice. I guess that is why we cannot just go and fix Rust's type system: this wouldn't be a backward compatible change.

I would consider the fact that this code compiles very much a problem (we are making something which deals with transient type 'static how can that work?), but yeah, I get your point.

@H2CO3

IOW it's simply good old-fashioned universal quantification: for every lifetime 'a and every type T, the concrete, instantiated fn item greet::<'a, T> satisfies typeof(greet::<'a, T>): 'static, even though it might be hard to express using the current well-formedness check of the compiler.

I would consider it a bug in the compiler, sorry. Because it makes reasoning about what is possible or what is not possible with types pretty convoluted. The idea that a certain type (and then, object of some time) may exist for longer than types of it's parameters just makes no sense to me.

@H2CO3

FYI, There's a much more convincing but completely unrelated reason why this might be impossible to relax: Any::downcast_ref() relying on 'static-fulfilling types being a single, concrete, and thus unique type. As demonstrated in the linked comment, this assumption is too simplistic, because function types violate it; however, Any and downcasting already relies on it for soundness, so it looks like we are stuck with it.

I wonder if something like Rust 2.0 where types would become in a somewhat sane fashion (not source-compatible but somewhat link-compatible) is possible. Because, frankly, what Any does looks sane to me and these relaxations for functions don't look sane… but if I understand correctly we cannot just fix it: to make Rust's typesystem sane and stop that madness we need to declare constructs which today are acceptable as invalid… and provide a replacement (most likely by lifting that “lifetime bounds cannot affect runtime behavior”)… and when code “before” and “after” cannot the same… it's very much a definition of a breaking change.