Second-order factory pattern and static lifetimes

I've been playing with what I call a second-order factory pattern, i.e., a factory which creates another factory. However, I appear to be stuck due to a lifetime problem I do not understand. The "final" factory (the "production chain" in my example) returns an Output with a lifetime of 'static (ie, owned data), which does not prevent Rust from giving me the good old E0515: "cannot return value referencing local variable materials".

So:

  • I would like to understand what is going on
  • I would like to convince the compiler that no, the output does not, in fact, reference anything since it's owned data
  • I would also like to know if there is a better/simpler way of architecturing the code (ie, is Buildable really necessary here?)

Here we go:

use async_trait::async_trait;

// Something which can produce raw materials
struct Mine;

impl Mine {
    fn extract_materials(&self) -> RawMaterials {
        RawMaterials
    }
}

// Raw materials to produce all widgets
struct RawMaterials;

// Available widgets
struct BlueWidget {
    engraving: String,
}

struct RedWidget {
    engraving: String,
}

// A way to customize widgets with a specific engraving
trait Blueprints {
    fn engraving(&self) -> String;
}

struct BlueWidgetBlueprints {
    engraving: String,
}

impl Blueprints for BlueWidgetBlueprints {
    fn engraving(&self) -> String {
        self.engraving.clone()
    }
}

struct RedWidgetBlueprints {
    engraving: String,
}

impl Blueprints for RedWidgetBlueprints {
    fn engraving(&self) -> String {
        self.engraving.clone()
    }
}

// A container for the mine.
struct Builder {
    mine: Mine,
}

trait Buildable<'a>: Blueprints {
    type Factory: Factory<'a, Self::Output, Blueprints = Self>;
    type Output: 'static + Send + Sync;
}

// A factory for widgets. Factories need different production chains for different widget types. A
// factory can setup production types for multiple widget types.
struct WidgetFactory<'a> {
    materials: &'a mut RawMaterials,
}

#[async_trait]
trait Factory<'a, Output: 'static> {
    type Blueprints: Blueprints;
    type ProductionChain<'b>: ProductionChain<'b, Output>
    where
        Self: 'b;

    async fn new(materials: &'a mut RawMaterials) -> Self;
    async fn setup_production_chain<'b>(
        &'b mut self,
        blueprints: Self::Blueprints,
    ) -> Self::ProductionChain<'b>;
}

#[async_trait]
impl<'a> Factory<'a, BlueWidget> for WidgetFactory<'a> {
    type Blueprints = BlueWidgetBlueprints;
    type ProductionChain<'b> = BlueWidgetProductionChain<'b> where Self: 'b;

    async fn new(materials: &'a mut RawMaterials) -> Self {
        Self { materials }
    }

    async fn setup_production_chain<'b>(
        &'b mut self,
        blueprints: Self::Blueprints,
    ) -> BlueWidgetProductionChain<'b> {
        BlueWidgetProductionChain {
            materials: self.materials,
            blueprints,
        }
    }
}

#[async_trait]
impl<'a> Factory<'a, RedWidget> for WidgetFactory<'a> {
    type Blueprints = RedWidgetBlueprints;
    type ProductionChain<'b> = RedWidgetProductionChain<'b> where Self: 'b;

    async fn new(materials: &'a mut RawMaterials) -> Self {
        Self { materials }
    }

    async fn setup_production_chain<'b>(
        &'b mut self,
        blueprints: Self::Blueprints,
    ) -> RedWidgetProductionChain<'b> {
        RedWidgetProductionChain {
            materials: self.materials,
            blueprints,
        }
    }
}

// Production chains for widgets. Production chains actually produce a widget, based on a set of
// blueprints.
struct BlueWidgetProductionChain<'a> {
    materials: &'a mut RawMaterials,
    blueprints: BlueWidgetBlueprints,
}

struct RedWidgetProductionChain<'a> {
    materials: &'a mut RawMaterials,
    blueprints: RedWidgetBlueprints,
}

#[async_trait]
trait ProductionChain<'a, Output: 'static> {
    async fn build(&mut self) -> Output;
}

#[async_trait]
impl<'a> ProductionChain<'a, BlueWidget> for BlueWidgetProductionChain<'a> {
    async fn build(&mut self) -> BlueWidget {
        BlueWidget {
            engraving: self.blueprints.engraving(),
        }
    }
}

#[async_trait]
impl<'a> ProductionChain<'a, RedWidget> for RedWidgetProductionChain<'a> {
    async fn build(&mut self) -> RedWidget {
        RedWidget {
            engraving: self.blueprints.engraving(),
        }
    }
}

impl<'a> Buildable<'a> for BlueWidgetBlueprints {
    type Factory = WidgetFactory<'a>;
    type Output = BlueWidget;
}

impl<'a> Buildable<'a> for RedWidgetBlueprints {
    type Factory = WidgetFactory<'a>;
    type Output = RedWidget;
}

impl Builder {
    async fn build_widget<B>(&self, blueprints: B) -> <B as Buildable>::Output
    where
        B: for<'t> Buildable<'t>,
    {
        let mut materials = self.mine.extract_materials();
        let mut factory = B::Factory::new(&mut materials).await;
        let mut production_chain = factory.setup_production_chain(blueprints).await;
        let result = production_chain.build().await;
        // This compiles fine when replacing `result` with `todo!()`
        result
    }
}

fn main() {
    let builder = Builder { mine: Mine };
    let _widget = builder.build_widget(BlueWidgetBlueprints {
        engraving: "Your first name here".to_string(),
    });
    let _widget = builder.build_widget(RedWidgetBlueprints {
        engraving: "Your last name here".to_string(),
    });
}

I think this is a case of an HRTB causing confusing error messages. If you force the Output to be a single type by adding it as a type parameter on build_widget it compiles

Playground

async fn build_widget<B, Output>(&self, blueprints: B) -> <B as Buildable>::Output
    where
        B: for<'t> Buildable<'t, Output = Output>,
        Output: 'static,
    {
        let mut materials = self.mine.extract_materials();
        let mut factory = B::Factory::new(&mut materials).await;
        let mut production_chain = factory.setup_production_chain(blueprints).await;
        let result = production_chain.build().await;

        result
    }

The difference here is that it constrains Output to be a type that doesn't depend on 't since the type is defined outside of the HRTB. As for exactly why that produced that error message despite the 'static bound on Output, I'm not sure.

5 Likes

Thanks, this was amazingly quick, and it indeed compiles.

Nice workaround.

When I encounter something with this much lifetime speghetti, I like to

#![deny(elided_lifetimes_in_paths)]

And we can immediately see that the full signature of the problematic method is

    async fn build_widget<B>(&self, blueprints: B) -> <B as Buildable<'_>>::Output
    // New --->                                                      ^^^^
    where
        B: for<'t> Buildable<'t>,

Which means the API says you're depending on the lifetime of the &self borrow for the return type ... but that's not at all true in the actual code. You're actually depending on the borrow of the locally created materials (a borrow which cannot be named in the API). That's where the error is ultimately coming from, I believe.

The workaround "lifts" the Output to be independent of the lifetime of B as Buildable<'_>, at which point it doesn't matter what lifetime is in the return type.


You should perhaps try to refactor things so that the trait bounds in the workaround are inherent in your traits.

4 Likes

Nice tip with the deny, I'll remember that for next time.

You're actually depending on the borrow of the locally created materials (a borrow which cannot be named in the API).

I'm not! That's why I specified that it has a 'static lifetime. The fact that the compiler does not realize that looks like a bug.

You should perhaps try to refactor things so that the trait bounds in the workaround are inherent in your traits.

I'm not sure what you're suggesting here. My original design included different builder traits, but did not have Buildable. I then got stuck with multiple HRTB bounds, with the compiler unable to reconcile them (because it won't let you write for<'t> (A: X<'t>, B: Y<'t>), but only for<'t> A: X<'t>, for<'t> B: Y<'t>, and then there is no way to express that the first 't is the same as the second 't). Using a GAT for the second-order factory and introducing Buildable let me work around the issue (a single trait bound also means a single HRTB). I'm open to something less complex, but I must admit that I was that close to giving up on it and just going with a macro...

I meant type-wise.

a       let mut materials = self.mine.extract_materials();
b       let mut factory = B::Factory::new(&mut materials).await;
c       let mut production_chain = factory.setup_production_chain(blueprints).await;
d       let result = production_chain.build().await;
  • _: &'s Self == &'s Builder
  • a: RawMaterials, about to be locally borrowed for 'x
  • b: <B as Buildable<'x>>::Factory about to be locally borrowed for 'y
  • c: <<B as Buildable<'x>>::Factory> as Factory<'x, B::Output, B>>::ProductionChain<'y>
  • d: The output parameter of <<<B as Buildable<'x>>::Factory> as Factory<'x, B::Output, B>>::ProductionChain<'y> as ProductionChain<'y, B::Output>, i.e. B::Output

Everywhere I wrote B::Output is really <B as Buildable<'x>>::Output. If those four lines compile, you definitely have a <B as Buildable<'x>>::Output there.

But what you want the compiler to understand is that it's the same as a <B as Buildable<'s>>::Output. Or even more, that <B as Buildable<'a>>::Output is always the same as <B as Buildable<'b>>::Output.

I think that has to be true today, due to the 'static bound. But some people want fn(Cell<&'not_static ()>) to be 'static, for exaple, which would make it possible to be not true.

If we decide that's never going to be a thing, then it's "just" a leap of logic that the compiler doesn't currently perform, I think.

Either way, you're making it explicit (and required) with the workaround.

    async fn build_widget<B, Output>(&self, blueprints: B) -> <B as Buildable>::Output
    // Must be a single type ^^^^^^
    where
        B: for<'t> Buildable<'t, Output = Output>,
// Same across all impl lifetimes ^^^^^^^^^^^^^^
        Output: 'static,

And my suggestion was to try and make this part of your traits so you don't have to add the extra bounds on your inherent methods. For example, are you ever going to have a Buildable that is not for<'any> Buildable<'any>? If that's not functionality you need, you can

-trait Buildable<'a>: Blueprints {
-    type Factory: Factory<'a, Self::Output, Blueprints = Self>;
+trait Buildable: Blueprints {
+    type Factory<'a>: Factory<'a, Self::Output, Blueprints = Self>;
     type Output: 'static + Send + Sync;
 }

- async fn build_widget<B, Output>(&self, blueprints: B) 
-  -> <B as Buildable<'_>>::Output
+ async fn build_widget<B>(&self, blueprints: B) 
+  -> <B as Buildable>>::Output
     where
-        B: for<'t> Buildable<'t, Output = Output>,
-        Output: 'static,
+        B: Buildable

As now a Buildable must define a Factory for every lifetime, and a singular Output type that cannot see the lifetime parameter of the Factory. The higher-ranked bound and the need to lift the Output out of the lifetime parameter's scope to make it definitely and explicitly singular both go away.

Playground.

1 Like

That's a good suggestion. I need to play with that a bit, but it could work for my use case. Thanks!