Lifetime inference problems when creating a trait object / returning opaque type

I've got some traits set up to allow writing some text, as well as some parameters to a sink. The goal is to allow the types that write to the sink (Expressions) to write parameters as borrows rather than requiring cloning. The parameters are stored in the sink until all of the expressions have been written.

That all works fine as long as the types are static, but when I try to use a trait object things fall apart completely. [1] Since Expression has a lifetime on the trait, I believe the compiler is being forced to pick a lifetime earlier when the type is coerced to a trait object, but I'm not sure what I can do to resolve the problem.

Playground

Code
use std::{fmt::Display, ops::Deref};

/// A type containing a trait object, which can have a blanket impl for types implementing that trait.
pub trait ParameterRef<'p, T> {
    fn from_param_ref(param: &'p T) -> Self;
}

/// A parameter for types implementing [Display]
pub struct DisplayParam<'a>(&'a dyn Display);

impl<'a> std::fmt::Debug for DisplayParam<'a> {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.debug_tuple("TestParam")
            .field(&self.0.to_string())
            .finish()
    }
}

impl<'a, T> ParameterRef<'a, T> for DisplayParam<'a>
where
    T: Display,
{
    fn from_param_ref(param: &'a T) -> Self {
        Self(param)
    }
}

pub trait Meta {
    type Param;
}

/// A sink for strings and parameters. Parameters can be borrowed.
pub trait Write<'param>: Meta + Sized
where
    <Self as Meta>::Param: 'param,
{
    fn param_ref<T: 'param>(&mut self, param: &'param T)
    where
        Self::Param: ParameterRef<'param, T>;

    fn write(&mut self, sql: impl AsRef<str>);

    fn expr<'e: 'param, E>(&mut self, expr: &'e E)
    where
        E: Expression<'param, Self> + 'e + ?Sized,
    {
        Expression::to_writer(expr, self)
    }
}

#[derive(Debug)]
pub struct TestWriter<Param> {
    sql: String,
    params: Vec<Param>,
}

impl<Param> Default for TestWriter<Param> {
    fn default() -> Self {
        Self::new()
    }
}

impl<Param> TestWriter<Param> {
    pub fn new() -> Self {
        Self {
            sql: String::new(),
            params: Vec::new(),
        }
    }
}

impl<Param> Meta for TestWriter<Param> {
    type Param = Param;
}

impl<'param, Param: 'param> Write<'param> for TestWriter<Param> {
    fn param_ref<T: 'param>(&mut self, param: &'param T)
    where
        Self::Param: ParameterRef<'param, T>,
    {
        self.params.push(Self::Param::from_param_ref(param));
    }

    fn write(&mut self, sql: impl AsRef<str>) {
        self.sql.push_str(sql.as_ref())
    }
}

/// A type which can write to a writer, potentially borrowing parameters from itself to place in the writer.
pub trait Expression<'param, Writer>
where
    Writer: ?Sized,
{
    fn to_writer<'e: 'param>(&'e self, writer: &mut Writer)
    where
        Writer: Write<'param>,
        Writer::Param: 'param;

    fn boxed<'a>(self) -> Box<dyn Expression<'param, Writer> + 'a>
    where
        Self: Sized + 'a,
    {
        Box::new(self)
    }
}

impl<'param, T, Writer> Expression<'param, Writer> for Box<T>
where
    T: ?Sized + Expression<'param, Writer>,
    Writer: ?Sized,
{
    fn to_writer<'e: 'param>(&'e self, writer: &mut Writer)
    where
        Writer: Write<'param>,
        Writer::Param: 'param,
    {
        Box::deref(self).to_writer(writer)
    }
}

/// A sample expression which writes a static string and a reference to its inner parameter.
struct Test(usize);

impl<'param, W> Expression<'param, W> for Test
where
    W: Meta,
    W::Param: ParameterRef<'param, usize>,
{
    fn to_writer<'e: 'param>(&'e self, writer: &mut W)
    where
        W: Write<'param>,
        W::Param: 'param,
    {
        writer.write("TEST ");
        writer.write(self.0.to_string());
        writer.param_ref(&self.0);
    }
}

impl<'param, E, W, const N: usize> Expression<'param, W> for [E; N]
where
    E: Expression<'param, W>,
{
    fn to_writer<'e: 'param>(&'e self, writer: &mut W)
    where
        W: Write<'param>,
        W::Param: 'param,
    {
        let mut first = false;
        for e in self {
            if first {
                first = true;
            } else {
                writer.write(", ");
            }

            e.to_writer(writer)
        }
    }
}

fn main() {
    static_ty();
    dyn_ty();
    opaque_ty();
}

fn static_ty() {
    let mut writer = TestWriter::<DisplayParam>::new();

    let expr = [Test(1), Test(2), Test(3)];

    writer.expr(&expr);

    println!("{:?}", writer);
}

fn dyn_ty() {
    let mut writer = TestWriter::<DisplayParam>::new();

    let expr = [Test(1).boxed(), Test(2).boxed(), Test(3).boxed()];

    // error: `expr` does not live long enough
    writer.expr(&expr);

    println!("{:?}", writer);

    drop(writer);
}

fn opaque_ty() {
    fn opaque<'p, E, W>(e: E) -> impl Expression<'p, W>
    where
        E: Expression<'p, W>,
        W: Write<'p>,
        W::Param: 'p,
    {
        e
    }

    let mut writer = TestWriter::<DisplayParam>::new();

    let expr = [opaque(Test(1)), opaque(Test(2)), opaque(Test(3))];

    // error: `expr` does not live long enough
    writer.expr(&expr);

    println!("{:?}", writer);

    drop(writer);
}

The errors are

error[E0597]: `expr` does not live long enough
   --> src/main.rs:183:17
    |
183 |     writer.expr(&expr);
    |                 ^^^^^ borrowed value does not live long enough
...
188 | }
    | -
    | |
    | `expr` dropped here while still borrowed
    | borrow might be used here, when `expr` is dropped and runs the destructor for type `[Box<dyn Expression<'_, TestWriter<DisplayParam<'_>>>>; 3]`

error[E0597]: `expr` does not live long enough
   --> src/main.rs:204:17
    |
204 |     writer.expr(&expr);
    |                 ^^^^^ borrowed value does not live long enough
...
209 | }
    | -
    | |
    | `expr` dropped here while still borrowed
    | borrow might be used here, when `expr` is dropped and runs the destructor for type `[impl Expression<'_, TestWriter<DisplayParam<'_>>>; 3]`

For more information about this error, try `rustc --explain E0597`.
error: could not compile `playground` due to 2 previous errors

I've tried

  1. Using HRTB's on the trait object. That actually works! As long as you don't actually want to borrow anything from the expression.
  2. Removing the lifetime from Expression. I think this is essentially impossible for this use case. Not having a life time makes specifying a bound on what parameters an expression contains on an Expression impl basically impossible (again it actually does work as long as you don't need borrowing)

Obviously the simple solution is to just use Clone, but it's does work as long as trait objects aren't involved. That has me curious if there's something less extreme that might make this setup work that I've missed.

Thanks for reading!


  1. This turns out to also be true for opaque types implementing Expression ↩ī¸Ž

I think basically the lifetimes on the dyn Expression become entangled with that on the writer -- you have something like

[Box<dyn Expression<'param, TestWriter<DisplayParam<'param>>> + '_>; 3]

And then you call some method with both of

&mut TestWriter<DisplayParam<'param>>
&'param [Box<... 'param ...>]

(You made things more general, but in a way that allows borrowing the expression for longer than 'param, and thus can be coerced down to borrow for exactly 'param.)

And this is the speculative part (as I've never seen the rules written down), but I think the analysis behaves as both the writer can observe the expression for 'param (as it does in your implementation), but also that the expression can observe the writer for 'param as well (perhaps the erased type has interior mutability).

Then you get an unworkable situation where dropping the boxes can perhaps observe the writer, but the writer can't last longer than the things it's borrowing from.

Smaller example.

I think this backs up the interpretation -- the param-stealing Sneaky follows your declared API, but a lifetime "flows" from the writer to the expression.

2 Likes

That's sort of what I was afraid of.

Playing around with Sneaky, it looks like the error happens only if

  1. The type has a custom Drop implementation
  2. The type has some interior mutability
  3. The interior mutability "captures" the lifetime

It would be nice to be able to forbid interior mutability on the trait object (or on the lifetime) somehow.

That was very helpful, thank you!

I think that's correct, or rather, I think those are the conditions where it makes sense to conclude the shared reference might observe the writer on drop.

I'm mildly surprised the opaque impl Trait hides interior mutability; I would have thought that was a leaky property like variance or auto traits. Or maybe it doesn't hide it and (lack of) interior mutability just isn't explicitly taken into account for this scenario, I'm not sure. Could be interesting to play with.

(What would be even better though... would be a specification.)

1 Like