Hello ,
I'm trying to understand the new precise capture syntax. To my understanding it has been introduced to opt-out of the new all parameter capture behaviour - the new default with Edition 2024. When playing around with it I noticed a few things I don't understand.
I created an (non-compiling) example on the rust playground for reference:
In the current state Rust requires you to mention all type parameters which may/can or may/can not be lifted in the future, but what I don't understand right now based on the example:
Why do you have to mention Self in a associated method? The opaque type can never capture Self (it does not take a self reference)
Why does Rust require me to mention it in the method declaration inside the trait use<'a, Self> (even on associated methods, previous point), but forbids the exact same signature on the Impl (i.e. you must define the method with use<'a>, not use<'a, Self>:
error[E0799]: Self can't be captured in use<...> precise captures list, since it is an alias
(See compilation error output you get when you hit run in the playground example)
What about cases there the returned opaque type has no relation to Self or any T? As I understand you're currently unable to express that.
How should trait authors know which lifetimes (and maybe at some day type parameters) will be part of the final opaque type? It sounds like a big restriction on the number of possible trait implementations, because it restricts you to use only specific parameters in the returned type. You probably can't forsee which parameter will be choosen in a implementation and the alternative would be to again capture everything, which in turn restrict what people can do with parameters passed into the trait method. (*e.g. in the playground example you can't return b inside use_t).
The RFC documents that the Self type as a use<> parameter means it may capture the generic arguments to the self type in the implementation - nothing about the & self lifetime (that's essentially use<'_>.
As with other trait definition bounds it is an upper boundary on what the implementation can declare, and doesn't make much sense in the context of the implementation itself (where it would imply it can capture the Self type itself, which doesn't make much sense)
This explains why it can be used in associated methods too, since implementation generic parameters are also available for associated methods.
It's not clear to me what you're asking specifically with:
If you just mean describing the Rust 2021 default that no generics parameters (including implicit lifetime) are captured, that's use<> (read, "capture nothing")
The last point is pretty much why the default changed to capture everything: it's better to assume that an implementation should be able to use every parameter in scope by default. If you're choosing as an author of a trait to restrict that, it's because that has some meaning in your case. In general, restricting allowed implementations of an interface means relaxing the allowed uses of an interface.
For capture, this is most obvious in that you might want to not capture a lifetime for some desired usage: eg an Iterator like interface that can return items that outlive the iterator itself needs to exclude '_ as mentioned.
Do you mean a Self in use<> bounds has a complete different meaning than it has everywhere else?
Why wouldn't this make sense? A method can of course have a self: Self receiver and pass selfdown to the returned opaque type.
I'm not sure what you mean with upper bounds. If you declare a bound, I used to interpret them as a lower bounds. Your opaque type must at least satisfy these and your Impl is not allowed to return a type without these Traits. You can add additional ones, but that currently produces warnings and would further restrict the returned type:
I mean that you can have methods which neither need to capture Self or the type paramter.
For example:
get_test() neither uses Self nor does it use T, but you are required to mention T because it appears as a generic parameter on the impl block.
If you now call get_test() and want to change T afterwards you get an error, because Rust assumes the ot is still in use. The compiler says:
41 | ot.set_value(&mut *other_num_ref);
| ------------------- `*other_num_ref` is borrowed here
42 | *other_num_ref = 42;
| ^^^^^^^^^^^^^^^^^^^ `*other_num_ref` is assigned to here but it was already borrowed
43 | }
| - borrow might be used here, when `_a` is dropped and runs the destructor for type `impl Test`
but _a has no connection with ot at all. Rust still seems to capture Self. (The example compiles if you comment out let _a = ot.get_test(); or replace impl Test with the concrete type HelloPrinter).
Where the opaque associated type is parameterized by all the captured lifetimes at the use site(s).
And in a generic context, the returned type -- the opaque's path -- is
<ImplementingType as Trait<'_, B>>::Opaque<'_, '_, Z>
Note that all the generic parameters from the trait, the implementing type, and the use sites are present in this path. Those are the generic parameters that might end up being part of the definition of the concrete type of the opaque in an implementation. They are the parameters you can name at the associated type location.
If you use precise capturing, you can remove some of the parameters of the opaque:
We've removed the ability to name some of the parameters at the associated type location.
You can't opt out of capturing Z so far, but I could see that being lifted in the future.
You can't opt out of capturing trait parameters or the implementing type (Self). Those are part of the type's path, and completely forgetting them tends to cause problems, including soundness holes.[1] (Beyond that, what would replace the path? How are the definitions from multiple implementations kept in sync? Etc.) So I think that restriction on the trait definition (where you must capture trait parameters and Self) will be around indefinitely.
It can capture generic parameters that are part of the implementing type, even if they're not present in the trait or method parameters. (See also my next paragraph.)
I don't know the exact or technical reasons why you can't use Self instead of spelling out all the parameters which are only Self in the implementation.[2] But note that if you impl<V> ... for Vec<V>, you do have to capture the V, for example.
For lifetime parameters which are part of the implementing type or trait parameters -- those required to be captured by in the definition -- you can either capture them in the use<> or you're writing a refined implementation.
I could see refinement being able to not capture V eventually.
As the trait author, you can use a normal associated type with a bound instead of -> impl Trait for that today. But it does mean that implementers have to use a type they can name, which may mean type erasing[3] unnameable types (like compiler generated futures or closures).
But the plan is that you will have more options in the future, like I've talked about above. It's just taking time to get there.
For example, one thing implementers will be able to do is write a refined implementation which may be able to normalize to something that is outside of the trait but still opaque (so they can still return something unnameable). But that won't necessarily help in a generic context.
Another planned feature is to be able to use impl Trait in your associated type definitions, which will also remove some of the restrictions on the implementers when the trait author chooses to use associated types instead of -> impl Trait.
trait Trait<'a, B> {
// Doesn't capture lifetimes from the method or `Z`.
// (`'a`, `B`, and the implementing type are still nameable here,
// and are still part of this associated type's path.)
type Return: Useful;
fn method<Z>(&self, _: &i32, _: Z) -> Self::Return;
}
impl<T> Trait<'_, T> for Vec<T> {
type Return = impl Useful; // <--- the feature in question ("ITIAT")
fn method<Z>(&self, _: &i32, _: Z) -> Self::Return {
// ...
}
}
That's just a specific case of the more general "how do I design a useful API". It's not specific to opaque types -- e.g. what parameters do you put on your associated types, if any? You have to commit to some vision of how the tool will be held at some point.
Some parts of the language get to assume implied bounds from the well-formedness of types in a path hold, for example, and erasing parameters can change those implied bounds. ↩︎
"something something Self is like a generic parameter in the definition but is completely normalized in the implementations something something", but that's just a vibe I have. ↩︎
Quinedot beat me to all the interesting stuff and better, but to address a couple minor points:
Apparently so... but so does use in use<> vs use std::io. Don't worry too much about the syntax here, but as Quinedot said you can think of Self in a trait as an implicit type parameter and you get close enough to it's meaning here.
Regarding bounds, yeah, sorry I was just trying to draw a parallel too strongly there and just ended up confusing the issue. What I was getting at was that in general precise captures in trait impls don't need to capture everything that the trait decl does:
What you're running into with that last example isn't that the reference to Self captures &self - AFAICT it's that it captures T, that T contains a reference, and that there's only one T so there's only one lifetime captured by that T. I poked at the code a bit, but I can't see a good way to express what you're trying to do there in Rust at the moment, perhaps others can?
If we're talking about generic consumers, e.g. a change at the trait so that capturing a lifetime in the implementing type is impossible in any implementation, I think the only thing that can help the example is a lifetime bound that rules out capturing (probably a 'static bound). If we're talking about non-generic consumers, there's other choices.
Let's look at some alternatives.
Refinement, which the author mentioned. Can only be at the implementation (doesn't help generic consumers). You can use a newtype to encapsulate SemVer hazards.
Lifetime bound on the opaque. Could be at the trait, or could be used with refinement only (with the same restrictions). The T cannot be completely erased (it's still part of the opaque's path), but it effectively won't participate in borrow checking.
Type erasure. Could be at the trait, or could take the place of a newtype at the implementation only (using refinement, with the same restrictions), if the trait author implemented Trait for Box<dyn Trait + '_>. The choice of the dyn lifetime effectively takes the place of a lifetime bound.
fn get_test(&self) -> Box<dyn Test + 'static> {
Box::new(HelloPrinter)
}
Use an associated type instead. Requires changes to the trait. Implementers can use a newtype or type erasure to encapsulate SemVer hazards. Implementers may have to use type erasure if their return type has unnameable types. Lifetime bounds may be required to help generic consumers.
(This is a more useful approach when the method is parameterized by a generic type you want to exclude, since you can make it impossible to name the type parameter in the associated type.)
trait TraitUsingOpaqueTest {
type Test: Test /* + 'static */;
fn get_test(&self) -> Self::Test;
// ...
}
impl <T> TraitUsingOpaqueTest for ValueHolder<T> {
type Test = HelloPrinter;
fn get_test(&self) -> Self::Test {
HelloPrinter
}
// ...
}
(Generic consumers do have the choice of adding their own lifetime requirements without any changes to the trait or implementation.)
In this specific example, the conflicting lifetimes being captured are the contained lifetime of T when it's instantiated with &mut u32, though, right? An I missing something?
Ah, took another look at the examples: the difference between + use<'a> and + 'a still trips me up!
With the model of opaques being like associated types, use<'a> is "can the associated type possibly name 'a at all", while + 'a is additionally "add a lifetime bound to the associated type".
So, if use<> were permitted it would be fine since it couldn't contain T's lifetime, and if the concrete type actually referenced T then the 'static would be illegal. Is it roughly readable as +use<'a> means "lives at least as long as 'a" while + 'a means "lives at most as long as 'a"? Not exactly because otherwise those would conflict, of course.
Yeah, it most senses. It can't actually contain T in its concrete type without erasing the lifetime, which is either itself unsafe or requires unsafe elsewhere to do anything useful.
impl<'a, U> Trait for Vec<&'a mut U>
where
&'a mut U: ContrivedExample<Assoc: Useful + Default + 'static>,
{
fn method() -> impl Useful + 'static {
// The implementor cannot normalize this type any further,
// even though we know from the bound the concrete type can't
// actually contain `'a` in its definition.
// vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
<&'a mut U as ContrivedExample>::Assoc::default()
}
}
// And for example this gives a confusing error forcing you to `use<'a, U>`
fn elsewhere<'a, U>(v: Vec<&'a mut U>) -> impl Sized + use<U> + 'static
where
Vec<&'a mut U>: Trait
{
Vec::<&'a mut U>::method()
}
...but these days, if there's a lifetime bound that makes it impossible for the concrete type to actually contain the 'a, the borrow checker is usually smart enough to recognize that, so the lifetime being around "hurts less". It has worked that way with associated types in generic contexts (which also cannot be normalized) since the very early days.
(The PR that made the borrow checker was smarter with opaque bounds had two attempts. In the first attempt, elsewhere would have compiled -- lifetimes were more completely erased. But that ended up being unsound due to interactions with the rest of the trait/type system. I can try to dig up the PRs if you're interested.)
It just means "'a is available to be used when defining the type". You don't have to use it. You could not use it and end up satisfying a 'static bound. You could use another lifetime and end up not satisfying a 'a bound (whether you used 'a or not).
trait Example {
// `'a` is available to be used, if you want. But you don't have to.
fn one<'a>(&self, &'a Input) -> impl Sized + use<'a, Self>;
type Uno<'a>: Sized;
fn uno<'a>(&self, &'a Input) -> Self::Uno<'a>;
// `'a` is not available to be used.
fn two<'a>(&self, &'a Input) -> impl Sized + use<Self>;
type Dos: Sized;
fn dos<'a>(&self, &'a Input) -> Self::Dos;
}
impl<'s> Example for Option<&'s mut i32> {
// Doesn't use `'a` or `'s`, always outlives `'static`
type Uno<'a> = u32;
// Doesn't use `'a`, doesn't always outlive `'static` (just `'s`)
type Dos = Self;
// ...
}
impl<'s> Example for Option<&'s mut ()> {
// Doesn't use `'a`, always outlives `'s`, doesn't always outlive `'a`
type Uno<'a> = Self;
// ...
}
Without refinement, or something like a compensating lifetime bound, consumers will treat it as if you did concretely use the lifetime. And it's considered a SemVer compatible change to start concretely using the lifetime. Sort of like how opaques always act like they have lifetime-observing destructors unless you add a Copy bound.
Consumers can draw conclusions about what an opaque outlives if they can prove all of the inputs to the opaque type -- everything it can use<..> -- meets the same bound. (Which is also how it works with associated types in generic context.) But they have to prove it for all the inputs, not just one.
So if you're considering impl Trait + use<'a> vs impl Trait + use<>, you can draw more conclusions that you can with impl Trait + use<'a, Self> vs impl Trait + use<Self>. (But in traits, you always have to use<Self> in the trait definition.)
Ah, I think this was the missing bit for me. The restrictions on implementations makes intuitive sense pretty easily once you hear the actual summary, but the implications for the caller are very confusing right now
I expect that it's something addressable with a borrow check note or the like, but this seems pretty early days and too in the weeds to be all that concerned about polish like that.
Hi,
first thank you for the in depth explanations.
Because of the many follow-up posts I should probably clarify first what I wanted to actually express. The intent was to express that get_test returns something which has no relation with T. It's effectivly just a constructor method for an instance of a opaque type, but I'm forced to define a relationship which does not exist, causing this lifetime and borrow-check issues in my example.
But as I understand it so far from the introductory example: Opaque types inside traits are implicitly associated with the trait mentioning them ("They are part of the types path"). I cannot remove parts which are part of the types path.
In your introductory example you sayed that precise capturing allows us to remove the ability to name some of the opaque types parameters. Do you mean that the type no longer can make use of specific parameters if I don't mention them in the use bound?
If I'm not mistaken you mentioned all in the second example, but 's and 'x are no longer mentioned. Can you maybe explain how the second one relates to the first one and which parameter can no longer be mentioned?
My mental model of use bounds was probably wrong here. I interpreted them as a promise that the returned type may at most make use of the mentioned lifetimes and imposed lifetimes of the type parameters and that the compiler in turn forces me to not use any lifetime, which is not part of that use bound.
Comeing back to my example I thought that for the the opaque impl Test the concrete T has no relevance and that the compiler in turn would refuse to accept any code which introduces a dependency to T. For example by constructing a T with the Default trait (which to my knowledge could be a case there I never directly mention T actually in the method or trait signature.
I probably also overestimated what capturing Self implies. I understood it as bounding the lifetime of the opaque type to the lifetime of any self reference. I read type Opaque0<'s, ...>: ... where Self: 's ... fn method<'s, 'x, Z>(&'s self, ...) -> Self::Opaque0 as a type which may contain some &self reference and is therefore bound to the lifetime of &'s.
But as it looks like, it's not happening. A slightly adjusted example shows, that the lifetime of the &self is not captured if I don't mention it in the use bounds:
i.e. if you only use<Self> and use<T> it works. It fails if you capture the lifetime with use<'_, Self> and use<'_, T>.[1]
I hope that if the trait type parameters + Self must be captured anyway, that it will become implicit at some day so that I don't have to spell it out everytime. At the current state it's at least confusing to me, that the signatures don't have to match ↩︎
It's true that you can't change the trait to completely get rid of the possibility of T (a parameter from the implementing type) having something to do with the return type. And you can't completely forbid implementors from using parameters from the implementing type in other ways, either -- they can put them in associated types, they can use them arbitrarily in the code,[1] etc.
You can restrict things in some ways by adding trait or lifetime bounds, but you can't completely forbid implementations from using parameters that are part of the implementing type. You can forbid implementations from using parameters of a specific method.
Here I give my associated type analogy another shot, but I'm not sure if it's helping and it runs a little long, so I'll collapse it by default.
Let's talk about the simplest case possible in traits: impl Trait + use <Self>. That's the most minimal precise capturing allowed in the trait definition, and it's only allowed when there are no trait parameters. That's the case in this example of yours.
In that case, the -> impl Trait is analogous to a non-generic associated type with a trait bound.
trait TraitUsingAssocTest {
type SetValueType;
type RetValue: Test; // <-- ~ impl Test + use<Self>
fn get_test(&self) -> Self::RetValue;
fn set_value(&mut self, v: Self::SetValueType) -> Self::SetValueType;
}
But if the implementer doesn't use refinement, it's like an associated type which you can never normalize, as if you were always in a generic context.[2] It's somewhat awkward to make this example applicable in a generic context, but not impossible; here's what I came up with.
In main (a non-generic context), the compiler is allowed to "see" that the associated type is HelloPrinter, which doesn't have any lifetime relationship to the &mut i32 (and that it has a trivial constructor). But in generic_context, it is not allowed to see these things. And that is the correct behavior, because some implementation could have such a relationship, and the generic_context is required to accommodate all possible implementations.
The same is mostly true of -> impl Trait, even in non-generic contexts: consumers must accomodate all possible implementations (except that auto traits leak). It's considered SemVer compatible for the concrete type of the opaque to change in any way that preserves the bounds (and the auto traits).
Then one may ask: Can we rewrite TraitUsingAssocTest so that an implementor is not allowed to name any parameters that come from their Self type? And the answer is no, the language does not support that directly. But we may be able to rule out lifetime relationships that cause problems by adding a 'static bound, or rule out destructor issues by adding a Copy bound, etc.
And the same is true for -> impl Trait + use<Self> in TraitUsingOpaqueTest. There's no way to completely forbid using generic parameters that come from Self; we can't use<> in a trait definition. But we can rule out some cases by adding bounds. (And implementors can choose to act more like associated types in non-generic contexts by opting into refinement.)
Correct. And if you elide + use<..> it's like having a + use<..> mentioning everything in scope.[3]
Where in the second one I removed 's and 'x from use<..>, so those can't be part of the returned opaque type in any implementation, and it won't act like they contain 's and 'x for consumers. In more practical terms, using the returned value from the first version will keep *self and *arg borrowed, but using the returned value from the second version will not.
I'm not exactly what you mean by "bounding", but with
it will act like it contains &'s self,[4] because 's is in the use<..>.
So the version without use<'_, ..> doesn't capture the outer lifetime from &self. But you are still capturing T.
I agree that it's never ideal when copying the signature from the trait doesn't work in an implementation.
(Incidentally you can sometimes get rid of capturing trait parameters by moving a method to a subtrait that doesn't have those parameters. But that doesn't help with Self.)
,