You can change as_slice
to
pub fn as_slice(&self) -> &'a str {
&self.tokens[self.position..]
}
to make it work playground
The reason for this is that &self.token[self.position..]
desugars to &(*self.token.index(self.position..))
. index
comes from Index
pub trait Index<Idx> {
type Output: ?Sized;
fn index(&self, index: Idx) -> &Self::Output;
}
for str
you get something like below from the Index
trait.
fn index(&self, index: Range<usize>) -> &str { ... }
If we track the lifetimes, we see that the input string (self
) and the output string have the same lifetime.
Now, back to the problem.
fn as_slice(&self) -> &'a str {
&self.tokens[self.position..]
}
desugars to below, as I showed before
fn as_slice(&self) -> &'a str {
&(*self.tokens.index(self.position..))
}
Now, reborrowing a shared reference (the combo &*
is called reborrowing) will keep the same lifetime as the input lifetime. This is because shared refereces can be copied around. So, you get the lifetime 'a
for the output.
The issue before when you had,
fn as_slice(&'a self) -> &'a str {
&self.tokens[self.position..]
}
was that you were declaring the lifetime of the output to be the same as the lifetime of the input. This is too restrictive in this case, so it didn't work with the closure you had.
One way to think of you type TokenStream
, is that it is a view into a string stored somewhere else. So when you call as_slice
you are producing a new view into the same string, but that view doesn't need to be tied to the TokenStream
(since the TokenStream
doesn't own the original string), so you can tie it to the lifetime of that original string (which is 'a
in TokenStream<'a>
).
It is bad style to reuse 'a
everywhere, it makes code harder to read and harder to write, so if you see yourself shadowing lifetime names, please rename some lifetimes to get rid of the shadowing.