Generic type parameters - what is going on here?

help
#1

I just wrote an async function to parse an http multipart body. The code is fairly straight-forward, the bit that I found difficult was the function declaration itself:

/// Parse multipart body.
async fn get_multipart<R>(br: &mut tokio::io::BufReader<R>, parts: &mut Vec<Part>) -> Result<()>
where
    R: AsyncReadExt + Unpin + Send,
{

I did look up what "Unpin" is, but it is mysterious, what is going on here?

#2

What is it exactly that you don't understand?

#3

Why I have to put "Unpin" in there I think. When I haven't the first idea what it even is.

I think it is all related to how extension traits work, but it feels strange to be putting identifiers in a program where I don't know what they are. ( The compiler did give a helpful hint or I would not have known what to do! )

#4

It has nothing to do with extension traits – "extension trait" is just a convention/pattern, they are just regular traits.

Here, one immediate reason why you need the Unpin bound is calling the <BufReader<R> as AsyncBufReadExt>::read_until() method. It has a very explicit where Self: Unpin bound.

As for what Unpin is, it's a marker trait that asserts that a type is safe to move around unconditionally (i.e., it's not self-referential, or if it is, it's self-referential in a way that moving doesn't invalidate self-references, for example, it's boxed). In general, async code often requires this constraint. (Another explanation.)

1 Like
#5

It means that R cannot be a self-referential type. The reason that you can't allow self-referential types is that reading from a self-referential type is only ok if the value is never moved again, but the owner who gave you that mutable reference could move it after get_multipart returns.

You could change the function to look like this:

async fn get_multipart<R>(br: Pin<&mut tokio::io::BufReader<R>>, parts: &mut Vec<Part>) -> Result<()>
where
    R: AsyncReadExt + Send,
{

Here, the Pin is a promise from the caller that the caller will never move it again, so reading from it would be safe.

As an aside, you should never use AsyncReadExt as a trait bound. You should instead use AsyncRead like this:

async fn get_multipart<R>(br: &mut tokio::io::BufReader<R>, parts: &mut Vec<Part>) -> Result<()>
where
    R: AsyncRead + Unpin + Send,
{

The reason is that the compiler will deduce that R is also an AsyncReadExt when it is AsyncRead. However, the logic does not go the other way, and the compiler will not deduce that R: AsyncRead just because you know it is AsyncReadExt.

Edit: I guess the compiler might be able to deduce R: AsyncRead either way, but it is still conceptually wrong.

5 Likes