How to run FuturesUnordered to completion without collect?

1

Hi!

I created a FuturesUnordered and execute futures concurrently:

use futures::stream::FuturesUnordered;
use futures::StreamExt;
use std::time::Duration;
use tokio::time;

async fn sleep(s: u64) {
    println!("begin {}", s);
    time::delay_for(Duration::from_secs(s)).await;
    println!("end {}", s);
}

#[tokio::main]
async fn main() {
    (1..10u64)
        .into_iter()
        .map(sleep)
        .collect::<FuturesUnordered<_>>()
        .collect::<Vec<()>>() // <--- how not to allocate here?
        .await;
}

But to execute them I have to write collect two times:

  • first to convert a bunch of futures into FuturesUnordered
  • second to be able to await on this construction

The problem is that there is an unneeded allocation. How can I avoid this collecting to a Vec?

1 Like
2

One option is to use while loop:

let futures = ... .collect::<FuturesUnordered<_>>();

while let Some(_result) = futures.next().await {}

There might be a way of using for_each() to drain the iterator but I think it might require async closures in this case. Still learning so wondering what others will suggest.

3

Vec<()> doesn't allocate on the heap, since () is a zero sized type. As long as you're not returning a value from the future, its already doing what you want.

4 Likes
4

@drewkett, thanks, didn't know that. Is it stated somewhere in reference?

BTW I could rewrite this

    (1..10u64)
        .into_iter()
        .map(sleep)
        .collect::<FuturesUnordered<_>>()
        .collect::<Vec<()>>()
        .await;

as

    futures::stream::iter((1..10u64).into_iter())
        .for_each_concurrent(None, sleep)
        .await;

with the same effect. It deviates a bit from original question but still can be useful.

5

In the Guarantees section of the Vec documentation:

Similarly, if you store zero-sized types inside a Vec, it will not allocate space for them. Note that in this case the Vec may not report a capacity of 0. Vec will allocate if and only if mem::size_of::<T>() * capacity() > 0 .

2 Likes
6

I was wondering what happens to the length. Does Vec become essentially an usize or is the Vec struct allocated without elements? As perhaps suggested by

Most fundamentally, Vec is and always will be a (pointer, capacity, length) triplet. No more, no less.

Code

fn test(_i: i64) {}

fn main() {
    let v = (0..100).map(test).collect::<Vec<_>>();
    println!("{}", v.len());
}

prints 100.

7

You could check the code for Vec::new(), which shows that a 3-usize struct is allocated with len = 0. Note that the underlying Raw_vec::new() initializes the capacity to zero when mem::size_of::<T>() > 0, and to usize::MAX whtn mem::size_of::<T>() = 0,

8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.