Tokio execute async for loop concurrently

Hi,

i have an array of things and want to do remote API calls for all of them. The API call itself is an async function, since it has to wait for the network. I'd like to parallelize the calls and join the results. I had a look at the tokio docs, and the tokio::join macro seems to work for tuples but not for iterables like Vec. Unfortunately, google searches for "tokio loop" and similar turn up a huge amount of unrelated results concerning tokio's event loop.

What i want is basically something like

async fn get_character_data_from_api(name : &str) -> anyhow::Result<String> {
  // do async request
}

let characters = vec!["gandalf", "frodo", "bilbo"];
let results : Vec<anyhow::Result<String>> = 
    join_futures(characters.iter().map(|name| get_character_data_from_api(name)))
    .await.collect();

If one call fails, the others should still be executed and i want to handle errors for each item separately.

This proabably isn't very complicated, but unfortunately i was unable to google it :expressionless:

Found it myself. The solution is to join_all(...).await the Vec of individual futures. The join_all method is part of the futures crate, not tokio :expressionless:

Yes, you will usually need to use the futures crate together with Tokio. Note that without the futures crate, you can still do it using spawning:

let mut handles = Vec::new();
for character in &characters {
    let job = tokio::spawn(get_character_data_from_api(character));
    handles.push(job);
}
let mut results = Vec::new();
for job in handles {
    results.push(job.await);
}
3 Likes

Thanks, your solution looks like it's more easily adaptable to more complex use cases, because it's possible to do other (non-blocking) stuff while that one specific loop is running.

You may also like the FuturesUnordered collection type, which is very powerful and allows you to do stuff like limiting it to at most 10 requests at the time.

2 Likes

i saw that one, but wasn't sure how to spawn/poll all of them while await-ing the results in order

As the name suggests, it wont give you the results in order. One option would be to include an index in the output of each job.

Does FuturesUnordered poll all Futures it contains when awaiting any one of them?

Yes, if you do futs.next().await, then every future inside the collection is polled until one of them completes. The output of the one that completed is then returned.

1 Like

Thanks!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.