I'm trying to use reqwest+tokio to asynchronously download and save an image using the URL. There's a variable called data_directory
that contains the location to save the image to and it needs to be an immutable shared state variable. How do I share this immutable reference passed to the function between tokio tasks? (See error message at the end)
This is what I'm trying to run:
pub async fn get_images_parallel(saved: &UserSaved, data_directory: &str) -> Result<(), ReddSaverError> {
let tasks: Vec<_> = saved
.data
.children
.clone()
.into_iter()
// filter out the posts where a URL is present
// not that this application cannot download URLs linked within the text of the post
.filter(|item| item.data.url.is_some())
.filter(|item| {
let url_unwrapped = item.data.url.as_ref().unwrap();
// If the URLs end with jpg/png it is assumed to be an image
url_unwrapped.ends_with("jpg") || url_unwrapped.ends_with("png")
})
.map(|item| {
let directory = data_directory.clone();
// since the latency for downloading an image from the network is unpredictable
// we spawn a new async task using tokio for the each of the images to be downloaded
tokio::spawn(async {
let url = item.data.url.unwrap();
let extension = String::from(url.split('.').last().unwrap_or("unknown"));
let subreddit = item.data.subreddit;
info!("Downloading image from URL: {}", url);
let file_name = generate_file_name(&url, &directory, &subreddit, &extension);
if check_path_present(&file_name) {
info!("Image already downloaded. Skipping...");
} else {
let image_bytes = reqwest::get(&url).await?.bytes().await?;
let image = match image::load_from_memory(&image_bytes) {
Ok(image) => image,
Err(_e) => return Err(ReddSaverError::CouldNotCreateImageError),
};
save_image(&image, &subreddit, &file_name)?;
info!("Successfully saved image: {}", file_name);
}
Ok::<(), ReddSaverError>(())
})
})
.collect();
// wait for all the images to be downloaded and saved to disk before exiting the method
for task in tasks {
if let Err(e) = task.await? {
return Err(e);
}
}
Ok(())
}
I tried to use Arc
with clone
(not shown here) but I'm still encountering the same error.
Error message:
error[E0759]: `data_directory` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
--> src/utils.rs:68:53
|
68 | pub async fn get_images_parallel(saved: &UserSaved, data_directory: &str) -> Result<(), ReddSaverError> {
| ^^^^^^^^^^^^^^ ---- this data with an anonymous lifetime `'_`...
| |
| ...is captured here...
...
87 | tokio::spawn(async {
| ------------ ...and is required to live as long as `'static` here
I'm new to Rust and this is my first time writing an application with it, so any help here would be appreciated. Thanks a lot!