Cannot move out from a shared reference

I am trying to iterate over a bunch of S3 files and process those. Not sure how to work with ByteStream because it seems I cannot use the collect() method on it due to the following error.

What am I missing?

async fn read_files(
    s3_client: S3Client,
    s3_bucket: &str,
    s3_prefixes: Vec<String>,
) -> Result<String, BoxedErr> {
    let futures = s3_prefixes.iter().map(|prefix| async {
        s3_client
            .get_object()
            .bucket(s3_bucket)
            .key(prefix.clone())
            .send()
            .await
    });

    let objects: Vec<Result<GetObjectOutput, SdkError<GetObjectError>>> = join_all(futures).await;

    let byte_streams: Vec<&ByteStream> = objects
        .iter()
        .filter(|y| y.is_ok())
        .map(|x| x.as_ref().unwrap().body())
        .collect::<Vec<_>>();

    let bytes = byte_streams
        .iter()
        .map(|x| async { x.collect().await.map(|data| data.into_bytes()) })
        .collect::<Vec<_>>();

    Ok("".to_string())
} 

The error:

Result<String, Box<dyn Error + Send + Sync, Global>>
Go to Result | String | Box | Error | Global

cannot move out of `**x` which is behind a shared reference
move occurs because `**x` has type `ByteStream`, which does not implement the `Copy` traitrustcClick for full compiler diagnostic
convert_logs_to_parquet.rs(165, 28): `**x` moved due to this method call
byte_stream.rs(300, 26): this function takes ownership of the receiver `self`, which moves `**x`

Don't copy error messages from IDEs. Use cargo check in the terminal and copy from there. Much more readable and useful to people trying to answer your questions :wink:

7 Likes

[T]::iter creates an implementation of Iterator<Item = &T>, hence x is borrowed. This becomes a problem when you try to move x by consuming it. You want to use Vec::into_iter here instead.

2 Likes

Thanks for the suggestions.

    let byte_streams: Vec<GetObjectOutput> = objects
        .into_iter()
        .filter(|y| y.is_ok())
        .map(|x| x.unwrap())
        .collect::<Vec<_>>();

    let bytes = byte_streams
        .iter()
        .map(|x| x.body().into_inner())
        //.map(|x| async move { x.body().collect().await.map(|data| data.into_bytes()) })
        .collect::<Vec<_>>();
error[E0507]: cannot move out of a shared reference
   --> src/convert_logs_to_parquet.rs:167:18
    |
167 |         .map(|x| x.body().into_inner())
    |                  ^^^^^^^^^------------
    |                  |        |
    |                  |        value moved due to this method call
    |                  move occurs because value has type `aws_sdk_s3::types::ByteStream`, which does not implement the `Copy` trait
    |
note: this function takes ownership of the receiver `self`, which moves value
   --> /Users/l1x/.cargo/registry/src/github.com-1ecc6299db9ec823/aws-smithy-http-0.51.0/src/byte_stream.rs:280:23
    |
280 |     pub fn into_inner(self) -> SdkBody {
    |                       ^^^^

For more information about this error, try `rustc --explain E0507`.
warning: `convert-logs-to-parquet` (bin "convert-logs-to-parquet") generated 2 warnings
error: could not compile `convert-logs-to-parquet` due to previous error; 2 warnings emitted

Based on the discussion and the rustc recommendation it try to use into_inner() without success.

I am going to use that going forward. Thanks.

Finally I figured out that using x.body instead of x.body() works. This is the internal details of this lib that was hard to understand at first.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.