Cannot move out from a shared reference

1

I am trying to iterate over a bunch of S3 files and process those. Not sure how to work with ByteStream because it seems I cannot use the collect() method on it due to the following error.

What am I missing?

async fn read_files(
    s3_client: S3Client,
    s3_bucket: &str,
    s3_prefixes: Vec<String>,
) -> Result<String, BoxedErr> {
    let futures = s3_prefixes.iter().map(|prefix| async {
        s3_client
            .get_object()
            .bucket(s3_bucket)
            .key(prefix.clone())
            .send()
            .await
    });

    let objects: Vec<Result<GetObjectOutput, SdkError<GetObjectError>>> = join_all(futures).await;

    let byte_streams: Vec<&ByteStream> = objects
        .iter()
        .filter(|y| y.is_ok())
        .map(|x| x.as_ref().unwrap().body())
        .collect::<Vec<_>>();

    let bytes = byte_streams
        .iter()
        .map(|x| async { x.collect().await.map(|data| data.into_bytes()) })
        .collect::<Vec<_>>();

    Ok("".to_string())
}

The error:

Result<String, Box<dyn Error + Send + Sync, Global>>
Go to Result | String | Box | Error | Global

cannot move out of `**x` which is behind a shared reference
move occurs because `**x` has type `ByteStream`, which does not implement the `Copy` traitrustcClick for full compiler diagnostic
convert_logs_to_parquet.rs(165, 28): `**x` moved due to this method call
byte_stream.rs(300, 26): this function takes ownership of the receiver `self`, which moves `**x`
2

Don't copy error messages from IDEs. Use cargo check in the terminal and copy from there. Much more readable and useful to people trying to answer your questions :wink:

7 Likes
3

[T]::iter creates an implementation of Iterator<Item = &T>, hence x is borrowed. This becomes a problem when you try to move x by consuming it. You want to use Vec::into_iter here instead.

2 Likes
4

Thanks for the suggestions.

    let byte_streams: Vec<GetObjectOutput> = objects
        .into_iter()
        .filter(|y| y.is_ok())
        .map(|x| x.unwrap())
        .collect::<Vec<_>>();

    let bytes = byte_streams
        .iter()
        .map(|x| x.body().into_inner())
        //.map(|x| async move { x.body().collect().await.map(|data| data.into_bytes()) })
        .collect::<Vec<_>>();

error[E0507]: cannot move out of a shared reference
   --> src/convert_logs_to_parquet.rs:167:18
    |
167 |         .map(|x| x.body().into_inner())
    |                  ^^^^^^^^^------------
    |                  |        |
    |                  |        value moved due to this method call
    |                  move occurs because value has type `aws_sdk_s3::types::ByteStream`, which does not implement the `Copy` trait
    |
note: this function takes ownership of the receiver `self`, which moves value
   --> /Users/l1x/.cargo/registry/src/github.com-1ecc6299db9ec823/aws-smithy-http-0.51.0/src/byte_stream.rs:280:23
    |
280 |     pub fn into_inner(self) -> SdkBody {
    |                       ^^^^

For more information about this error, try `rustc --explain E0507`.
warning: `convert-logs-to-parquet` (bin "convert-logs-to-parquet") generated 2 warnings
error: could not compile `convert-logs-to-parquet` due to previous error; 2 warnings emitted

Based on the discussion and the rustc recommendation it try to use into_inner() without success.

5

I am going to use that going forward. Thanks.

6

Finally I figured out that using x.body instead of x.body() works. This is the internal details of this lib that was hard to understand at first.

7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.