Listing all objects in S3 bucket with aws-sdk-s3

I'm looking at the AWS SDK for Rust, in particular S3, trying to work out how to use it.

I came up with the following and I'm wondering if that looks reasonable or if there's some nicer way to list all objects in a bucket.

use anyhow::{anyhow, Result};
use aws_config::meta::region::RegionProviderChain;
use aws_sdk_s3::{error::ListObjectsV2Error, output::ListObjectsV2Output, types::SdkError, Client};

#[tokio::main]
async fn main() -> Result<()> {
    let bucket = std::env::args()
        .nth(1)
        .ok_or(anyhow!("need to name a bucket"))?;
    let prefix = std::env::args().nth(2).ok_or(anyhow!("need a prefix"))?;
    let client = get_client().await;
    let mut cont_token = None;

    loop {
        match list_next(&client, &bucket, &prefix, &cont_token).await {
            Ok(resp) => {
                for obj in resp.contents().unwrap_or_default() {
                    println!("{}", obj.key().unwrap_or_default())
                }
                if resp.is_truncated() {
                    cont_token = resp.next_continuation_token().map(|s| s.to_string());
                } else {
                    break Ok(());
                }
            }
            Err(e) => {
                eprintln!("Failed to fetch object: {:?}", e)
            }
        }
    }
}

pub async fn get_client() -> Client {
    let region_provider = RegionProviderChain::default_provider().or_else("eu-central-1");
    let config = aws_config::from_env().region(region_provider).load().await;
    Client::new(&config)
}

async fn list_next(
    client: &Client,
    bucket: &str,
    prefix: &str,
    cont_token: &Option<String>,
) -> Result<ListObjectsV2Output, SdkError<ListObjectsV2Error>> {
    client
        .list_objects_v2()
        .bucket(bucket)
        .prefix(prefix)
        .set_continuation_token(cont_token.to_owned())
        .send()
        .await
}

Might want to add some logic to limit retires, otherwise it will keep trying again and again on errors (there is no break after the eprintln). But otherwise looks good.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.