Forwarding SSL data through multiple proxies

1

I am looking for some advice on how to handle the concurrency for forwarding the data through 500-1000 proxies. I have written the code that forwards the data through the proxies so the hard part is done. I would like the code the be pretty lightweight on the system resources but I cannot think of a way to get this done.

here is my code for the main function which contains the current way I am handling the concurrency for my program. Any suggestions will be appreciated as I am not by any means a professional with rust.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {

    let proxies = read_proxies_from_file("../proxies.txt")?;
    
    let mut handles = Vec::new();

    for proxy in proxies {
        let proxy_clone = proxy.clone(); 


        let handle = tokio::spawn(async move {
            if let Err(e) = go(proxy_clone).await {
                eprintln!("error: {:?}", e.to_string());
                // eprintln!("Error in go function for proxy {}: {:?}", proxy_clone, e);
            }
        });

        handles.push(handle);
    }

    for handle in handles {
        handle.await.unwrap();
    }

    Ok(())
}
3

The biggest change I'd make is to use either a JoinSet or a TaskTracker to keep track of your in-progress tasks, rather than a Vec of handles. Both of these are optimized for handling very large numbers of tasks efficiently.