How to download files from the Internet?

I found information on Google:

https://users.rust-lang.org/t/download-file-from-web/19863

extern crate reqwest;

use std::io;
use std::fs::File;

fn main() {
    let mut resp = reqwest::get("https://sh.rustup.rs").expect("request failed");
    let mut out = File::create("rustup-init.sh").expect("failed to create file");
    io::copy(&mut resp, &mut out).expect("failed to copy content");
}

Cargo.toml

[package]
name = "example"
version = "0.1.0"
authors = ["fang <a@q.com>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
reqwest = "0.11.0"

But it cannot be compiled!

error[E0599]: no method named `expect` found for opaque type `impl Future` in the current scope
 --> src/main.rs:7:57
  |
7 |     let mut resp = reqwest::get("https://sh.rustup.rs").expect("request failed");
  |                                                         ^^^^^^ method not found in `impl Future`
  |
help: consider awaiting before this method call
  |
7 |     let mut resp = reqwest::get("https://sh.rustup.rs").await.expect("request failed");
  |                                                         ^^^^^^

Prompt to add await, I changed it to the following and still reported an error.

extern crate reqwest;

use std::io;
use std::fs::File;

async fn main() {
    let mut resp = reqwest::get("https://sh.rustup.rs").await.expect("request failed");
    let mut out = File::create("rustup-init.sh").expect("failed to create file");
    io::copy(&mut resp, &mut out).expect("failed to copy content");
}

Error:

error[E0277]: the trait bound `Response: std::io::Read` is not satisfied
  --> src/main.rs:9:5
   |
9  |     io::copy(&mut resp, &mut out).expect("failed to copy content");
   |     ^^^^^^^^ the trait `std::io::Read` is not implemented for `Response`
   | 
  ::: /Users/fangnan/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/io/util.rs:51:8
   |
51 |     R: Read,
   |        ---- required by this bound in `std::io::copy`

Hi,
In order to use reqwest, you either need the tokio async runtime or use the reqwest::blocking module.

You can find more information about the former here: Calling a Web API - Rust Cookbook

And about the latter here: Feedback on my first Rust project

Is it like this?

[dependencies]
reqwest = { version = "0.11", features = ["json"] }
tokio = { version = "1", features = ["full"] }
extern crate reqwest;

use std::io;
use std::fs::File;

#[tokio::main]
async fn main() {
    let mut resp = reqwest::get("https://sh.rustup.rs").await.expect("request failed");
    let mut out = File::create("rustup-init.sh").expect("failed to create file");
    io::copy(&mut resp, &mut out).expect("failed to copy content");
}

Still can't compile

There's two issues:

  • reqwest returns the Response as soon as the headers arrive, so you can't read the body directly from that object - the data might not have finished sending yet! You need to call one of the awaitable methods that decode the body when it's done being read. In this case, you probably want .text().
  • That gives you a String, but String doesn't implement Read, so you need to call .as_bytes() on it.
use std::io;
use std::fs::File;

#[tokio::main]
async fn main() {
    let resp = reqwest::get("https://sh.rustup.rs").await.expect("request failed");
    let body = resp.text().await.expect("body invalid");
    let mut out = File::create("rustup-init.sh").expect("failed to create file");
    io::copy(&mut body.as_bytes(), &mut out).expect("failed to copy content");
}
2 Likes

Note that since you are not writing a web server or something like that, I recommend just using reqwest::blocking.

[dependencies]
reqwest = { version = "0.11", features = ["blocking", "json"] }
use std::io;
use std::fs::File;

fn main() {
    let resp = reqwest::blocking::get("https://sh.rustup.rs").expect("request failed");
    let body = resp.text().expect("body invalid");
    let mut out = File::create("rustup-init.sh").expect("failed to create file");
    io::copy(&mut body.as_bytes(), &mut out).expect("failed to copy content");
}

@17cupsofcoffee You aren't supposed to use std file apis in async code because they are blocking. Instead you should prefer tokio::fs. These kind of details are why I recommend just using reqwest::blocking for non-web-server use-cases.

4 Likes

Oh whoops, I totally missed the fact they weren't using the tokio io::copy. My bad!

I agree, using the blocking client is probably a better idea unless you're building an application that more widely makes use of async.

1 Like

I found that the downloaded file is different from the file on the server. I really don't understand why, so I opened a new post.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.