Lightweight alternative for `reqwest`?

Reqwest is nice, but it is very heavy (LoC-wise when considered with its own dependencies). In many contexts (especially security-sensitive) this is not great. Things like async support are not really needed.

Any recommendations for less-featureful, but more lightweight alternatives? Thanks for all the suggestions!

2 Likes

I think there aren't any alternatives. You could use hyper with a custom https connector and something like futures(-preview), but this would be quite unergonomic.

Do you know, that the new async version of reqwest supports disabling some parts of the library with features?

1 Like

There’s isahc (formerly known as chttp), which is based on curl, so while it is still „heavy“ (curl is no small thing anymore), maybe it fits what’s you need?

Use surf. :slight_smile:

Here's an example that prints the HTML contents of a web page using async/await:

use async_std::task;

fn main() -> Result<(), surf::Exception> {
    task::block_on(async {
        let mut response = surf::get("https://www.rust-lang.org").await?;
        println!("{}", response.body_string().await?);
        Ok(())
    })
}
2 Likes

If you don't need async, then ureq fits the bill. Has minimal dependencies so it compiles a lot faster than mentioned alternatives.

If you look at the options, you can even disable, tls, json, and codecs, so it's really minimal.

Yeah, that's even worse, from security perspective if you ask me. :smiley:

That's looks a bit better, but it will still require bringing in tokio, right? Or can I use something smaller to power the futures? I realize that async/await is trendy, but from the perspective of a lot of software that does like a handful of http request async await is just increasing the complexity.

Awesome! I think that's what I was looking for.

Regrettably, I reviewed the source code of ureq and had to give it a negative rating.

Would some of this feedback be more constructive as Bug Reports and PRs? Even if this review is meant to be helpful it can come across as lazy and hostile to some people. I would certainly feel uncomfortable about receiving a review like this if the author did not contact me first.

6 Likes

Although, having read the critique, the tone is not hostile. I definitely think we need a culture of review, and that does fight with the need to be nice sometimes. When evaluating something for production the bar is pretty high. We've stuck with reqwest because it's solid, even if there's a smaller leaner crate in there somewhere wanting to come out.

1 Like

I did report it in a github issue https://github.com/algesten/ureq/issues/15 right away.

I was just casually making notes and a bit in rush to go to bed, before my wife gets angry that I stay too late again. I pasted them into review, linked here and there, and was not trying to make it looks nice or mean. When reporting it did occur to me that pointing out alleged problems and mistakes is a bit uncomfortable for both me and probably the author. I would rather always write "this code is awesome", of course.

On a daily basis I review other people code, and get my code reviewed at $dayjob, and I'm used to it, and generally I have an attitude that one person never arrives at the perfect solution, and the only way forward is to always acknowledge any feedback, especially critical, and iteratively improve. But it is easier with people you know and chat face to face with. I'll definitely have to think about this for crevs purposes.

3 Likes

I haven't looked at the source cure, but to me it sounds like maybe some reqwest features could be made optional by putting them behind build flags, which would reduce the amount of code compiled for the minimal build.

You'll see this with reqwest v0.10

5 Likes

I'm curious why you think async/await increases complexity. It is built-in to Rust, and it replaces many combinators, so the end result is that libraries are smaller and less complex, so you don't have as much code to review.

And as far as implementation goes, an async/await executor is about the same complexity as a Futures 0.1 executor.

I believe dpc is comparing async.await to a fully sync API rather than futures 0.1. If all you're using the HTTP client for is to fetch some very small number of resources over HTTP in a build script, the difference between an asynchronous backend and a synchronous one is basically nothing.

1 Like

@CAD97 Ahh, that makes a lot more sense.

My gut feeling is that for almost any client full sync api using a threadpool is enough. A server can expect to have 10k inbound connections. A client maintaining 10k outbound connections? I'd like to hear about what are you doing there. :smiley:

4 Likes

Just remembered and wanted to point out:

If the async-capable library is executor agnostic, you can use futures::block_on and/or LocalPool as an executor.

In practice, though, right now futures-0.3 experimentation is mostly assuming tokio is available. I expect that as the ecosystem matures, there will arrive a lighter "runtime" over mio to fill the niche of clients that really only need LocalPool and don't need the full tokio runtime.

(That said, there is some worth to "just using tokio", so long as you can fit it into your trust model.)

(Also, executor/reactor/runtime all of these futures bits I don't know the exact separation of concerns forgive me)

1 Like

I may be a bit late to the party, but having similar requirements, I have had good experiences using attohttpc recently. I have not done a full code review, but the crate itself is quite minimal and does not contain a single unsafe block itself.

1 Like

Perfect suggestion! Thank you!

I've just did a basic review: https://github.com/dpc/crev-proofs/commit/77cf6ac1f401c5c8e84adbdb50df24826e5a7c16 and it looks exactly like what I was looking for.

Also, with all optional features disabled, it looks like almost all its dependencies are already reviewed by one of the cargo-crev users:

$ cargo crev c v --target --no-default-features --no-dev-dependencies                                                      
status reviews     downloads    owner  issues lines  geiger flgs crate                version         latest_t           
pass    1  1  3667133   7488040  0/1    0/0      54       0      matches              0.1.8           =
pass    2  3  2061299  11428613  2/2    0/0      89       0      cfg-if               0.1.9           ↑0.1.10
pass    2  2  2199019   9542824  1/1    0/0     308       1      itoa                 0.4.4           =                  
pass    1  2  4945377   6340515  0/2    0/0     247       3      percent-encoding     1.0.1           ↑2.0.0
pass    1  1  4626859   5208296  0/2    0/0     257       0      fnv                  1.0.6           =                  
warn    2  7  1834735   7402415  0/3    0/0    1926     342      smallvec             0.6.10          ↓0.6.7
pass    1  2  1338361  16109385  5/5    0/0    2354      32 CB   log                  0.4.8           =
none    0  0  5291155   6938235  0/6    0/0    2039       0      unicode-bidi         0.3.4           
pass    1  3  1873481  10777396  1/1    0/0    2436     226 CB   byteorder            1.3.2           =
none    0  0  2440672   7318535  3/6    0/0   12912      20      unicode-normalization 0.1.8           
pass    1  2  3684117   7278412  0/2    0/0   16499       1      idna                 0.1.5           ↑0.2.0             
none    0  0  2967585   8624610  1/4    0/0    3457       2      url                  1.7.2                              
pass    1  1  1105435  19510584  4/4    0/0   58231      37 CB   libc                 0.2.62          =
pass    1  1  3978248   4858110  1/1    0/0     275      75      iovec                0.1.2           =                  
pass    1  1  2015676   5586938  1/1    0/0    2885     274      bytes                0.4.12          =                  
none    0  0   633031   3084299  2/2    0/0    8156     154      http                 0.1.18          
2 Likes