Lightweight alternative for `reqwest`?

If you don't need async, then ureq fits the bill. Has minimal dependencies so it compiles a lot faster than mentioned alternatives.

If you look at the options, you can even disable, tls, json, and codecs, so it's really minimal.

4 Likes

Yeah, that's even worse, from security perspective if you ask me. :smiley:

That's looks a bit better, but it will still require bringing in tokio, right? Or can I use something smaller to power the futures? I realize that async/await is trendy, but from the perspective of a lot of software that does like a handful of http request async await is just increasing the complexity.

Awesome! I think that's what I was looking for.

1 Like

Regrettably, I reviewed the source code of ureq and had to give it a negative rating.

1 Like

Would some of this feedback be more constructive as Bug Reports and PRs? Even if this review is meant to be helpful it can come across as lazy and hostile to some people. I would certainly feel uncomfortable about receiving a review like this if the author did not contact me first.

7 Likes

Although, having read the critique, the tone is not hostile. I definitely think we need a culture of review, and that does fight with the need to be nice sometimes. When evaluating something for production the bar is pretty high. We've stuck with reqwest because it's solid, even if there's a smaller leaner crate in there somewhere wanting to come out.

1 Like

I did report it in a github issue `cargo-crev` review · Issue #15 · algesten/ureq · GitHub right away.

I was just casually making notes and a bit in rush to go to bed, before my wife gets angry that I stay too late again. I pasted them into review, linked here and there, and was not trying to make it looks nice or mean. When reporting it did occur to me that pointing out alleged problems and mistakes is a bit uncomfortable for both me and probably the author. I would rather always write "this code is awesome", of course.

On a daily basis I review other people code, and get my code reviewed at $dayjob, and I'm used to it, and generally I have an attitude that one person never arrives at the perfect solution, and the only way forward is to always acknowledge any feedback, especially critical, and iteratively improve. But it is easier with people you know and chat face to face with. I'll definitely have to think about this for crevs purposes.

5 Likes

I haven't looked at the source cure, but to me it sounds like maybe some reqwest features could be made optional by putting them behind build flags, which would reduce the amount of code compiled for the minimal build.

You'll see this with reqwest v0.10

5 Likes

I'm curious why you think async/await increases complexity. It is built-in to Rust, and it replaces many combinators, so the end result is that libraries are smaller and less complex, so you don't have as much code to review.

And as far as implementation goes, an async/await executor is about the same complexity as a Futures 0.1 executor.

I believe dpc is comparing async.await to a fully sync API rather than futures 0.1. If all you're using the HTTP client for is to fetch some very small number of resources over HTTP in a build script, the difference between an asynchronous backend and a synchronous one is basically nothing.

2 Likes

@CAD97 Ahh, that makes a lot more sense.

My gut feeling is that for almost any client full sync api using a threadpool is enough. A server can expect to have 10k inbound connections. A client maintaining 10k outbound connections? I'd like to hear about what are you doing there. :smiley:

4 Likes

Just remembered and wanted to point out:

If the async-capable library is executor agnostic, you can use futures::block_on and/or LocalPool as an executor.

In practice, though, right now futures-0.3 experimentation is mostly assuming tokio is available. I expect that as the ecosystem matures, there will arrive a lighter "runtime" over mio to fill the niche of clients that really only need LocalPool and don't need the full tokio runtime.

(That said, there is some worth to "just using tokio", so long as you can fit it into your trust model.)

(Also, executor/reactor/runtime all of these futures bits I don't know the exact separation of concerns forgive me)

1 Like

I may be a bit late to the party, but having similar requirements, I have had good experiences using attohttpc recently. I have not done a full code review, but the crate itself is quite minimal and does not contain a single unsafe block itself.

3 Likes

Perfect suggestion! Thank you!

I've just did a basic review: Add review for attohttpc v0.4.5 · dpc/crev-proofs@77cf6ac · GitHub and it looks exactly like what I was looking for.

Also, with all optional features disabled, it looks like almost all its dependencies are already reviewed by one of the cargo-crev users:

$ cargo crev c v --target --no-default-features --no-dev-dependencies                                                      
status reviews     downloads    owner  issues lines  geiger flgs crate                version         latest_t           
pass    1  1  3667133   7488040  0/1    0/0      54       0      matches              0.1.8           =
pass    2  3  2061299  11428613  2/2    0/0      89       0      cfg-if               0.1.9           ↑0.1.10
pass    2  2  2199019   9542824  1/1    0/0     308       1      itoa                 0.4.4           =                  
pass    1  2  4945377   6340515  0/2    0/0     247       3      percent-encoding     1.0.1           ↑2.0.0
pass    1  1  4626859   5208296  0/2    0/0     257       0      fnv                  1.0.6           =                  
warn    2  7  1834735   7402415  0/3    0/0    1926     342      smallvec             0.6.10          ↓0.6.7
pass    1  2  1338361  16109385  5/5    0/0    2354      32 CB   log                  0.4.8           =
none    0  0  5291155   6938235  0/6    0/0    2039       0      unicode-bidi         0.3.4           
pass    1  3  1873481  10777396  1/1    0/0    2436     226 CB   byteorder            1.3.2           =
none    0  0  2440672   7318535  3/6    0/0   12912      20      unicode-normalization 0.1.8           
pass    1  2  3684117   7278412  0/2    0/0   16499       1      idna                 0.1.5           ↑0.2.0             
none    0  0  2967585   8624610  1/4    0/0    3457       2      url                  1.7.2                              
pass    1  1  1105435  19510584  4/4    0/0   58231      37 CB   libc                 0.2.62          =
pass    1  1  3978248   4858110  1/1    0/0     275      75      iovec                0.1.2           =                  
pass    1  1  2015676   5586938  1/1    0/0    2885     274      bytes                0.4.12          =                  
none    0  0   633031   3084299  2/2    0/0    8156     154      http                 0.1.18          
4 Likes

With a reqwest 0.10 reqwest v0.10 - seanmonstar the dependency bloat might be lower. I need to investigate.

Edit:

> cargo crev c v  reqwest 0.10.0 --target --no-dev-dependencies --recursive
status issues  lines geiger flgs crate                version         
none    0   0    105      2      futures-sink         0.3.1          
none    0   0    347      3      percent-encoding     2.1.0          
pass    0   0    308      1      itoa                 0.4.4          
pass    0   0    257      0      fnv                  1.0.6          
pass    0   0     54      0      matches              0.1.8          
none    0   0    304      2      slab                 0.4.2          
none    0   0    482     69      futures-task         0.3.1          
none    0   0    257      0      pin-project-lite     0.1.1          
none    0   0    308     32      futures-core         0.3.1          
pass    0   0    134      8      lazy_static          1.4.0          
none    0   0    538      0      unicode-xid          0.2.0          
none    0   0   2091    345      smallvec             1.1.0          
pass    0   0    118      0      cfg-if               0.1.10         
none    0   0     44      0      tower-service        0.3.0          
none    0   0   1431     79      futures-channel      0.3.1          
none    0   0   3125      0      base64               0.11.0         
none    0   0     47      0      pin-utils            0.1.0-alpha.4  
none    0   0    380      0      version_check        0.9.1          
none    0   0   1036      2      mime                 0.3.14         
pass    0   0    105     14      try-lock             0.2.2          
none    0   0   3042      0 CB   unicase              2.6.0          
none    0   0   1606     97 CB   httparse             1.3.4          
none    0   0   6048      2 CB   mime_guess           2.0.1          
none    0   0   3590      0 CB   proc-macro2          1.0.6          
none    0   0    509      0      dtoa                 0.4.4          
none    0   0    383      0      autocfg              0.1.7          
none    0   0   2924    257      bytes                0.5.3          
none    0   0   4525      0 CB   quote                1.0.2          
none    0   0   2063    299 CB   memchr               2.2.1          
none    0   0   2093      0      unicode-bidi         0.3.4          
none    0   0  25835    365      unicode-normalization 0.1.11         
pass    0   0   2472     32 CB   log                  0.4.8          
none    0   0  11495    419      http                 0.2.0          
none    0   0  30124     35 CB   syn                  1.0.11         
none    0   0  64369     23 CB   libc                 0.2.66         
none    0   0  31450     35 CB   pin-project-internal 0.4.6          
none    0   0  14258    548      futures-util         0.3.1          
none    0   0  44504    366      idna                 0.2.0          
none    0   0   5004      5 CB   indexmap             1.3.0          
none    0   0   2871     46 CB   want                 0.3.0          
none    0   0  66022    382 CB   net2                 0.2.33         
none    0   0  11739    427      http-body            0.3.1          
none    0   0  48341    371      url                  2.1.0          
none    0   0  31491     78 CB   pin-project          0.4.6          
none    0   0  64649     98 CB   iovec                0.1.4          
none    0   0  36761     35 CB   serde_derive         1.0.104        
pass    0   0 165863    241 CB   winapi               0.3.8          
none    0   0 233126    470 CB   time                 0.1.42         
none    0   0  46584     35 CB   serde                1.0.104        
none    0   0  96934    415 CB   serde_urlencoded     0.6.1          
none    0   0  81054   1472 CB   mio                  0.6.21         
none    0   0 105948   3379 CB   tokio                0.2.6          
none    0   0 107674   3428 CB   tokio-util           0.2.0          
none    0   0 151671   4126 CB   h2                   0.2.1          
none    0   0 367088   4839 CB   hyper                0.13.1         
none    0   0 131316    935 CB   encoding_rs          0.8.22         
none    0   0 580696   6187 CB   reqwest              0.10.0  

Well, that's still quite a bit.

https://crates.io/crates/http_req is also a no-async HTTP client with no egregious dependencies, and it even has a reasonable amount of downloads.

1 Like

So I've founds a bunch more HTTP clients and smoke-tested them all to figure out if they're any good:

TL;DR: found serious bugs in all of them.

7 Likes

I'm not a reqwest user, but all the bugs you faithfully reported (including some in http listed under reqwest in your writeup) have been fixed, as far as I can tell? Perhaps some errata would be kind?

I'm a hyper user so the http one's could effect me as well (I've contributed fixes to some prior ones.) It might be interesting for you to compare the dependency reduction with strait hyper, even though its obviously not entirely a fair comparison, since hyper is by design missing some necessary HTTP client features.

You could look at body-image-futio crate (uses hyper by feature gate) as it does expose a simple synchronous interface option, though one that is obviously opinionated and not exactly a general purpose http client on its own. While I say its not general purpose, it does include things like timeouts, missing in some of of your other crates, and body decompression.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.