How to use `hyper::Client` in Axum?

help
1

Hello,

I'm working on an API gateway in Axum, which should detect requests on multiple routes and call the correct service based on the route. For example:

  • request on localhost:2000/users/<path> is translated to a request to localhost:3000/users/<path>
  • request on localhost:2000/profiles/<path> is translated to a request to localhost:4000/profiles/<path>

To do this I took the Axum example of a reverse proxy (Axum reverse proxy example) and modified it like that:

#[tokio::main]
async fn main() {
    // Hyper client for making requests
    let client = hyper::Client::new();

    // Axum router
    let router: Router<Body> = Router::new()
        .route("/users/*path"   , any(request_handler))
        .route("/profiles/*path", any(request_handler))
        .layer(Extension(client));

    // Start server
    Server::bind(&"0.0.0.0:2000".parse().unwrap())
        .serve(router.into_make_service())
        .await
        .unwrap();
}

async fn request_handler(
    Extension(client): Extension<hyper::Client<HttpConnector>>,
    mut req: Request<Body>,
) -> impl IntoResponse {
    // Extract path
    let path = req.uri().path();
    let path_query = req
        .uri()
        .path_and_query()
        .map(|v| v.as_str())
        .unwrap_or(path);

    // Get first part of the path
    let path_parts: Vec<&str> = path.split("/").collect();
    let path_base = path_parts.get(1).expect("no base path");

    // Map first part of path to host
    let host: String;
    match *path_base {
        "users"     => host = "http://0.0.0.0:3000".into(),
        "profiles"  => host = "http://0.0.0.0:4000".into(),
        &_          => todo!()
    }

    // Construct URI
    let uri = format!("{}{}", host, path_query);

    // Prepare request and return
    *req.uri_mut() = Uri::try_from(uri).unwrap();
    client.request(req).await.unwrap()
}

According to the documentation of hyper::Client cloning an instance of a client is the recommended way to share a hyper::Client. And as far as I understand the use of axum::extension::Extension as a layer clones the hyper::Client. So far so good.

What I'm curious about now is the following:

  1. Is the shared hyper::Client a problem regarding performance and concurrency?
  2. Which connection pool does my instance of hyper::Client use? Does it share the connection pool with Axum? If so, how does my instance of hyper::Client know which pool to use or that there is an existing pool?

And some general questions that came up during research:

  1. Where does Axum create an instance/multiple instances of hyper::Client? I looked through the source code but couldn't find it.
  2. Does Axum or Hyper take care of spawning Tokio tasks? Is every request handled in a seperate tokio::task?

Thanks to everyone who made it through all of my text :smiley:
Any help is very much appreciated. Thanks in advance.

2

This will depend on the exact details of how your application works, and how it gets used in practice. Using a shared Client is a good default though.

Since you appear to be connecting to another local server port, connection pooling might not be as beneficial as it is when connecting over the internet and using a single shared Client might not matter much.

Axum doesn't use a client connection pool, because it's a server. Each newly constructed hyper::client::Client gets it's own connection pool, which is why sharing a single Client is recommended[1].

An HTTP server that supports keep-alive will keep connections around under some circumstances in case the client sends another request over the same connection. That's a little different from the usual usage of the term "connection pool".

Axum is an HTTP server, it doesn't need a client instance internally. HTTP clients make requests, HTTP servers respond to requests.

As I understand it, hyper spawns a task for each new connection. Multiple requests on the same connection will reuse the existing task. The service module docs page has some more details on the distinction between what happens per connection and what happens per request.

I wasn't able to find any documentation that explicitly says that a single connection maps to a single spawned task, but skimming through some of the source code in hyper's server.rs it does look like that's probably the case based on some of the comments.

  1. cloning a Client is still sharing a single Client ↩︎

3 Likes
3

Oh wow, thank you so much for your detailed reply @semicoleon. You have helped me a lot!

4

May I ask why you chose hyper::Client instead of reqwest? (I'm not implying judgement, I'm only curious if/how you considered the choice to help inform myself)

5

Hello @EdmundsEcho, of course you can ask :slight_smile: Axum is build on top of Hyper and as I wrote in my original question I took inspiration from the reverse proxy example in the Axum repository. In this example they of course use Hyper and that’s why I chose Hyper.
I’ve never really used Reqwest, only heard of it. Do you think it would be beneficial?

6

Well, reqwest as an async client is using hyper internally, too, so if you're concerned about "not using two HTTP stacks at once" - that's no problem. It might be easier to use, however, since hyper is rather low-level.

1 Like
7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.