A proxy with axum 0.7 and reqwest 0.12 based on http 1?

help
1

I have the below code:

#[tokio::main]
async fn main() {
  let app = Router::new()
    .route("/", get(handler))
    .route("/custom/*key", get(proxy).post(proxy));

  // ...
}

pub async fn proxy(mut req: Request<Body>) -> Result<Response<Body>, StatusCode> {
  let path_query = req
    .uri()
    .path_and_query()
    .map_or_else(|| req.uri().path(), |v| v.as_str());

  let uri = format!("http://localhost:9000/{}", path_query);

  *req.uri_mut() = Uri::try_from(uri).unwrap();

  let http_client = reqwest::Client::new();

  let res = http_client.execute(req).await.unwrap()

  if res.status().is_server_error() {
    return Err(StatusCode::INTERNAL_SERVER_ERROR);
  }

  Ok(res)
}

But obviously this doesn't work:

error[E0308]: mismatched types
     |
35   |     let res = http_client.execute(req).await.unwrap();
     |                           ------- ^^^ expected `Request`, found `Request<Incoming>`
     |                           |
     |                           arguments to this method are incorrect
     |
     = note: `Request<Incoming>` and `reqwest::Request` have similar names, but are actually distinct types
note: `Request<Incoming>` is defined in crate `http`
    --> C:\Users\Fred\.cargo\registry\src\index.crates.io-6f17d22bba15001f\http-1.1.0\src\request.rs:158:1
     |
158  | pub struct Request<T> {
     | ^^^^^^^^^^^^^^^^^^^^^
note: `reqwest::Request` is defined in crate `reqwest`
    --> C:\Users\Fred\.cargo\registry\src\index.crates.io-6f17d22bba15001f\reqwest-0.12.4\src\async_impl\request.rs:22:1
     |
22   | pub struct Request {
     | ^^^^^^^^^^^^^^^^^^
note: method defined here
    --> C:\Users\Fred\.cargo\registry\src\index.crates.io-6f17d22bba15001f\reqwest-0.12.4\src\async_impl\client.rs:1965:12
     |
1965 |     pub fn execute(
     |            ^^^^^^^

and:

error[E0308]: mismatched types
    |
43  |     Ok(res)
    |     -- ^^^ expected `Response<Incoming>`, found `Response`
    |     |
    |     arguments to this enum variant are incorrect
    |
    = note: `reqwest::Response` and `Response<Incoming>` have similar names, but are actually distinct types
note: `reqwest::Response` is defined in crate `reqwest`
   --> C:\Users\Fred\.cargo\registry\src\index.crates.io-6f17d22bba15001f\reqwest-0.12.4\src\async_impl\response.rs:29:1

I'm using axum 0.7.5 and reqwest 0.12 (both on http 1).

Is it possible to make it work?

@jofas here you said to wait, and here we are today! :slight_smile:

2

You need to convert http::Request to reqwest::Request. You can do that using this conversion. Same goes for reqwest::Response to http::Response<reqwest::Body>, which can be done using this conversion, or rather the corresponding Into implementation you get for free:

use std::convert::TryFrom;

pub async fn proxy(mut req: Request<Body>) -> Result<Response<Body>, StatusCode> {
  // ...

  let http_client = reqwest::Client::new();

  let res = http_client.execute(reqwest::Request::try_from(req).unwrap()).await.unwrap()

  if res.status().is_server_error() {
    return Err(StatusCode::INTERNAL_SERVER_ERROR);
  }

  Ok(res.into())
}
1 Like
3

Thanks. I tried before opening this question:

error[E0277]: the trait bound `reqwest::Request: std::convert::TryFrom<hyper::Request<axum::body::Body>>` is not satisfied
   |
37 |         .request(reqwest::Request::try_from(req).unwrap())
   |                  ^^^^^^^^^^^^^^^^ the trait `std::convert::TryFrom<hyper::Request<axum::body::Body>>` is not implemented for `reqwest::Request`
   |
   = help: the trait `std::convert::TryFrom<hyper::Request<T>>` is implemented for `reqwest::Request`

Why?

4

If I use instead:

let res = http_client
    .execute(reqwest::Request::try_from(req.extract()).unwrap())
    .await
    .unwrap();

the error is:

error[E0277]: the trait bound `reqwest::Request: std::convert::TryFrom<std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<_, _>> + std::marker::Send>>>` is not satisfied
   |
38 |         .execute(reqwest::Request::try_from(req.extract()).unwrap())
   |                  ^^^^^^^^^^^^^^^^ the trait `std::convert::From<std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<_, _>> + std::marker::Send>>>` is not implemented for `reqwest::Request`, which is required by `reqwest::Request: std::convert::TryFrom<std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<_, _>> + std::marker::Send>>>`
   |
   = help: the trait `std::convert::TryFrom<hyper::Request<_>>` is implemented for `reqwest::Request`
   = help: for that trait implementation, expected `hyper::Request<_>`, found `std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<_, _>> + std::marker::Send>>`
   = note: required for `std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<_, _>> + std::marker::Send>>` to implement `std::convert::Into<reqwest::Request>`
   = note: required for `reqwest::Request` to implement `std::convert::TryFrom<std::pin::Pin<std::boxed::Box<dyn futures::Future<Output = std::result::Result<_, _>> + std::marker::Send>>>`

Why?

5

This is indeed a bit annoying, as we can't convert axum::body::Body to reqwest::Body and vice versa. We must find some type that we can use in-between converting one to the other. In the example below I used Bytes as that middle type, retrieving it from the request and the response via http_body_util::BodyExt::collect:

use axum::body::{Body, Bytes};
use axum::extract::Request;
use axum::response::Response;

use http_body_util::BodyExt;

pub async fn proxy(req: Request<Body>) -> Response<Body> {
    let http_client = reqwest::Client::new();

    let (headers, body) = req.into_parts();

    let body = body.collect().await.unwrap().to_bytes();

    let req: Request<Bytes> = Request::from_parts(headers, body);

    let req = reqwest::Request::try_from(req).unwrap();

    let res: Response<reqwest::Body> = http_client.execute(req).await.unwrap().into();

    let (headers, body) = res.into_parts();

    let body = body.collect().await.unwrap().to_bytes().into();

    Response::from_parts(headers, body)
}

You can remove the response conversion by returning Response<reqwest::Body> instead of Response<axum::body::Body>, which should be fine as reqwest::Body implements the Body trait:

use axum::body::{Body, Bytes};
use axum::extract::Request;
use axum::response::Response;

use http_body_util::BodyExt;

pub async fn proxy(req: Request<Body>) -> Response<reqwest::Body> {
    let http_client = reqwest::Client::new();

    let (headers, body) = req.into_parts();

    let body = body.collect().await.unwrap().to_bytes();

    let req: Request<Bytes> = Request::from_parts(headers, body);

    let req = reqwest::Request::try_from(req).unwrap();

    http_client.execute(req).await.unwrap().into()
}

I haven't figured out a better way yet, or what changes to the reqwest API can be made to make conversion between axum::body::Body and reqwest::Body easier and without the need of first collecting the body in memory. I tried reqwest::Body::wrap_stream with both axum::Body::into_data_stream and http_body_util::BodyStream, both failed because axum::body::Body is not Sync.

So for now I have no better solution. There might be some way to do the proxying over tower's interface, as reqwest::Client implements the Service trait, but I don't know tower nor axum well enough to give you any hints on how that might work. Maybe I'll give this some more thought and see if this might be worth raising an issue in reqwest.

1 Like
6

Thank you very much.

1 Like
7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.