Help with Hyper Proxy client CONNECT method

help
1

Hi!

I'm trying to use a hyper/hyper_proxy/hyper_tls to make requests through an Apache proxy via HTTP CONNECT. I've followed this: hyper_proxy - Rust

I can make the request via curl successfully:

curl -v https://$targethostname:$targetport/$targetendpoint \
    --cert api_user.crt \
    --key api_user.key \
    --cacert combined_root.crt \
    --proxy https://$proxyhostname:$proxyport \
    --proxy-cert api_user.crt \
    --proxy-key api_user.key \
    --proxy-cacert combined_root.crt
# ...
# * Proxy replied 200 to CONNECT request
# ...
# < HTTP/1.1 200 OK
# ...
# <Expected Response Body>

Running the rust code below gives me an error:

./canary --proxy-host $proxyhostname \
             --proxy-port $proxyport \
             --api-url https://$targethostname:$targetport/$targetendpoint 
2023-05-09T12:10:08.223020Z  INFO canary: Canary starting up
2023-05-09T12:10:08.223615Z  INFO canary: Canary (canary) version: 0.1.0, git commit hash: b1889ff
2023-05-09T12:10:08.240346Z TRACE hyper::client::pool: checkout waiting for idle connection: ("https", $targethostname:11839)
2023-05-09T12:10:08.241067Z TRACE hyper::client::connect::http: Http::connect; scheme=Some("https"), host=Some("$proxyhostname"), port=Some(Port(11839))
2023-05-09T12:10:08.241617Z DEBUG hyper::client::connect::http: connecting to $proxyhostname:11839
2023-05-09T12:10:08.242665Z DEBUG hyper::client::connect::http: connected to $proxyhostname:11839
2023-05-09T12:10:08.244929Z TRACE hyper::client::pool: checkout dropped for ("https", $RESOURCE:11839)
Error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919: (self signed certificate in certificate chain)

Caused by:
    0: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919: (self signed certificate in certificate chain)
    1: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919:
command terminated with exit code 1

I think I need to supply the CA to the inner client, but it's unclear how to do that. Suggestions?

Code example:

use anyhow::{Context, Result};
use clap::Parser;
use futures::{TryFutureExt, TryStreamExt};
use hyper::{Client, Request};
use hyper_proxy::{Intercept, Proxy, ProxyConnector};
use hyper_tls::HttpsConnector;
use native_tls::TlsConnector;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let cli = Cli::parse();

    // Setup tracing
    tracing_subscriber::registry()
        .with(tracing_subscriber::EnvFilter::new(
            std::env::var("RUST_LOG")
                .unwrap_or_else(|_| "canary=trace,tower_http=trace,hyper=trace,hyper_proxy=trace,hyper_tls=trace,native_tls=trace,http=trace".into()),
        ))
        .with(tracing_subscriber::fmt::layer())
        .init();

    tracing::info!("Starting up");

    let cert = std::fs::read("api_user.crt").context("missing user cert")?;
    let key = std::fs::read("api_user.key").context("missing user key")?;
    let identity =
        native_tls::Identity::from_pkcs8(&cert, &key).context("From PKCS8 PEM failed")?;

    let pem = std::fs::read("combined_root.crt").context("missing server ca")?;
    let ca = native_tls::Certificate::from_pem(&pem).context("From PEM failed")?;

    let proxy = {
        let proxy_uri = format!("https://{}:{}", cli.proxy_host, cli.proxy_port)
            .parse()
            .unwrap();
        let proxy = Proxy::new(Intercept::All, proxy_uri);

        let connector = TlsConnector::builder()
            .identity(identity)
            .add_root_certificate(ca)
            .build()
            .ok();
        let https = HttpsConnector::new();
        let mut proxy_connector = ProxyConnector::from_proxy(https, proxy).unwrap();
        proxy_connector.set_tls(connector);
        proxy_connector
    };

    let req = Request::get(cli.api_url)
        .body(hyper::Body::empty())
        .unwrap();

    let client = Client::builder().build(proxy);
    let fut_https = client
        .request(req)
        .and_then(|res| res.into_body().map_ok(|x| x.to_vec()).try_concat())
        .map_ok(move |body| ::std::str::from_utf8(&body).unwrap().to_string());

    let https_res = fut_https.await?;
    tracing::info!(?https_res, "Result");

    Ok(())
}

#[derive(Parser)]
#[command(author, version, about, long_about = None)]
struct Cli {
    ///
    #[arg(long)]
    api_url: String,

    ///
    #[arg(long)]
    proxy_host: String,

    ///
    #[arg(long)]
    proxy_port: u16,
}

THANK YOU!

2

I don't think I have a good answer, but since no one else has chimed in yet... The error message, "self signed certificate in certificate chain" means that TLS is failing. You might need to set the CA root on the HttpsConnector in addition to the ProxyConnector. Just like the curl example does.

It should be done pretty much the same way, but I haven't tested it:

        let connector = TlsConnector::builder()
            .identity(identity)
            .add_root_certificate(ca)
            .build()
            .ok();
        let https = HttpsConnector::from((HttpConnector::new(), connector.clone().unwrap()));
        let mut proxy_connector = ProxyConnector::from_proxy(https, proxy).unwrap();
        proxy_connector.set_tls(connector);

AFAICT, that should do it.

Also, I found client: tunneling via CONNECT method example · Issue #1884 · hyperium/hyper (github.com). There is some discussion there which might be of use. Basically, the CONNECT method doesn't appear to be automatically used by the proxy. You might run into this issue after solving the TLS problem.

3

Thanks for the response and thanks for the suggestion.

I tried a few variants on the following but I'm still getting the "self signed certificate in certificate chain" error, which indicates to me that I don't have the CA set properly.

let connector = TlsConnector::builder()
    .identity(identity)
    .add_root_certificate(ca)
    .danger_accept_invalid_hostnames(true)
    .danger_accept_invalid_certs(true)
    .build()
    .ok();
let tokio_connector: TokioTlsConnector = connector.clone().unwrap().into();
let https = HttpsConnector::from((HttpsConnector::new(), tokio_connector));
let mut proxy_connector = ProxyConnector::from_proxy(https, proxy).unwrap();
proxy_connector.set_tls(connector);
4

I was able to get this working using Reqwest:

use reqwest::{
    tls::{Certificate, Identity},
    ClientBuilder, Proxy,
};

// .....

        let proxy = match Proxy::all(format!("https://{}:{}", cli.proxy_host, cli.proxy_port)) {
            Ok(p) => p,
            Err(e) => {
                tracing::error!(?e, "Failed to create proxy");
                continue;
            }
        };

        let client = match ClientBuilder::new()
            .user_agent(APP_USER_AGENT)
            .add_root_certificate(ca)
            .proxy(proxy)
            .https_only(true)
            .identity(identity)
            .build()
        {
            Ok(c) => c,
            Err(e) => {
                tracing::error!(?e, "Failed to build client");
                continue;
            }
        };

        let url = cli.get_api_url();
        tracing::debug!(url, "Making API Proxy request");

        let resp = match client.get(&url).send().await {
            Ok(r) => r,
            Err(e) => {
                tracing::error!(?e, "Error making request");
                continue;
            }
        };

        tracing::info!(?url, status=?resp.status(), body=?resp.text().await?, "Response");
5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.