Using Proxies in Rust with reqwest & hyper

Using proxies in Rust with reqwest involves configuring a ClientBuilder with proxy settings, while hyper, being a lower-level HTTP library, requires manual connection establishment through a proxy server.

Proxy servers act as intermediaries for network requests, offering benefits such as anonymity, access to geo-restricted content, load balancing, and traffic filtering. Rust's asynchronous HTTP clients, reqwest and hyper, provide different approaches to integrating proxy functionality. reqwest offers high-level, built-in support, while hyper demands more granular control over the connection process.

Using Proxies with `reqwest`

reqwest is a popular, user-friendly HTTP client for Rust, built on top of hyper. It simplifies common HTTP tasks, including proxy configuration. reqwest's ClientBuilder provides methods to set up various proxy types.

Proxy Configuration

To use a proxy with reqwest, create a reqwest::Client instance using reqwest::ClientBuilder and its proxy() method. The proxy() method accepts a reqwest::Proxy object, which can be constructed for different proxy schemes:

HTTP Proxy: Configured using Proxy::http(url), for plain HTTP requests.
HTTPS Proxy (CONNECT method): Configured using Proxy::https(url), for HTTPS requests. reqwest uses the CONNECT method to tunnel the TLS connection through the proxy.
SOCKS5 Proxy: Configured using Proxy::socks5(url), for SOCKS5 proxy servers.
Unified Proxy: Proxy::all(url) configures a single proxy for both HTTP and HTTPS requests.

Proxy Authentication

Many proxy servers require authentication. reqwest::Proxy supports basic authentication using the basic_auth() method.

Environment Variable Support

By default, reqwest's ClientBuilder automatically checks for HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables. If these are set, reqwest will use them unless explicitly overridden by ClientBuilder::no_proxy() or by configuring a specific proxy.

Example: `reqwest` Proxy Usage

This example demonstrates configuring reqwest to use HTTP, HTTPS, and SOCKS5 proxies, including basic authentication.

use reqwest::{Client, Error, Proxy};
use std::time::Duration;

#[tokio::main]
async fn main() -> Result<(), Error> {
    // 1. HTTP Proxy
    let http_proxy_url = "http://user:password@your-http-proxy.com:8080";
    let http_client = Client::builder()
        .proxy(Proxy::http(http_proxy_url)?)
        .timeout(Duration::from_secs(10))
        .build()?;

    println!("HTTP Proxy request...");
    let http_res = http_client.get("http://httpbin.org/ip").send().await?;
    println!("HTTP Proxy Response: {:?}", http_res.text().await?);

    // 2. HTTPS Proxy (CONNECT method)
    // Note: For HTTPS, the proxy URL itself might be HTTP, but it tunnels HTTPS traffic.
    let https_proxy_url = "http://user:password@your-https-proxy.com:8080";
    let https_client = Client::builder()
        .proxy(Proxy::https(https_proxy_url)?)
        .timeout(Duration::from_secs(10))
        .build()?;

    println!("\nHTTPS Proxy request...");
    let https_res = https_client.get("https://httpbin.org/ip").send().await?;
    println!("HTTPS Proxy Response: {:?}", https_res.text().await?);

    // 3. SOCKS5 Proxy
    let socks5_proxy_url = "socks5://user:password@your-socks5-proxy.com:1080";
    let socks5_client = Client::builder()
        .proxy(Proxy::socks5(socks5_proxy_url)?)
        .timeout(Duration::from_secs(10))
        .build()?;

    println!("\nSOCKS5 Proxy request...");
    let socks5_res = socks5_client.get("http://httpbin.org/ip").send().await?;
    println!("SOCKS5 Proxy Response: {:?}", socks5_res.text().await?);

    // 4. Unified Proxy (applies to both HTTP and HTTPS requests)
    let all_proxy_url = "http://user:password@your-all-proxy.com:8080";
    let all_client = Client::builder()
        .proxy(Proxy::all(all_proxy_url)?)
        .timeout(Duration::from_secs(10))
        .build()?;

    println!("\nUnified Proxy (HTTP) request...");
    let all_http_res = all_client.get("http://httpbin.org/ip").send().await?;
    println!("Unified Proxy (HTTP) Response: {:?}", all_http_res.text().await?);

    println!("\nUnified Proxy (HTTPS) request...");
    let all_https_res = all_client.get("https://httpbin.org/ip").send().await?;
    println!("Unified Proxy (HTTPS) Response: {:?}", all_https_res.text().await?);

    Ok(())
}

Note: Replace your-http-proxy.com:8080, your-https-proxy.com:8080, and your-socks5-proxy.com:1080 with actual proxy server addresses and ports, including user:password for authenticated proxies.

Using Proxies with `hyper`

hyper is a low-level, high-performance HTTP library that forms the foundation for many other Rust HTTP crates, including reqwest. Unlike reqwest, hyper does not offer built-in, direct proxy configuration methods for its client. Instead, proxy usage with hyper requires manual management of the underlying TCP connection or integrating with external proxy-specific crates.

Manual Connection Management

To use a proxy with hyper, you must establish the connection to the proxy server first, then instruct hyper to use this pre-established connection or a custom connector that handles the proxy logic.

HTTP Proxy for HTTP Requests

For plaintext HTTP requests through an HTTP proxy, the client establishes a TCP connection to the proxy server. It then sends the HTTP request directly to the proxy, specifying the full target URL in the request line (e.g., GET http://target.com/path HTTP/1.1). The Host header should still refer to the target server.

HTTP Proxy for HTTPS Requests (CONNECT Method)

For HTTPS requests, the client first sends an HTTP CONNECT request to the proxy, instructing it to establish a TCP tunnel to the target host and port. Once the proxy responds with 200 OK, the client then performs the TLS handshake directly with the target server through the established tunnel.

SOCKS5 Proxy

SOCKS5 proxies operate at a lower level than HTTP proxies and support various protocols, including TCP and UDP. Integrating SOCKS5 with hyper typically involves using a dedicated SOCKS5 client library (e.g., tokio-socks) to establish the proxied connection, which is then passed to hyper's connection builder.

Example: `hyper` Proxy Usage (HTTP Proxy for HTTP)

This example demonstrates how to make a basic HTTP request through an HTTP proxy using hyper by manually constructing the request and managing the TCP stream. This illustrates the underlying mechanism; for robust solutions, consider wrapping this logic or using a dedicated connector.

use hyper::{Body, Client, Request, Uri};
use hyper::client::conn::Builder;
use tokio::net::TcpStream;
use std::str::FromStr;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let proxy_addr = "127.0.0.1:8080"; // Replace with your HTTP proxy address
    let target_url = "http://httpbin.org/ip";

    println!("Connecting to proxy: {}", proxy_addr);
    let stream = TcpStream::connect(proxy_addr).await?;

    // For HTTP proxy, we connect to the proxy and send the full URI in the request.
    // No CONNECT method needed for plain HTTP targets.
    let (mut sender, conn) = Builder::new()
        .http1_only(true) // Ensure HTTP/1.1 for HTTP proxy compatibility
        .handshake(stream)
        .await?;

    tokio::spawn(async move {
        if let Err(e) = conn.await {
            eprintln!("Connection error: {:?}", e);
        }
    });

    let req = Request::builder()
        .uri(Uri::from_str(target_url)?) // Absolute URI for HTTP proxy forwarding
        .header("Host", "httpbin.org") // Host header for the target server
        .body(Body::empty())?;

    println!("Sending request through proxy to: {}", target_url);
    let res = sender.send(req).await?;

    println!("Response Status: {}", res.status());
    let body_bytes = hyper::body::to_bytes(res.into_body()).await?;
    println!("Response Body: {}", String::from_utf8_lossy(&body_bytes));

    Ok(())
}

Note: This example assumes a basic HTTP proxy that doesn't require authentication and directly forwards HTTP requests. For HTTPS or authenticated proxies, the logic becomes significantly more complex, involving CONNECT requests or specific proxy negotiation libraries.

Using External Connectors for `hyper`

For more complex proxy scenarios with hyper, especially for SOCKS5 or authenticated HTTP/HTTPS proxies, it is common to implement a custom hyper::client::connect::Connection or use a crate that provides such an implementation. For instance, for SOCKS5, you would typically use a crate like tokio-socks to establish a SOCKS5 proxied TcpStream, then use hyper::client::conn::Builder::handshake with this stream.

`reqwest` vs. `hyper` for Proxy Usage

Feature	`reqwest`	`hyper`
Proxy Configuration	Built-in, high-level methods (`ClientBuilder::proxy()`)	Manual implementation of `hyper::client::connect::Connection` or integration with external libraries
Ease of Use	High-level, simple for common proxy types	Low-level, complex for proxies, requires more boilerplate
Proxy Types Supported	HTTP, HTTPS (CONNECT), SOCKS5	Requires manual handling for each type; often uses external crates for SOCKS5 or complex authentication
Authentication	Built-in (`Proxy::basic_auth()`)	Manual header injection (`Proxy-Authorization`) or specific proxy library features
Control Level	Moderate, abstracts connection details	High, granular control over connections and HTTP protocol details
Dependencies	Simpler for proxying, `reqwest` handles underlying complexity	May require additional crates (`tokio-socks`, custom connectors) for robust proxy logic
Typical Use Case	Most applications requiring HTTP requests with proxy support, prioritizing development speed and simplicity	Building custom HTTP clients, servers, or highly optimized network components where fine-grained control over network connections is paramount

Best Practices and Considerations

Error Handling

Proxy connections can fail for various reasons (network issues, proxy down, authentication failure). Implement robust error handling for proxy-related operations, especially when manually managing connections with hyper. reqwest returns reqwest::Error which can be inspected for proxy-specific issues.

Proxy Types and Protocol Compatibility

HTTP Proxies: Primarily for HTTP traffic. Can tunnel HTTPS traffic using the CONNECT method.
SOCKS5 Proxies: Protocol-agnostic, supporting TCP and UDP. Often preferred for non-HTTP traffic or when a higher degree of anonymity is desired.
Ensure the proxy type configured matches the proxy server's capabilities and the type of traffic you intend to send.

Authentication

Always provide correct credentials for authenticated proxies. Misconfigured authentication is a common source of proxy connection errors. For reqwest, use Proxy::basic_auth(). For hyper or custom connectors, send the Proxy-Authorization header with appropriate credentials (e.g., Base64 encoded username:password).

Performance Implications

Using a proxy introduces an additional hop in the network path, which can increase latency. The performance overhead depends on the proxy server's location, load, and network conditions between your client, the proxy, and the target server.

Security Considerations

Trust: Only use trusted proxy servers. Malicious proxies can inspect, modify, or log your traffic.
HTTPS Proxying: When using an HTTPS proxy with the CONNECT method, the proxy establishes a tunnel, and your client performs the TLS handshake directly with the target server. This means the proxy cannot decrypt the content of your HTTPS traffic (unless it's a "man-in-the-middle" proxy with a custom root certificate installed on your client). reqwest handles this securely by default.
Proxy Headers: Proxies often add headers like X-Forwarded-For. Be aware of these if anonymity is a primary concern.

Environment Variable Interaction

reqwest automatically respects HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables. If you need to override this behavior, use ClientBuilder::no_proxy() or explicitly configure proxies. When using hyper directly, you must manually parse and apply these environment variables if desired.

Analysis & Check

Security & Network

Generators

9 tools

Using Proxies in Rust