Charles Proxy

Charles Proxy facilitates debugging HTTP requests by acting as a man-in-the-middle proxy, capturing and allowing inspection and modification of all network traffic between a client application and target servers.

Charles Proxy functions as a local HTTP proxy server, enabling developers and QA engineers to observe, intercept, and manipulate network communication. This capability is critical for understanding client-server interactions, diagnosing integration issues, testing edge cases, and verifying API behavior, especially when working with or developing services that interact with an upstream proxy.

Setting Up Charles Proxy

Installation and Initial Launch

Download and install Charles Proxy from its official website. Upon first launch, Charles typically attempts to configure itself as the system proxy. Accept this prompt to allow Charles to intercept traffic from applications respecting system proxy settings.

Configuring System Proxy Settings

When Charles is running, it automatically sets the system's HTTP/HTTPS proxy settings to 127.0.0.1 (localhost) on port 8888 (default). Most applications, including web browsers, respect these settings.

To verify or manually configure:
* macOS: System Settings > Network > Wi-Fi/Ethernet > Details > Proxies. Ensure "Web Proxy (HTTP)" and "Secure Web Proxy (HTTPS)" are enabled and point to 127.0.0.1:8888.
* Windows: Settings > Network & Internet > Proxy > Manual proxy setup. Ensure "Use a proxy server" is enabled with 127.0.0.1 and port 8888.
* Linux (GNOME): Settings > Network > Network Proxy. Set Method to "Manual" and configure HTTP/HTTPS proxy to 127.0.0.1:8888.

Configuring Client Applications

Some applications, particularly command-line tools or specific SDKs, may not respect system-wide proxy settings. In these cases, configure the application directly.

Example: curl with proxy

curl -x http://127.0.0.1:8888 https://api.example.com/data

Example: Node.js axios with proxy

const axios = require('axios');

axios.get('https://api.example.com/data', {
  proxy: {
    host: '127.0.0.1',
    port: 8888
  }
})
.then(response => {
  console.log(response.data);
})
.catch(error => {
  console.error(error);
});

SSL Proxying for HTTPS Traffic

To inspect HTTPS traffic, Charles Proxy must perform SSL (or TLS) proxying. This involves Charles generating and signing certificates for the target domains, presenting them to the client, and then establishing its own secure connection to the real server.

Installing Charles Root Certificate

The client application must trust Charles's root certificate for SSL proxying to work without security warnings.

• Desktop Browsers/System:
  • Navigate to Help > SSL Proxying > Install Charles Root Certificate. Follow OS-specific prompts to install it into the system's trusted root certificate store.
• Mobile Devices:
  • Configure the mobile device to use Charles as its HTTP proxy (e.g., your_mac_ip:8888).
  • On the device, browse to chls.pro/ssl and follow instructions to download and install the certificate.

Enabling SSL Proxying for Specific Hosts

After installing the root certificate, enable SSL proxying for the specific hosts you wish to inspect.

1. Go to Proxy > SSL Proxying Settings....
2. Click "Add".
3. Enter the Host (e.g., api.example.com, *.example.com for a wildcard, or * for all hosts) and Port (typically 443 for HTTPS, or * for all ports).
4. Click "OK" and then "OK" again.

Traffic to the specified hosts will now be decrypted and visible in Charles.

Key Debugging Features

Traffic Monitoring and Filtering

The "Structure" and "Sequence" views display captured requests.
* Sequence View: Shows requests in chronological order.
* Structure View: Organizes requests by host and path.

To focus on relevant traffic:
* Filter Field: Use the filter field at the top of the Charles window to match URLs, hosts, or content.
* Focus Tool: Right-click a request and select "Focus" to hide all other hosts from the current view.
* Record Button: Toggle the record button (red circle) to start/stop capturing traffic.

Breakpoints

Breakpoints allow you to pause and modify requests before they are sent to the server, or responses before they are sent to the client.

1. Go to Proxy > Breakpoints Settings....
2. Click "Add" for requests or responses.
3. Specify the Host and Path (e.g., /login).
4. When a request/response matches, Charles will pause, and a "Breakpoints" tab will appear.
5. Modify headers, query parameters, or the body.
6. Click "Execute" to send the modified request/response.

Rewrite Tool

The Rewrite tool provides automated, persistent modifications to requests and responses. This is useful for simulating specific server behaviors or client conditions without manual intervention.

1. Go to Tools > Rewrite....
2. Enable Rewrite and add a new "Set".
3. Add "Locations" (URLs to apply rules to) using Host and Path.
4. Add "Rules" within the set:
   • Add Header/Remove Header: Modify HTTP headers.
   • Change Query Parameter: Modify URL query parameters.
   • Body Replace: Find and replace text within the request/response body.
   • URL Replace: Modify parts of the request URL.

Example: Modifying an API response body
A Rewrite rule could replace {"status": "active"} with {"status": "maintenance"} in a JSON response to test how a client handles different states.

Map Local / Map Remote

These tools redirect requests to different sources.

• Map Local: Serves a local file in place of a remote response.

  1. Go to Tools > Map Local....
  2. Enable Map Local and add a new entry.
  3. Specify the Host and Path of the remote URL.
  4. Select the local File path to serve.
     This is useful for quickly testing UI changes or simulating specific API responses without modifying the backend.

• Map Remote: Redirects a request for one URL to an entirely different URL.

  1. Go to Tools > Map Remote....
  2. Enable Map Remote and add a new entry.
  3. Specify the Host and Path of the original request.
  4. Specify the new Host and Path for the redirected request.
     Useful for redirecting API calls to a staging environment or a local development server.

Throttling

Throttling simulates slow network conditions, helping to identify performance bottlenecks or UI issues related to slow data loading.

1. Go to Proxy > Throttle Settings....
2. Enable "Enable Throttling".
3. Configure Bandwidth, Utilisation, and Latency parameters.
4. Optionally, select "Only for selected hosts" and add specific hosts to throttle.

Repeat / Advanced Repeat

These features allow re-sending requests.

• Repeat: Right-click a request and select "Repeat" to send it again immediately.
• Advanced Repeat: Right-click a request and select "Advanced Repeat" to send it multiple times with customizable concurrency and iteration counts. This is useful for load testing or checking for race conditions.

Debugging Upstream Proxy Interactions

When your client application is configured to use Charles, and Charles itself is configured to use an upstream proxy (e.g., a corporate proxy), Charles acts as an intermediary.

To configure Charles to use an upstream proxy:
1. Go to Proxy > External Proxy Settings....
2. Enable "Use external proxy".
3. Enter the Host, Port, and any required Username/Password for your upstream proxy.
4. Enable "SOCKS proxy" if your upstream proxy is a SOCKS proxy.

In this setup, Charles captures traffic from your client, then forwards it to the upstream proxy. This allows you to inspect the traffic before it reaches the upstream proxy, which is crucial for diagnosing issues where the upstream proxy might be modifying requests or blocking certain endpoints.

Common Issues and Troubleshooting

| Problem | Solution in Charles Proxy CHARLES PROXY: DEBUGGING HTTP REQUESTS THROUGH PROXY

Charles Proxy debugs HTTP requests by acting as a man-in-the-middle proxy, capturing all network traffic between a client application and target servers, allowing for detailed inspection and modification.

Overview

Charles Proxy is an HTTP proxy (HTTP/HTTPS/SOCKS) and HTTP monitor that runs on your local machine. It allows you to view all HTTP and HTTPS traffic between your machine and the Internet. When debugging services that interact with an upstream proxy, Charles can be configured to sit between the client and the upstream proxy, providing visibility into the exact requests being sent and received by your application.

Initial Setup

Installation

Download and install Charles Proxy from the official website (charlesproxy.com). Charles supports macOS, Windows, and Linux.

System Proxy Configuration

Upon launch, Charles typically prompts to configure itself as the system proxy. Accepting this allows Charles to automatically intercept traffic from applications configured to use system proxy settings (e.g., most web browsers, Java applications, .NET applications).

To manually verify or configure:
* Default HTTP/HTTPS Proxy Address: 127.0.0.1
* Default HTTP/HTTPS Proxy Port: 8888

For environments where Charles needs to connect to an external (upstream) proxy:
1. Navigate to Proxy > External Proxy Settings....
2. Check "Use external proxy".
3. Enter the Host and Port of the upstream proxy.
4. Provide Username and Password if required by the upstream proxy.
5. Check "SOCKS proxy" if the upstream proxy is a SOCKS proxy.

This configuration ensures Charles captures traffic from your client, then routes it through the specified external proxy, allowing you to inspect what is being sent to and received from the external proxy.

SSL Proxying for HTTPS Traffic

HTTP Secure (HTTPS) traffic is encrypted, preventing direct inspection. Charles employs SSL Proxying (man-in-the-middle attack) to decrypt and inspect HTTPS traffic.

Charles Root Certificate Installation

To perform SSL Proxying, Charles generates dynamic SSL certificates for each HTTPS domain. Your client application must trust Charles's root certificate to avoid SSL errors.

• Desktop OS (macOS/Windows/Linux):
  1. Go to Help > SSL Proxying > Install Charles Root Certificate.
  2. Follow the operating system's prompts to install the certificate into the trusted root certificate store.
• Mobile Devices (iOS/Android):
  1. Configure the mobile device's Wi-Fi settings to use Charles as its HTTP proxy (Charles's IP address and port 8888).
  2. On the mobile device, open a browser and navigate to chls.pro/ssl.
  3. Follow the instructions to download and install the Charles Root Certificate as a trusted credential.

Enabling SSL Proxying for Specific Hosts

After installing the root certificate, specify which HTTPS hosts Charles should decrypt.
1. Go to Proxy > SSL Proxying Settings....
2. Click "Add".
3. Enter the Host (e.g., api.yourdomain.com, *.yourdomain.com, or * for all hosts) and Port (typically 443, or * for all ports).
4. Click "OK".

Core Debugging Functionality

Traffic Capture and Filtering

Charles displays captured traffic in two primary views:
* Sequence: Chronological list of requests.
* Structure: Hierarchical view grouped by host and folder path.

To locate specific requests:
* Filter Field: Use the filter bar at the top to search by host, URL path, or content.
* Focus Tool: Right-click a request and select "Focus" to hide all other hosts from the current view.
* Record Button: Toggle the red circle icon to start/stop capturing traffic.

Breakpoints

Breakpoints allow for manual interception and modification of requests before they are sent, or responses before they reach the client.

1. Go to Proxy > Breakpoints Settings....
2. Click "Add" for "Request" or "Response" breakpoints.
3. Specify Host and Path patterns for the target URL.
4. When a matching request/response occurs, Charles pauses, displaying a "Breakpoints" tab.
5. Modify Headers, Query Parameters, or Body content.
6. Click "Execute" to proceed with the modified transaction.

Rewrite Tool

The Rewrite tool automates modifications to requests and responses based on predefined rules. This is useful for consistent testing scenarios without manual intervention.

1. Go to Tools > Rewrite....
2. Enable Rewrite and add a new "Set".
3. Define "Locations" (URLs where rules apply) using Host and Path.
4. Add "Rules" within the set:
   • Type: Add Header, Remove Header, Change Query Parameter, Host, Path, URL, Body.
   • Action: Specify Value or Regex for the modification.

Example: Simulating an API version change
A Rewrite rule could change the Accept header from application/json;v=1 to application/json;v=2 for all requests to a specific API endpoint.

Map Local / Map Remote

These features redirect requests to alternative resources.

• Map Local: Serves a local file in place of a remote server response.

  1. Go to Tools > Map Local....
  2. Enable Map Local and add a new entry.
  3. Specify the Host and Path of the remote URL to be mapped.
  4. Select the Local Path to the file that Charles should serve.
     This is useful for frontend development to test UI changes with specific mock data without backend modifications.

• Map Remote: Redirects a request from one URL to an entirely different URL.

  1. Go to Tools > Map Remote....
  2. Enable Map Remote and add a new entry.
  3. Specify the Host and Path of the original request.
  4. Specify the Host and Path of the new target URL.
     This is useful for redirecting production API calls to a staging environment or a local development server.

Throttling

Throttling simulates various network conditions (e.g., dial-up, 3G, custom speeds) to test application performance and responsiveness under suboptimal network connectivity.

1. Go to Proxy > Throttle Settings....
2. Check "Enable Throttling".
3. Select a Preset or configure Bandwidth, Utilisation, and Latency manually.
4. Optionally, check "Only for selected hosts" and add specific hosts to throttle.

Repeat / Advanced Repeat

These functions allow re-sending requests.
* Repeat: Right-click a request in the Charles session and select "Repeat" to resend it once.
* Advanced Repeat: Right-click a request and select "Advanced Repeat" to configure multiple repetitions, concurrency, and delays, useful for basic load testing or verifying idempotency.

Troubleshooting Common Scenarios

| Problem | Solution