Playwright Proxy Setup

Playwright enables proxy setup for Python and Node.js scripts primarily through browserType.launch() or browser.newContext() options, allowing specification of proxy server address, port, and authentication credentials. This configuration directs all subsequent network requests from the launched browser instance or context through the specified proxy.

Playwright Proxy Configuration Overview

Playwright supports proxy configurations at two main levels:

1. Browser Launch Level: Applying a proxy to the entire browser instance. All contexts and pages created within this browser will use the specified proxy. This is suitable when all automation tasks require the same proxy.
2. Browser Context Level: Applying a proxy to a specific browser context. This allows different contexts within the same browser instance to use different proxies, or some contexts to use no proxy, which is beneficial for managing diverse scraping or testing scenarios.

Playwright's proxy configuration object or dictionary typically includes the server address (e.g., http://proxy.example.com:8080, socks5://localhost:9050), and optionally username and password for authenticated proxies.

Proxy Types Supported

Playwright supports standard proxy protocols:

• HTTP Proxies: Specified with http:// or https:// schemes.
• SOCKS Proxies: Specified with socks5:// or socks4:// schemes.

When the scheme is omitted (e.g., proxy.example.com:8080), Playwright defaults to an HTTP proxy. Explicitly specifying the scheme is recommended for clarity and to ensure the correct protocol is used.

Python Playwright Proxy Setup

In Python, proxy configuration is passed as a dictionary to the proxy argument of browser_type.launch() or browser.new_context().

Basic HTTP/HTTPS Proxy Setup

To use an unauthenticated HTTP or HTTPS proxy:

from playwright.sync_api import sync_playwright

# For browser-wide proxy
with sync_playwright() as p:
    browser = p.chromium.launch(proxy={"server": "http://your.proxy.server:8080"})
    page = browser.new_page()
    page.goto("https://httpbin.org/ip")
    print(page.content())
    browser.close()

# For context-specific proxy
with sync_playwright() as p:
    browser = p.chromium.launch()
    context = browser.new_context(proxy={"server": "http://another.proxy.server:8080"})
    page = context.new_page()
    page.goto("https://httpbin.org/ip")
    print(page.content())
    context.close()
    browser.close()

Authenticated HTTP/HTTPS Proxy Setup

For proxies requiring authentication, include username and password keys in the proxy dictionary:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(
        proxy={
            "server": "http://user.proxy.server:8080",
            "username": "proxy_user",
            "password": "proxy_password"
        }
    )
    page = browser.new_page()
    page.goto("https://httpbin.org/ip")
    print(page.content())
    browser.close()

SOCKS Proxy Setup

Specify the SOCKS protocol in the server string:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(
        proxy={"server": "socks5://your.socks.proxy:9050"}
    )
    page = browser.new_page()
    page.goto("https://httpbin.org/ip")
    print(page.content())
    browser.close()

SOCKS proxies also support authentication by adding username and password to the dictionary.

Bypassing Proxy for Specific Domains

The bypass key in the proxy dictionary allows specifying a comma-separated list of hosts or domains that should not use the proxy. Wildcards (*) are supported.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(
        proxy={
            "server": "http://your.proxy.server:8080",
            "bypass": "*.local, .example.com, 192.168.1.1"
        }
    )
    page = browser.new_page()
    page.goto("http://example.com") # Will bypass proxy if example.com is in bypass list
    page.goto("https://google.com") # Will use proxy
    browser.close()

Node.js Playwright Proxy Setup

In Node.js, proxy configuration is passed as an object to the proxy property of browserType.launch() or browser.newContext().

Basic HTTP/HTTPS Proxy Setup

To use an unauthenticated HTTP or HTTPS proxy:

const { chromium } = require('playwright');

(async () => {
  // For browser-wide proxy
  const browser = await chromium.launch({
    proxy: {
      server: 'http://your.proxy.server:8080'
    }
  });
  const page = await browser.newPage();
  await page.goto('https://httpbin.org/ip');
  console.log(await page.content());
  await browser.close();

  // For context-specific proxy
  const browser2 = await chromium.launch();
  const context = await browser2.newContext({
    proxy: {
      server: 'http://another.proxy.server:8080'
    }
  });
  const page2 = await context.newPage();
  await page2.goto('https://httpbin.org/ip');
  console.log(await page2.content());
  await context.close();
  await browser2.close();
})();

Authenticated HTTP/HTTPS Proxy Setup

For proxies requiring authentication, include username and password properties in the proxy object:

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch({
    proxy: {
      server: 'http://user.proxy.server:8080',
      username: 'proxy_user',
      password: 'proxy_password'
    }
  });
  const page = await browser.newPage();
  await page.goto('https://httpbin.org/ip');
  console.log(await page.content());
  await browser.close();
})();

SOCKS Proxy Setup

Specify the SOCKS protocol in the server string:

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch({
    proxy: {
      server: 'socks5://your.socks.proxy:9050'
    }
  });
  const page = await browser.newPage();
  await page.goto('https://httpbin.org/ip');
  console.log(await page.content());
  await browser.close();
})();

SOCKS proxies also support authentication by adding username and password to the object.

Bypassing Proxy for Specific Domains

The bypass property in the proxy object allows specifying a comma-separated list of hosts or domains that should not use the proxy. Wildcards (*) are supported.

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch({
    proxy: {
      server: 'http://your.proxy.server:8080',
      bypass: '*.local, .example.com, 192.168.1.1'
    }
  });
  const page = await browser.newPage();
  await page.goto('http://example.com'); // Will bypass proxy if example.com is in bypass list
  await page.goto('https://google.com'); // Will use proxy
  await browser.close();
})();

Playwright Proxy Configuration Comparison

Feature/Language	Python (dict)	Node.js (object)	Description
Server	{"server": "http://host:port"}	{ server: "http://host:port" }	Required. Proxy address and port, with optional scheme.
Username	{"username": "user"}	{ username: "user" }	Optional. For authenticated proxies.
Password	{"password": "pass"}	{ password: "pass" }	Optional. For authenticated proxies.
Bypass	{"bypass": "*.local,domain.com"}	{ bypass: "*.local,domain.com" }	Optional. Comma-separated list of hosts/domains to bypass the proxy.
Launch Level	browser_type.launch(proxy={...})	browserType.launch({ proxy: {...} })	Applies proxy to entire browser instance.
Context Level	browser.new_context(proxy={...})	browser.newContext({ proxy: {...} })	Applies proxy to a specific browser context.

Advanced Proxy Considerations

Proxying Specific Requests with page.route()

For highly granular control, page.route() can intercept network requests and forward them to different proxies based on specific criteria. This method provides more flexibility than global or context-level proxy settings but requires manual request handling.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()

    def handle_route(route):
        if "example.com" in route.request.url:
            route.fulfill(status=200, body="Blocked by route handler.")
        else:
            # Forward other requests through a specific proxy
            # This requires manual HTTP client setup or could be complex for direct proxying
            # For simple use cases, global/context proxy is preferred.
            # For complex proxy routing, consider using an external proxy manager.
            route.continue_({"url": route.request.url}) # Example: continue without modification

    page.route("**/*", handle_route)
    page.goto("https://www.google.com") # Will be routed
    page.goto("https://www.example.com") # Will be fulfilled
    browser.close()

page.route() is generally used for mocking, modifying requests, or blocking, rather than dynamically re-routing to different proxies unless integrated with a custom proxy handling logic.

Verifying Proxy Functionality

After configuring a proxy, verify its operation by navigating to an IP address checking service, such as https://httpbin.org/ip or https://whatismyipaddress.com/. The displayed IP address should correspond to the proxy server, not the local machine.

Handling Proxy Errors and Timeouts

Proxies can introduce additional points of failure. Implement robust error handling for network requests when using proxies. This includes:

• Retries: Attempting the request again if a proxy connection fails or times out.
• Timeouts: Configuring appropriate network timeouts for Playwright operations.
• Proxy Rotation: For large-scale operations, rotate through a pool of proxies to mitigate rate limiting, IP bans, or single-proxy failures. This typically involves managing a list of proxies and updating the proxy configuration dynamically for new contexts or retries.

Security Considerations

When using authenticated proxies, avoid hardcoding credentials directly in source code. Instead, use environment variables or a secure configuration management system to retrieve username and password. This prevents credential exposure and simplifies credential management across different environments.