Proxies in Playwright and Puppeteer: setup, authentication and per-context IPs

The two levels a proxy can be set at

A proxy can be given at browser launch, applying to everything, or when creating a context. A context in these libraries is an isolated session with its own cookies and storage, and each one can have its own address. For data collection that beats a single shared proxy: profiles overlap neither in state nor in address.

Playwright: credentials right in the option

Here it is straightforward: username and password go in the same setting as the address, and this works both at launch and per context.

import { chromium } from 'playwright'

// Playwright takes the username and password in the same option as the address,
// both at launch and per browser context.
const browser = await chromium.launch({
  proxy: { server: 'http://GATEWAY:PORT', username: 'LOGIN', password: 'PASSWORD' },
})

// A separate address per context: one browser then works
// through several IPs at the same time.
const ctx = await browser.newContext({
  proxy: { server: 'http://GATEWAY:PORT', username: 'LOGIN', password: 'PASSWORD' },
})
const page = await ctx.newPage()
await page.goto('https://example.com')

Puppeteer: the password goes in a separate call

Chromium ignores credentials passed in the launch argument and shows a native authentication dialog instead — the driver will not fill it in, and the page simply never loads. So credentials go in a separate call, and it must happen before navigating.

import puppeteer from 'puppeteer'

// Chromium ignores credentials passed in --proxy-server: it shows a native auth
// dialog that the driver will not fill in. Credentials go in a separate call.
const browser = await puppeteer.launch({
  args: ['--proxy-server=http://GATEWAY:PORT'],
})
const page = await browser.newPage()
await page.authenticate({ username: 'LOGIN', password: 'PASSWORD' })  // before goto
await page.goto('https://example.com')

// A separate context with its own proxy; credentials still via page.authenticate
const ctx = await browser.createBrowserContext({
  proxyServer: 'http://GATEWAY:PORT',
  proxyBypassList: ['127.0.0.1'],
})

Why IP authentication is easier here

Both frameworks need extra steps to pass a password, and with SOCKS5 it may not work at all. Whitelist authentication removes the question entirely: the machine's address is added once and the browser connects with no credentials, regardless of framework or protocol.

Both schemes are active at once here, so this is not a trade-off: browser jobs on a server go by whitelist while scripts from other machines keep using the login.

How many contexts to open

It is easy to overestimate here. One context with an open page holds not one connection but several: the document itself, images, scripts, API calls. So context count does not equal your plan's thread count — it is considerably lower.

• Reckon on 5–10 concurrent connections per active context loading a full page.
• A 50-thread plan comfortably holds 5–8 contexts, not fifty.
• Blocking unneeded resources — images, fonts, analytics — cuts connection use several times over.

That last point is worth doing first: for collecting prices or text you do not need images or fonts, and they account for most of the connections and traffic.

When you do not need a browser

Before launching Chromium, check whether you need it: a browser consumes several times more connections and memory than a plain request. For pages where the data arrives in a separate JSON response, a browser is not needed at all.

// Proxy from environment variables, no libraries needed.
// The flag and the variable arrived in recent Node versions and are marked as
// actively developing — check that your version supports them.
//   node --use-env-proxy script.js
//   NODE_USE_ENV_PROXY=1 node script.js
// Applies to fetch(), http.request() and https.request().

// An explicit address in code requires installing undici:
import { fetch, ProxyAgent } from 'undici'
const agent = new ProxyAgent({ uri: 'http://GATEWAY:PORT' })
await fetch('https://example.com', { dispatcher: agent })

// axios: the proxy option works in Node ONLY, and a custom httpAgent or
// httpsAgent overrides it — set an agent and the option is ignored.
// proxy: false disables the proxy and also ignores environment variables.

Two traps in that snippet deserve attention. The proxy option in axios works server-side only — in a browser build it silently does nothing. And it stops applying if you set your own agent: the agent overrides the option, without warning.

What to disable so the browser stops eating your limit

This is the single most effective change available: for collecting prices or text you do not need images, fonts or analytics, and those account for most of the connections and traffic. Blocking unneeded resource types cuts usage several times over, not by a few percent.

• Images and fonts — almost always surplus for data collection.
• Analytics and advertising scripts — they open connections to third-party hosts absent from your arithmetic.
• Media and font downloads — the heaviest class by traffic.