Preloader
Others
  • Estimated reading time: 6 Minutes

curl Proxy From the Terminal: Flags, Environment Variables and Route Checks

curl Proxy From the Terminal: Flags, Environment Variables and Route Checks

What the -x flag hands to curl

Passing -x, or its long form --proxy, tells curl to connect to the proxy instead of the website. Its value is a URL shaped like scheme://user:password@host:port. According to the curl manual, a string with no scheme, or with http://, is treated as an HTTP proxy, and a string with no port is assumed to use port 1080.

That default is the first trap. Port 1080 is the traditional SOCKS port, so an HTTP proxy URL copied without its port usually fails at the connection step and looks like an outage. Write both the scheme and the port every time, and default to --proxy in shared scripts, keeping -x for one-off commands.

curl --proxy "http://PROXY_HOST:PORT" https://example.com/

HTTP, HTTPS and SOCKS schemes, and where each one stops

With an http:// proxy and an https:// target, curl first sends a CONNECT request, the proxy opens a TCP tunnel, and TLS with the website starts inside it. Hostname and port are visible to the proxy; the page is not.

CONNECT is also where plain HTTP proxies stop. Many only allow CONNECT to port 443, so an API on an unusual port can fail through a proxy that handles ordinary HTTPS without complaint. When only one endpoint breaks, suspect the port policy before the credentials.

An https:// proxy encrypts the hop between curl and the proxy itself, which plain HTTP and SOCKS connections never do. Choose it whenever proxy credentials cross a network you do not control.

SOCKS schemes relay TCP without reading HTTP. What matters is DNS: the h in socks5h means the proxy resolves the hostname, while socks5:// makes curl resolve it locally. Pick socks5h unless you deliberately want local resolution.

Environment variables versus the -x flag

Shell-wide proxies come from an environment variable named after the scheme: http_proxy for http:// URLs, https_proxy for https:// URLs, and ALL_PROXY as the fallback. The everything curl book explains one quirk. Only lowercase http_proxy is accepted, because a CGI script receives an incoming Proxy header as HTTP_PROXY, and curl must not let a stranger's request pick its proxy.

Setting http_proxy and then fetching an https:// URL sends the request direct, since that variable covers plain HTTP destinations only. Export https_proxy too whenever a job mixes schemes.

export http_proxy="http://PROXY_HOST:PORT"

export https_proxy="http://PROXY_HOST:PORT"

export NO_PROXY="localhost,.internal.example"

Anything given with -x beats those variables for one command and leaves the shell untouched. Prefer the flag in shared automation unless the whole runner is meant to use one egress point.

NO_PROXY overrides even an explicit -x

Here the manual contradicts a common assumption: NO_PROXY disables the proxy even when one is given with --proxy. Entries are comma-separated, each matches a domain and its subdomains, and a lone asterisk matches every host.

IP ranges can be written in CIDR notation, a feature the manual dates to curl 7.86.0. Older build agents ignore it, so run curl -V on each runner image before trusting a range.

Two flags handle exceptions. --noproxy "*" sends one command direct regardless of every variable, while --noproxy "" cancels NO_PROXY for that run. Reach for the empty form when a global list blocks a host you want to test through the proxy.

Plugging a residential gateway into curl

Commercial gateways use the same syntax. EZXLabs documents its residential HTTP connection as http://user:[email protected]:39080, with credentials issued to the account; clients with separate fields take the HTTP protocol, the proxy.ezxlabs.com host and port 39080.

curl --proxy "http://proxy.ezxlabs.com:39080" --proxy-user "USER:PASS" https://example.com/

Credentials inside a proxy URL are URL-decoded, which the manual says lets you write @ as %40 and a colon as %3a. Better still, keep them in --proxy-user or a -K config file so the password never lands in shell history.

Session mode is the other decision. In rotating mode the backconnect gateway assigns a new IP to each request, which suits independent fetches. A sticky session holds one exit for a fixed 10-minute TTL that users cannot change, so a multi-step flow must finish inside it. Default to rotating unless later requests depend on cookies set earlier. Sticky sessions keep the exit consistent, but cookies and login state stay your job: store them with -c and -b.

Proving the request really went through the proxy

Verbose output is the quick check, since the Trying line of curl -v names the address curl contacted. For logs, use --write-out. In the manual, http_connect is the code from the proxy's reply to CONNECT, and proxy_used returns 1 when the previous transfer used a proxy and 0 otherwise. That variable arrived in curl 8.7.0, described as a way to tell whether a NO_PROXY pattern matched.

curl -sS -o /dev/null --proxy "$PROXY" \
-w "proxy_used=%{proxy_used} connect=%{http_connect}\n" https://example.com/

Log that line on every scheduled run. A job that has gone direct for weeks looks healthy on every other metric, and a single 0 settles it. For a longer reference, including a measured table of which setting won when several overlapped, the EZXLabs walkthrough on using a curl proxy covers each case on a current Windows build.

Why a clean exit code can hide a failed proxy login

Script authors rarely expect this. When a proxy rejects credentials for a plain http:// URL, it answers 407 as an ordinary response, and in a test published on the EZXLabs blog curl exited with 0; adding -f turned that into exit code 22. On an https:// URL the rejection arrives during CONNECT and produces a hard error. Add -f to every scripted proxy call unless the job inspects error bodies itself.

Exit codes map to layers, and the Proxy Exit-Code Map lists the ones the manual defines on this path. Fix them in order: name, connection, login.

Exit code Manual meaning First thing to check
5 Could not resolve proxy Spelling of the proxy hostname
7 Failed to connect Scheme and port in the proxy URL
22 HTTP error of 400 or more, with -f Proxy credentials when the code is 407
97 Proxy handshake error SOCKS login or a scheme mismatch

SOCKS logins fail at that last layer. In the same blog test, a SOCKS5 proxy that needs a login rejected missing and wrong credentials with exit code 97.

Gateways add statuses of their own. At proxy.ezxlabs.com, exhausted plan traffic returns HTTP 429 with EZP_QUOTA_DENIED, and a timed-out upstream or target tunnel returns 504 with EZP_CONNECT_TIMEOUT. The documentation asks users to check the plan instead of retrying a 429, so keep that status out of retry loops written for rate limits.

Limits: what a proxy flag does not change

Routing a request is not permission to make it. A proxy alters the network path; a site's terms of service, its robots.txt rules, the request rate it will serve and the personal-data laws covering what you collect stay exactly where they were. Use the official API where one exists, and keep automated traffic within published limits.

Know what sits on the other end, too. EZXLabs describes its residential proxy service as network access only: it is not a crawler, scraper, extraction API, managed scraping service or automation platform, so parsing and storage remain in your code. Mobile, datacenter and ISP proxy services are listed as in development, and its SLA guarantees no minimum uptime, success rate, response time or IP pool size.

Pre-flight list for a scripted curl proxy

  • Scheme and port written explicitly in --proxy.
  • https_proxy or ALL_PROXY exported when targets use https://.
  • NO_PROXY reviewed on the runner, with --noproxy "" where a listed host must be proxied.
  • socks5h chosen when DNS should resolve at the proxy.
  • Option -f present, and exit codes 5, 7 and 97 mapped to their layers.
  • proxy_used and http_connect written to the job log.
  • Credentials held in --proxy-user or a -K file.
  • Rotating or sticky session picked per workflow, with a cookie jar for sticky flows.
  • Terms of service, robots.txt and rate limits of the target checked before the first run.

Run through the list once per new runner image rather than once per project. Surprises come from the inherited environment, not the typed command.

Our Sponsors

Our blog is proudly supported by industry-leading sponsors.