Most free proxy lists are 95 % dead, and a good part of the rest are honeypots or proxies that inject scripts into your pages. proxy-scraper collects public HTTP, SOCKS4 and SOCKS5 proxies from 700+ sources and keeps only the ones that pass every check. It learns with every run which sources are worth it, and it can turn the result into a single rotating proxy.
Table of contents
With pipx (recommended – gives you a proxy-scraper command in its own environment):
pipx install git+https://github.com/maximilianfeix/proxy-scraper.git
proxy-scraperOther ways: Docker, pip, a faster event loop, or straight from the repo
# Docker – learned state and results stay in two folders next to you
mkdir -p proxy-data results
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD/proxy-data:/data" -v "$PWD/results:/work/results" \
ghcr.io/maximilianfeix/proxy-scraper --want 50 --https-only
# pip into the current environment
pip install git+https://github.com/maximilianfeix/proxy-scraper.git
# optional: faster event loop on macOS/Linux
pipx install "proxy-scraper[fast] @ git+https://github.com/maximilianfeix/proxy-scraper.git"
# no install at all
git clone https://github.com/maximilianfeix/proxy-scraper.git
cd proxy-scraper
pip install -r requirements.txt
python3 proxy_scraper.pyEvery release also ships a wheel you can install with pip install <file>.whl, and a multi-arch image (amd64/arm64) on ghcr.io. In the container the wizard never shows up, it runs straight away. For the proxy server use --serve --serve-host 0.0.0.0 with -p 127.0.0.1:8899:8899, so the port is only open on your own machine.
Tab completion for bash, zsh and fish:
eval "$(proxy-scraper --completion zsh)" # in ~/.zshrc (after compinit), same for bash in ~/.bashrc
proxy-scraper --completion fish > ~/.config/fish/completions/proxy-scraper.fishInstalled, the learned state lives in your user data folder (~/Library/Application Support/proxy-scraper, %LOCALAPPDATA%\proxy-scraper or ~/.local/share/proxy-scraper; override with PROXY_SCRAPER_HOME) and results go to ./results. Run from a clone, both stay inside the project.
Started without arguments, a wizard asks what you are looking for:
Keys: ↑↓ select · Space toggle · 1–9 jump · Enter next · Esc back · q quit. In scripts and cron jobs the wizard never shows up – pass options or -y.
Don't want to scan yourself? Every hour GitHub Actions runs the tool and publishes the hits to the proxy-list branch – every entry worked in the last run, fastest first.
→ Browse it on the website – search, filter by type, country, HTTPS, provider and latency, see how long each proxy has been up, copy or download exactly the proxies you need. streaks.json on the branch has the number of runs in a row for every proxy.
Every protocol and country also has its own page with a plain download, e.g. SOCKS5 or Germany (country/de/proxies.txt).
curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/socks5.txt | headOr let the tool start from it: proxy-scraper --recheck live downloads the list and checks it again from your network – about 30 seconds instead of a full scan (517 of 1,169 worked from here). With --serve you have a rotating proxy in under a minute.
# 50 proxies that can do HTTPS – then stop
proxy-scraper --want 50 --https-only
# only Germany, Austria and Switzerland, the 20,000 most promising candidates
proxy-scraper --country DE,AT,CH -l 20000
# fast elite SOCKS5 proxies
proxy-scraper --types socks5 --anonymity elite --max-latency 1500
# 20 proxies that really reach Google AND Discord
proxy-scraper --target google.com --target discord.com --want 20
# only recheck the last hits (plus history) – takes seconds
proxy-scraper --recheck
# wizard with defaults – it keeps what you already passed
proxy-scraper -i --country DE
# which sources deliver the most?
proxy-scraper --list-sourcesRunning from a clone? Replace proxy-scraper with python3 proxy_scraper.py.
from proxyscraper import check_proxies, find_proxies
if __name__ == "__main__": # needed on macOS/Windows, the parser uses a process pool
for p in find_proxies(want=20, https=True, countries=["DE", "NL"], no_datacenter=True):
print(p.url, p.latency, p.country, p.org)
alive = check_proxies(["socks5://1.2.3.4:1080", "5.6.7.8:3128"]) # your own listSame run as the command line – sources, learning, every check, result files – just without terminal output. Each result has url, latency, exit_ip, https, anonymity, country, asn, org and hosting. There's an async version of both (find_proxies_async, check_proxies_async).
A list is nice – but usually you just want to enter one proxy that always works:
proxy-scraper --recheck --serve # recheck the last hits, then go – takes secondscurl -x http://127.0.0.1:8899 https://api.ipify.org # a different IP every time
curl -x socks5h://127.0.0.1:8899 https://api.ipify.org # SOCKS5 on the same port
curl -x http://country-de:x@127.0.0.1:8899 https://api.ipify.org # only German exits
curl -x http://session-cart42:x@127.0.0.1:8899 https://shop.example # same proxy for this session
curl http://127.0.0.1:8899/__proxy-scraper/status # pool and counters as JSON
curl http://127.0.0.1:8899/__proxy-scraper/metrics # the same for Prometheus/GrafanaLike commercial rotating proxies, the username carries what you want: country-XX, type-http|socks4|socks5 and session-NAME, combinable (country-us-type-socks5-session-a). It works for HTTP (Proxy-Authorization) and SOCKS5 (username/password auth). By default the password is ignored and the server only listens on 127.0.0.1.
To reach it from other machines, give it a password – every client then has to send it, over HTTP and SOCKS5 alike, and the status page wants it as Basic auth:
export PROXY_SCRAPER_SERVE_PASSWORD=$(openssl rand -hex 16) # the env var keeps it out of `ps`
proxy-scraper --recheck --serve --serve-host 0.0.0.0
curl -x "http://country-de:$PROXY_SCRAPER_SERVE_PASSWORD@your-server:8899" https://api.ipify.orgWithout a password, --serve-host means anyone who reaches the port can use it. The ready-made compose.yaml starts the server in Docker from the live list, with a password, a health check and learned state in a volume: put PROXY_PASSWORD=… into .env, then docker compose up -d.
- every connection goes through a different proxy (unless sticky); fast and proven ones are preferred
- CONNECTfor HTTPS and plain HTTP requests; HTTP, SOCKS4 and SOCKS5 proxies can sit behind it (SOCKS5 with DNS through the proxy)
- HTTPS only uses proxies that passed the test with verified TLS – no broken encryption
- if a proxy stays silent inside the tunnel or returns an error page instead of TLS, the same first packet quietly goes to the next one
- three failures in a row and a proxy leaves the rotation – every 5 minutes those get re-checked and come back if they work again
- --serve-refill 6checks fresh proxies every 6 hours in the background (the live list with- --recheck live, otherwise the last run + history) with the same checks and filters, and adds the hits – a server that runs for days doesn't run dry
- listens on 127.0.0.1only (unless--serve-hostsays otherwise), optionally with a password; live view with requests, success rate, pool and the latest connections
In testing: 20 of 20 HTTPS requests succeeded, over 15 different exit IPs. In the wizard this is Proxy server right away.
The live list can also come to you: bot/ is a Discord bot that posts every run into a server – a summary with the fastest proxies, the full lists per protocol as files, and slash commands like /proxies type:socks5 country:DE https:true. It sets up its own read-only channels when you invite it, and a GitHub Action deploys it to a server as a systemd service. Setup in bot/README.md.
flowchart LR
A[sources.json<br/>meta lists<br/>GitHub discovery] --> B[Fetch & parse<br/>in parallel on all cores]
B --> C[Prioritize<br/>history → good sources → rest]
C --> D[Check<br/>HTTP · SOCKS4 · SOCKS5]
D --> E[Details<br/>HTTPS · anonymity · country]
E --> F[results/]
D -. hit rate per source .-> G[(learned state)]
G -. next run .-> C
- Sources – the curated list in sources.json, meta sources (other projects that maintain lists of proxy sources) and, once a day, a GitHub search for actively maintained repos. Every search adds to what earlier ones found; a list the search hasn't seen for three weeks drops out.
- Collect – plain text, HTML tables, JSON APIs and type://ip:portlines are recognized; private and reserved address ranges are dropped. Lists that haven't changed since the last run answer304and come from a local cache – a second run right after the first loads 0 MB instead of ~160 MB.
- Prioritize – known working proxies first, then by the learned hit rate of their sources.
- Check – every proxy has to fetch its exit IP from a check target (checkip.amazonaws.com, withifconfig.me,ipinfo.io,wtfismyip.comandident.meas reserves – none of them behind Cloudflare) and return a valid, foreign IP. If the target goes down mid-run, the tool switches and re-checks the proxies that were affected, so the statistics don't learn from an outage. Anyone passing your own IP through is out. Then comes the confirmation viahttpbin.org: fake proxies that only answer the first check with “200 + IP” fail here. Finally a static HTML page has to arrive byte for byte as it does without a proxy – anyone injecting ads or scripts is out.
- Countries and providers – looked up offline in the free DB-IP databases, including the provider (ASN) and whether it's probably a datacenter (about 45 % of working proxies are) (downloaded once a month, ~2 µs per lookup); ip-api.com is only asked for the few addresses it doesn't know.
- Blocklists – one DNS lookup per exit IP against SpamCop, cached for the run: about 29 % of working proxies exit from a listed IP, and sites that use the list show those captchas or block them. --no-blocklisteddrops them. If your DNS resolver is refused by SpamCop (large public resolvers are), the lookup is skipped instead of guessing.
- Learn – hit rates and history are stored. Sources without hits, with content unchanged for a week or permanently unreachable are skipped.
Every run gets its own folder; results/latest.txt always names the newest one (on macOS/Linux there is also the symlink results/latest):
results/2026-09-24_18-42-07/
├── all.txt socks5://203.0.113.10:1080 (fastest first)
├── http.txt 203.0.113.20:8080 (plain ip:port lists per type)
├── socks4.txt
├── socks5.txt
├── proxies.json latency, country, HTTPS, anonymity, exit IP
└── proxies.csv
Use the fastest proxy from the last run – free proxies die quickly, so --recheck first if the run is older than a few minutes
proxy-scraper --recheck -y
curl -x "$(head -1 results/latest/all.txt)" http://api.ipify.orgFor HTTPS, pick a proxy with "https": true from proxies.json – like the Python example below does.
Python requests (pip install "requests[socks]" for SOCKS)
import json
from pathlib import Path
import requests
run = Path("results") / Path("results/latest.txt").read_text().strip() # works on every OS
proxies = json.loads((run / "proxies.json").read_text()) # fastest first
for p in proxies:
if not p["https"]:
continue
try:
r = requests.get("https://api.ipify.org", proxies={"http": p["url"], "https": p["url"]}, timeout=8)
print(p["url"], "→", r.text)
break
except requests.RequestException:
continue # free proxies come and go – just take the next oneproxychains, Clash / Mihomo – ready-made configs with --export
proxy-scraper --want 30 -y --export proxychains,clash
proxychains4 -f results/latest/proxychains.conf curl https://api.ipify.orgclash.yaml has all HTTP and SOCKS5 proxies plus a url-test group that always picks the fastest. Both configs leave out HTTP proxies that can't tunnel (CONNECT), because these tools tunnel everything.
In a pipe – -o - prints the hits to stdout, the interface moves to stderr
proxy-scraper --recheck live --want 20 -y -o - | grep '^socks5://' > socks.txtAny tool, through the rotating server
proxy-scraper --recheck --serve &
export HTTPS_PROXY=http://127.0.0.1:8899 HTTP_PROXY=http://127.0.0.1:8899
pip download requests # git, pip, npm & co. now go through the poolWithout installing anything – straight from the live list
curl -s https://raw.githubusercontent.com/maximilianfeix/proxy-scraper/proxy-list/https.txt | head -5The shell snippets are for macOS and Linux, where results/latest points to the newest run. On Windows, results/latest.txt holds the folder name instead – in PowerShell:
$run = "results\$(Get-Content results\latest.txt)"
curl.exe -x (Get-Content "$run\all.txt" -TotalCount 1) http://api.ipify.orgShow all options
Everything else: proxy-scraper --help
Tip
For the GitHub search a logged-in gh or the GITHUB_TOKEN environment variable is enough. Without a token the API limit is 60 requests per hour, and only 40 repos are searched.
The repo does part of the work itself:
Almost nothing gets through.
Many company, school and university networks block proxy connections. The tool notices a hit rate below 0.2 % and warns you – a different network such as a phone hotspot helps. The learned statistics are not downgraded in such a run.
Does it work on Windows?
Yes, in PowerShell and Windows Terminal. uvloop doesn't exist there and is skipped automatically. In the old cmd.exe window some symbols may be missing depending on the font.
Why does it find fewer proxies than other lists claim to have?
Because only proxies that pass every check are kept. Many lists count anything that accepts a TCP connection; here a proxy must fetch two independent pages and show a foreign IP. That's usually a few hundred out of a million candidates – but they work.
What about proxies with a username and password?
Lines like socks5://user:pass@1.2.3.4:1080 keep their login: HTTP proxies get a Proxy-Authorization header, SOCKS5 uses username/password auth (RFC 1929), SOCKS4 the user ID. The same works for --recheck with your own list. The result files keep the credentials, the terminal only shows user:•••.
How fresh is the live list?
It's rebuilt every hour; the “updated” badge shows the last run. Free proxies come and go quickly, so for anything important run proxy-scraper --recheck right before use.
Is it safe to use free proxies?
Only for things that don't matter. Public proxies are run by strangers who can read everything that isn't encrypted. Never send passwords or personal data through them, and only use them for legal purposes.
What's next is tracked in the milestone v1.7 – ideas and wishes are welcome as an issue. During Hacktoberfest there are beginner-friendly issues with pointers on where to start.
- Publish on PyPI so it's just pipx install proxy-scraper
- Protocol detection on the same connection
- sing-box export, PowerShell completion, shareable filters on the website
Shipped in v1.6: spam blocklist check for every exit IP, a live list refreshed every hour, pages per protocol and country, an optional password for the proxy server and a pool that refills itself while it runs, compose.yaml, -o - for pipes, a Discord bot, and a new website and README – everything in English now.
Shipped in v1.5: content tampering check, live list website with trend and stable proxies, provider/datacenter info, a much bigger proxy server (rotation strategies, sticky sessions, SOCKS5 inbound, status and Prometheus metrics), --recheck live, Python API, shell completion. Measured and dropped earlier: protocol detection with an extra connection (#3) and IPv6 (#1).
Bug reports, new sources and pull requests are very welcome – see CONTRIBUTING.md, and docs/ARCHITECTURE.md for how the pieces fit together. The short version:
pip install -e ".[dev]"
python3 -m pytest # runs offline – fake proxies on localhost
ruff check .
python3 docs/make_demo.py # regenerate the images in this READMEProject structure
proxy_scraper.py entry point when run from a clone
bot/ Discord bot for the live list (own requirements, deployed by GitHub Actions)
proxyscraper/
├── cli.py arguments, wizard or direct start
├── app.py one run in phases: network → jobs → check → learn & report
├── options.py RunOptions + Filters – all settings in one place
├── pipeline.py collect sources, prioritize, check loop
├── checker.py checks, honeypot confirmation, HTTPS test
├── handshake.py HTTP/SOCKS4/SOCKS5 handshakes incl. login
├── judges.py check targets, Cloudflare filter, failover
├── sources.py source lists, meta sources, GitHub discovery, statistics
├── sources.json curated sources
├── fetchcache.py ETag cache for unchanged lists
├── parsing.py find proxies in text, HTML and JSON
├── history.py history of working proxies
├── geo.py countries: offline first, ip-api.com as fallback
├── asndb.py DB-IP provider database, datacenter heuristic
├── geodb.py DB-IP country database (monthly, binary search)
├── targets.py target sites for --target
├── output.py result files
├── exporters.py proxychains, Clash and curl formats (--export)
├── server/ rotating proxy server (--serve): pool · http · upstream · socks · status · core
├── api.py find_proxies() / check_proxies() for Python
├── publish.py live list for GitHub Actions
├── paths.py where state and results are stored
├── compat.py differences between Unix and Windows
├── netio.py small HTTP client on asyncio
└── ui/ widgets · dashboard · report · wizard · serve · keys
proxy-scraper stands on the work of the people who publish free proxy lists. Thanks to everyone listed in sources.json, and especially to
- monosans/proxy-scraper-checker and gfpcom/free-proxy-list, whose curated source collections are read as meta sources
- Textualize/rich, which draws the whole terminal UI
- IP Geolocation by DB-IP – the free country database (CC BY 4.0) used for offline country lookups
- httpbin, checkip.amazonaws.com, ifconfig.me, ipinfo.io, wtfismyip.com, ident.me and ip-api.com, used as check targets and for country lookups
This tool only collects publicly listed proxies and checks whether they work. You are responsible for how you use them – respect the terms of the sites you visit and the laws where you live.