Python3，ProxyIPOrScrapingSpeedOfMethod，Used forIPAddress。ProxyCanProxyAndProxy，ProxyStable，ProxyStableReliable。
Python3 ProxyOfAndApplication：
IP Lock：Settings IP Frequency，， IP ，ProxyAble to。
ScrapingSpeed：ProxyCanConnection，FastScrapingGoalDataOfOf。
：OfOfService，NeedOf，ProxyCan，CanOfData。
，ProxyIP Python3 ImportantOfRole。ProxySecurityIssue，Proxy，NeedNoteSelectOfProxyService，NetworkSecurity。
Work
NeedGetAvailableProxy，Proxy IP AddressAndPortOfGroup， ip:Port Of。ProxyNeedCertification，NeedOfUserPasswordInformation。
OfInstallProxySoftware，Local 7890 PortCreate HTTP ProxyService，Proxy 127.0.0.1:7890。，Software 7891 PortCreate SOCKS ProxyService，Proxy 127.0.0.1:7891，SettingsProxy，CanSuccess IP ProxySoftwareConnectionOfServiceOf IP 。
OfExample，ProxySettingsMethod，CanReplaceOfAvailableProxy。
SettingsProxy，TestOfURL http://httpbin.org/get LinkCanOfInformation，ResultOf origin CustomerOf IP，CanProxySettingsSuccess，Success IP。
，OfProxySettingsMethod。
GetPython3Proxy
DataOf，。ProxyServiceCan，OfMay，ScrapingSuccess。
StaticProxyIP
IPRoyalOfProxyServiceResidentialProxyResolveSolution
IPRoyal
OfStaticProxy
Proxy-seller Data CenterProxy，InternetMarketingOf。
Proxy-seller
OfStaticProxy
Shifter.ioOfProxyService，UserPrivacyProtectionAndOfInternet。
Shifter.io
2. urllib
Of urllib ，ProxyOfSettingsMethod，Code：
from urllib.error import URLError
from urllib.request import ProxyHandler, build_opener
proxy = '127.0.0.1:7890'
proxy_handler = ProxyHandler({
'http': 'http://' + proxy,
'https': 'http://' + proxy
})
opener = build_opener(proxy_handler)
try:
response = opener.open('https://httpbin.org/get')
print(response.read().decode('utf-8'))
except URLError as e:
print(e.reason)
Result：
{
"args": {},
"headers": {
"Accept-Encoding": "identity",
"Host": "httpbin.org",
"User-Agent": "Python-urllib/3.7",
"X-Amzn-Trace-Id": "Root=1-60e9a1b6-0a20b8a678844a0b2ab4e889"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
Need ProxyHandler SettingsProxy，ParameterType，ProtocolType，Proxy。Note，ProxyNeedProtocol， http:// Or https://，OfLink HTTP ProtocolOf， http OfProxy，OfLink HTTPS ProtocolOf， https OfProxy。ProxySettings HTTP Protocol，Settings
http://， HTTP HTTPS ProtocolOfLink，ConfigurationOf HTTP ProtocolOfProxy。
Create ProxyHandler ，Need build_opener MethodCreate Opener， Opener SettingsProxy。 Opener Of open Method，OfLink。
Result JSON， origin，CustomerOf IP。Validation，Of IP ProxyOf IP，Of IP。SuccessSettingsProxy，CanHide IP 。
NeedCertificationOfProxy，CanOfMethodSettings：
from urllib.error import URLError
from urllib.request import ProxyHandler, build_opener
proxy = 'username:password@127.0.0.1:7890'
proxy_handler = ProxyHandler({
'http': 'http://' + proxy,
'https': 'http://' + proxy
})
opener = build_opener(proxy_handler)
try:
response = opener.open('https://httpbin.org/get')
print(response.read().decode('utf-8'))
except URLError as e:
print(e.reason)
Of proxy ，NeedProxyProxyCertificationOfUserPassword， username User，password Password， username foo，Password bar，Proxy foo:bar@127.0.0.1:7890。
Proxy SOCKS5 Type，CanSettingsProxy：
import socks
import socket
from urllib import request
from urllib.error import URLError
socks.set_default_proxy(socks.SOCKS5, '127.0.0.1', 7891)
socket.socket = socks.socksocket
try:
response = request.urlopen('https://httpbin.org/get')
print(response.read().decode('utf-8'))
except URLError as e:
print(e.reason)
Need socks ，CanPassInstall：
pip3 install PySocks
NeedLocal SOCKS5 Proxy， 7891 Port，SuccessAnd HTTP ProxyResultOf：
{
"args": {},
"headers": {
"Accept-Encoding": "identity",
"Host": "httpbin.org",
"User-Agent": "Python-urllib/3.7",
"X-Amzn-Trace-Id": "Root=1-60e9a1b6-0a20b8a678844a0b2ab4e889"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
ResultOf origin ProxyOf IP，ProxySettingsSuccess。
3.requests OfProxySettings
requests ，ProxySettingsSimple，Need proxies Parameter。
OfProxy， requests Of HTTP ProxySettings，Code：
import requests
proxy = '127.0.0.1:7890'
proxies = {
'http': 'http://' + proxy,
'https': 'http://' + proxy,
}
try:
response = requests.get('https://httpbin.org/get', proxies=proxies)
print(response.text)
except requests.exceptions.ConnectionError as e:
print('Error', e.args)
Result：
{
"args": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Host": "httpbin.org",
"User-Agent": "python-requests/2.22.0",
"X-Amzn-Trace-Id": "Root=1-5e8f358d-87913f68a192fb9f87aa0323"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
And urllib ，OfLink HTTP ProtocolOf， http OfProxy，OfLink HTTPS ProtocolOf， https OfProxy， HTTP ProtocolOfProxy。
ResultOf origin ProxyServiceOf IP，ProofProxySettingsSuccess。
ProxyNeedCertification，ProxyOfUserAndPassword，ProxyOfWrite：
proxy = 'username:password@127.0.0.1:7890'
Need username And password Replace。
Need SOCKS Proxy，CanSettings：
import requests
proxy = '127.0.0.1:7891'
proxies = {
'http': 'socks5://' + proxy,
'https': 'socks5://' + proxy
}
try:
response = requests.get('https://httpbin.org/get', proxies=proxies)
print(response.text)
except requests.exceptions.ConnectionError as e:
print('Error', e.args)
NeedInstall requests[socks]，：
pip3 install "requests[socks]"
ResultOf：
{
"args": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Host": "httpbin.org",
"User-Agent": "python-requests/2.22.0",
"X-Amzn-Trace-Id": "Root=1-5e8f364a-589d3cf2500fafd47b5560f2"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
，Settings， socks ，NeedInstall socks 。SettingsMethod：
import requests
import socks
import socket
socks.set_default_proxy(socks.SOCKS5, '127.0.0.1', 7891)
socket.socket = socks.socksocket
try:
response = requests.get('https://httpbin.org/get')
print(response.text)
except requests.exceptions.ConnectionError as e:
print('Error', e.args)
MethodCanSettings SOCKS Proxy，Result。Method，MethodSettingsOf。CanSympathyOfMethod。
4. httpx OfProxySettings
httpx OfAnd requests Of，Pass proxies ParameterSettingsProxyOf，And requests Of，proxies ParameterOf http Or https，Need http:// Or https://，OtherOfSettingsOf。
HTTP Proxy，SettingsMethod：
import httpx
proxy = '127.0.0.1:7890'
proxies = {
'http://': 'http://' + proxy,
'https://': 'http://' + proxy,
}
with httpx.Client(proxies=proxies) as client:
response = client.get('https://httpbin.org/get')
print(response.text)
NeedCertificationOfProxy， proxy Of：
proxy = 'username:password@127.0.0.1:7890'
Need username And password Replace。
ResultAnd requests Of，Result：
{
"args": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Host": "httpbin.org",
"User-Agent": "python-httpx/0.18.1",
"X-Amzn-Trace-Id": "Root=1-60e9a3ef-5527ff6320484f8e46d39834"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
SOCKS Proxy，NeedInstall httpx-socks ，InstallMethod：
pip3 install "httpx-socks[asyncio]"
InstallSyncAndAsyncPatternOfSupport。
SyncPattern，SettingsMethod：
import httpx
from httpx_socks import SyncProxyTransport
transport = SyncProxyTransport.from_url(
'socks5://127.0.0.1:7891')
with httpx.Client(transport=transport) as client:
response = client.get('https://httpbin.org/get')
print(response.text)
NeedSettings transport ，Configuration SOCKS ProxyOfAddress， httpx Of Client Of transport Parameter，ResultAndOf。
AsyncPattern，SettingsMethod：
import httpx
import asyncio
from httpx_socks import AsyncProxyTransport
transport = AsyncProxyTransport.from_url(
'socks5://127.0.0.1:7891')
async def main():
async with httpx.AsyncClient(transport=transport) as client:
response = await client.get('https://httpbin.org/get')
print(response.text)
if __name__ == '__main__':
asyncio.get_event_loop().run_until_complete(main())
AndSyncPatternOf，transport Of AsyncProxyTransport SyncProxyTransport，Need Client AsyncClient ，OtherOf，ResultOf。
5. Selenium OfProxySettings
Selenium CanSettingsProxy， Chrome SettingsMethod。
CertificationOfProxy，SettingsMethod：
from selenium import webdriver
proxy = '127.0.0.1:7890'
options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=http://' + proxy)
browser = webdriver.Chrome(options=options)
browser.get('https://httpbin.org/get')
print(browser.page_source)
browser.close()
Result：
{
"args": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "zh-CN,zh;q=0.9",
"Host": "httpbin.org",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-5e8f39cd-60930018205fd154a9af39cc"
},
"origin": "210.173.1.204",
"url": "http://httpbin.org/get"
}
ProxySettingsSuccess，origin Proxy IP OfAddress。
ProxyCertificationProxy，SettingsMethod，：
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import zipfile
ip = '127.0.0.1'
port = 7890
username = 'foo'
password = 'bar'
manifest_json = """{"version":"1.0.0","manifest_version": 2,"name":"Chrome Proxy","permissions": ["proxy","tabs","unlimitedStorage","storage","<all_urls>","webRequest","webRequestBlocking"],"background": {"scripts": ["background.js"]
}
}
"""
background_js = """
var config = {
mode: "fixed_servers",
rules: {
singleProxy: {
scheme: "http",
host: "%(ip) s",
port: %(port) s
}
}
}
chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});
function callbackFn(details) {
return {
authCredentials: {username: "%(username) s",
password: "%(password) s"
}
}
}
chrome.webRequest.onAuthRequired.addListener(
callbackFn,
{urls: ["<all_urls>"]},
['blocking']
)
""" % {'ip': ip, 'port': port, 'username': username, 'password': password}
plugin_file = 'proxy_auth_plugin.zip'
with zipfile.ZipFile(plugin_file, 'w') as zp:
zp.writestr("manifest.json", manifest_json)
zp.writestr("background.js", background_js)
options = Options()
options.add_argument("--start-maximized")
options.add_extension(plugin_file)
browser = webdriver.Chrome(options=options)
browser.get('https://httpbin.org/get')
print(browser.page_source)
browser.close()
NeedLocalCreate manifest.json ConfigurationFileAnd background.js ScriptSettingsCertificationProxy。Code，Local proxy_auth_plugin.zip FileSaveConfiguration。
ResultAnd，origin Proxy IP。
SOCKS ProxyOfSettingsSimple，OfProtocolModify socks5 ，PasswordCertificationOfProxySettingsMethod：
from selenium import webdriver
proxy = '127.0.0.1:7891'
options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=socks5://' + proxy)
browser = webdriver.Chrome(options=options)
browser.get('https://httpbin.org/get')
print(browser.page_source)
browser.close()
ResultOf。
6.aiohttp OfProxySettings
aiohttp ，CanPass proxy ParameterSettings。HTTP ProxySettings：
import asyncio
import aiohttp
proxy = 'http://127.0.0.1:7890'
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://httpbin.org/get', proxy=proxy) as response:
print(await response.text())
if __name__ == '__main__':
asyncio.get_event_loop().run_until_complete(main())
ProxyUserAndPassword， requests ， proxy ModifyContent：
proxy = 'http://username:password@127.0.0.1:7890'
Need username And password Replace。
SOCKS Proxy，NeedInstallSupport aiohttp-socks，Install：
pip3 install aiohttp-socks
CanOf ProxyConnector Settings SOCKS Proxy，Code：
import asyncio
import aiohttp
from aiohttp_socks import ProxyConnector
connector = ProxyConnector.from_url('socks5://127.0.0.1:7891')
async def main():
async with aiohttp.ClientSession(connector=connector) as session:
async with session.get('https://httpbin.org/get') as response:
print(await response.text())
if __name__ == '__main__':
asyncio.get_event_loop().run_until_complete(main())
ResultOf。
，SupportSettings SOCKS4、HTTP ProxyOfProxyCertification，CanReference。
7. Pyppeteer OfProxySettings
Pyppeteer ，DefaultOf Chrome Of Chromium Browser，SettingsMethodAnd Selenium Of Chrome ， HTTP CertificationProxySettingsMethodPass args SettingsOf，：
import asyncio
from pyppeteer import launch
proxy = '127.0.0.1:7890'
async def main():
browser = await launch({'args': ['--proxy-server=http://' + proxy], 'headless': False})
page = await browser.newPage()
await page.goto('https://httpbin.org/get')
print(await page.content())
await browser.close()
if __name__ == '__main__':
asyncio.get_event_loop().run_until_complete(main())
Result：
{
"args": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "zh-CN,zh;q=0.9",
"Host": "httpbin.org",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3494.0 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-5e8f442c-12b1ed7865b049007267a66c"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
CanSettingsSuccess。
SOCKS Proxy，NeedProtocolModify socks5 ，Code：
import asyncio
from pyppeteer import launch
proxy = '127.0.0.1:7891'
async def main():
browser = await launch({'args': ['--proxy-server=socks5://' + proxy], 'headless': False})
page = await browser.newPage()
await page.goto('https://httpbin.org/get')
print(await page.content())
await browser.close()
if __name__ == '__main__':
asyncio.get_event_loop().run_until_complete(main())
ResultOf。
8. Playwright OfProxySettings
Selenium And Pyppeteer ，Playwright OfProxySettings， proxy Parameter，CanStart Playwright OfSettings。
HTTP Proxy，CanSettings：
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(proxy={
'server': 'http://127.0.0.1:7890'
})
page = browser.new_page()
page.goto('https://httpbin.org/get')
print(page.content())
browser.close()
launch MethodOf，Can proxy Parameter，。Of server，CanWrite HTTP ProxyOfAddress。
Result：
{
"args": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "zh-CN,zh;q=0.9",
"Host": "httpbin.org",
"Sec-Ch-Ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"92\"",
"Sec-Ch-Ua-Mobile": "?0",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4498.0 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-60e99eef-4fa746a01a38abd469ecb467"
},
"origin": "210.173.1.204",
"url": "https://httpbin.org/get"
}
SOCKS Proxy，SettingsMethodOf，Need server Of SOCKS ProxyOfAddress：
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(proxy={
'server': 'socks5://127.0.0.1:7891'
})
page = browser.new_page()
page.goto('https://httpbin.org/get')
print(page.content())
browser.close()
ResultAndOf。
UserAndPasswordOfProxy，Playwright OfSettingsSimple，Need proxy ParameterSettings username And password ，UserAndPassword foo And bar，SettingsMethod：
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(proxy={
'server': 'http://127.0.0.1:7890',
'username': 'foo',
'password': 'bar'
})
page = browser.new_page()
page.goto('https://httpbin.org/get')
print(page.content())
browser.close()
Playwright CertificationProxyOfSettings。
9. Summary
SummaryOfProxy，OfSettingsMethod，Method， IP OfIssue，CanPassProxyOfResolve。
PassGeographyOfProxyService，CanOfGeographyScraping，GetOfData。NeedGetGeographyDataOfTask。PassProxyService，CanHideOf IP Address，OrFrequency。ProtectionOfPrivacyAndSecurity。PassIPAddressOf。ProxyServiceCan，OfSpeedAndOfSuccessScrapingData。
