Issue

This Content is from Stack Overflow. Question asked by Saeed

I want to use proxy lists but I’m not sure how to do this:

Check if first connection is established (it does not matter it is the first proxy IP or not), run something and then close the program and exit, else go to the next proxy.

This is my code that loops over all proxies:

import requests
from Proxy_List_Scrapper import Scrapper, Proxy, ScrapperException
ALL = 'ALL'
scrapper = Scrapper(category=ALL, print_err_trace=False)
# Get ALL Proxies According to your Choice
data = scrapper.getProxies()
n = 0
while True:
    n += 1
    try:
        for item in data.proxies:
            # Enter a proxy IP address and port.
            url = 'http://api.ipify.org'

            # Send a GET request to the url and pass the proxy as a parameter.
            page = requests.get(url, proxies={"http": f'{item.ip}:{item.port}', "https": f'{item.ip}:{item.port}'})

            # Prints the content of the requested url.
            print(f'proxy is {item.ip}:{item.port} and content is {page.text}')
    except requests.exceptions.ProxyError:
        print(f'error #{n}')

Solution

Maybe you should run try/except inside for-loop – to catch problem with proxy server and continue with next proxy on list. And you could use break to finish loop when you find working proxy.

Something like this:

import requests
from Proxy_List_Scrapper import Scrapper, Proxy, ScrapperException

scrapper = Scrapper(category='ALL', print_err_trace=False)
data = scrapper.getProxies()

url = 'http://api.ipify.org'

for n, item in enumerate(data.proxies):
    try:
        page = requests.get(url, proxies={"http": f'{item.ip}:{item.port}', "https": f'{item.ip}:{item.port}'})

        print(f'proxy is {item.ip}:{item.port} and content is {page.text}')
        
        # exit `for`-loop when connection finished without error and it gives different IP
        if page.text != my_original_ip:
            break  
    except requests.exceptions.ProxyError:
        print(f'error #{n}  {item.ip}:{item.port}')

This Question was asked in StackOverflow by Saeed and Answered by furas It is licensed under the terms of CC BY-SA 2.5. - CC BY-SA 3.0. - CC BY-SA 4.0.