[SOLVED] Once reached last page, Python Selenium Script to continue instead of wait indefinitely


This Content is from Stack Overflow. Question asked by Cave Johnson123

I am trying to click all next pages until last page which it does successfully on this website. However, it reaches the last page and then waits indefinitely. How can I best achieve this script to then proceed to the rest of the script once it reaches the last page? I could do an explicit wait time of 15 seconds timeout but this feels very slow and not the best way of doing this. Thanks

i = 1
while i < 6:
        #time.sleep(random, 3)
        WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR,  ".name:nth-child(1)")))
        WebDriverWait(driver, 100).until(lambda driver: driver.execute_script('return document.readyState') == 'complete')
        element = WebDriverWait(driver, 20).until(lambda driver: driver.find_element(By.CSS_SELECTOR, ".name:nth-child(1) , bf-coupon-table:nth-child(1) tr:nth-child(1) .matched-amount-value"))
        scroll = driver.find_element(By.CSS_SELECTOR, ".coupon-page-navigation__label--next")
        driver.execute_script("arguments[0].scrollIntoView();", scroll)
        link = driver.find_element_by_css_selector('[href^=http://somelink.com/]')
        WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "bf-coupon-page-navigation > ul > li:nth-child(4) > a")))
        NextStory = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'bf-coupon-page-navigation > ul > li:nth-child(4) > a')))
        link = driver.find_element_by_css_selector('bf-coupon-page-navigation > ul > li:nth-child(4) > a')
        i = 6


You are not getting to the except block, otherwise the loop would stop.

I think that this line is causing the slowness:

driver.execute_script("arguments[0].scrollIntoView();", scroll)

That is bacause all your other actions are driver actions with timeouts. Try clicking on the webpage and then just click page down key in a loop:

from selenium.webdriver.common.keys import Keys
# Click on a static element in your page
for _ in range(100):

This Question was asked in StackOverflow by Cave Johnson123 and Answered by Tal Angel It is licensed under the terms of CC BY-SA 2.5. - CC BY-SA 3.0. - CC BY-SA 4.0.

people found this article helpful. What about you?