Optimize interval timings

[sibalzer]

Example for Contribution:

No Shadowban after 2 Days:

Settings

• sleep_between_requests_in_s=300
• sleep_between_failed_requests_in_s=30
• sleep_after_ipban_in_min=180
• jitter=15

[sibalzer]

Just tested the last days, should be fine too:

Settings:

• sleep_between_requests_in_s=120
• sleep_between_failed_requests_in_s=10
• sleep_after_ipban_in_min=180
• jitter=10

going now lower (60s)

[copakahuna]

Are you guys actually sure, that the ipBan really exists? Never experienced such thing using:

Code

import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

import json
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

from seleniumwire import webdriver

from selenium.webdriver.support.ui import WebDriverWait


datum="1.2.2020"
plzahl="123456"
caps = DesiredCapabilities.CHROME
caps['goog:loggingPrefs'] = {'performance': 'ALL'}
driver = webdriver.Chrome(desired_capabilities=caps)
#driver = webdriver.Chrome()
driver.get("https://www.impfportal-niedersachsen.de/portal/#/appointment/public")

#sleep(1000)

checkBox = driver.find_element_by_name("datenschutz")
checkBox.click()


#weiter=driver.find_element_by_link_text(" Weiter ")
time.sleep(1)

weiter=driver.find_element_by_xpath("//span[contains(text(), ' Weiter ')]")
weiter.click()
time.sleep(1)
weiter.click()
time.sleep(1)

age = driver.find_element_by_id('mat-input-2')

age.send_keys(datum)
weiter.click()
time.sleep(1)
weiter.click()
time.sleep(1)
#time.sleep(5)
sick = driver.find_element_by_tag_name("mat-radio-button")
sick = driver.find_element_by_class_name("mat-radio-container")
driver.execute_script("arguments[0].click();", sick);
gotIt=driver.find_element_by_xpath("//span[contains(text(), 'Verstanden')]")
gotIt.click()
time.sleep(1)
weiter.click()
time.sleep(1)
weiter.click()
time.sleep(1)
plz = driver.find_element_by_name("zipCode")
plz.send_keys(plzahl)
check=driver.find_element_by_xpath("//span[contains(text(), 'Suchen')]")
check.click()
time.sleep(3)

jay= json.loads(driver.last_request.response.body)
outOfStock=jay["resultList"][0]["outOfStock"]
while(outOfStock):
    try:
        check.click()
    except:
        print("Error")
        continue
    #time.sleep(1)
    onStock="Fehler"
    outOfStock=jay["resultList"][0]["outOfStock"]
    onStock=not outOfStock
    print("Impfe verfügbar: "+str(onStock))
print("yeeees")


time.sleep(100000000000000)


driver.close()

[sibalzer]

Yes there are ip bans (403s) and shadowbans with captchas. When and why (and which conditions) lead to a ban are unknown. With 15s on my home connection i had relatively fast only captchas and shortly after 403s, with a bussiness connection and 1s cooldown there was none. How long has your code been running?

[copakahuna]

script has been running for hours with zero sleeptime in between requests. captchas do appear when vaccine was found and selected, on the "enter your telephone number" page, but never on the "search for vaccine" page which i am refreshing in a the while loop. I'm using selenium so the request is getting build by chrome. So maybe the server is detecting by header, that the request is produced by a bot?

[sibalzer]

Using selenium/a real browser don't manke any difference, as far as i can tell. Please report back in a few days if anything has changed.