Py-StackExchange icon indicating copy to clipboard operation
Py-StackExchange copied to clipboard

Too Many Requests, but I thought I only made 1

Open nueverest opened this issue 7 years ago • 2 comments

For fun I wanted to add up all of the reputation across all users.

Here is the code sample.

stack_auth = StackAuth()
so = Site(StackOverflow, impose_throttling=True)

# Get all users
users = so.users()             # Thought this is a single request.
total_reputation = 0

for user in users:
    total_reputation += user.reputation

print('Total Reputation:', total_reputation)

But after running the code and waiting a minute or two I get this Throttling Error:

raise StackExchangeError(error_ob.get('error_id', StackExchangeError.UNKNOWN), error_ob.get('error_name'), error_ob.get('error_message')) stackexchange.core.StackExchangeError: 502 [throttle_violation]: too many requests from this IP, more requests available in 45500 seconds

Does user.reputation cause a request, and if so why?

nueverest avatar Feb 23 '18 13:02 nueverest

Hi,

user.reputation doesn't cause a request, but there are some 'hidden' requests made when you iterate over so.users(). The Stack Exchange API returns the users in pages of (I believe) 60 at a time. In attempt to give you all the users in the request (in this case, the whole site), the Py-StackExchange library will make a request for the next page once the loop has passed the sixtieth user. It will keep requiring new pages as needed until it reaches the end. There thousands of users on Stack Overflow, so this will result in hundreds of requests.

I'm not sure there is an efficient way to sum the total reputation through the API. Perhaps https://data.stackexchange.com/ is a more helpful tool here?

On 23 Feb 2018 1:13 pm, "nueverest" [email protected] wrote:

For fun I wanted to add up all of the reputation across all users.

Here is the code sample.

stack_auth = StackAuth() so = Site(StackOverflow, impose_throttling=True)

Get all users

users = so.users() total_reputation = 0

for user in users: total_reputation += user.reputation

print('Total Reputation:', total_reputation)

But after running the code and waiting a minute or two I get this Throttling Error:

raise StackExchangeError(error_ob.get('error_id', StackExchangeError.UNKNOWN), error_ob.get('error_name'), error_ob.get('error_message')) stackexchange.core.StackExchangeError: 502 [throttle_violation]: too many requests from this IP, more requests available in 45500 seconds

Does user.reputation cause a request, and if so why?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lucjon/Py-StackExchange/issues/69, or mute the thread https://github.com/notifications/unsubscribe-auth/AAClWHkaU3Foxech1nKf6lYp68d1dn4_ks5tXrmRgaJpZM4SQ3Og .

lucjon avatar Feb 24 '18 17:02 lucjon

Thanks I looked at the link. It seems there is around 1B total rep on stackoverflow, but it is unclear how to check stats on other sites in the stackexchange family.

nueverest avatar Feb 27 '18 14:02 nueverest