Gracefully handle negative page offsets

[andrewpollock]

As seen in a review of the logs, someone or something is feeding negative page offsets, which results in undesired noise in the logs:

Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/srv/frontend_handlers.py", line 171, in list_vulnerabilities
    results = osv_query(query, page, False, ecosystem)
  File "/srv/frontend_handlers.py", line 377, in osv_query
    bugs, _, _ = query.fetch_page(
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/query.py", line 1214, in wrapper
    return wrapped(self, *dummy_args, _options=query_options)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/utils.py", line 121, in wrapper
    return wrapped(*args, **new_kwargs)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/utils.py", line 153, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/query.py", line 2281, in fetch_page
    return self.fetch_page_async(None, _options=kwargs["_options"]).result()
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 210, in result
    self.check_success()
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 157, in check_success
    raise self._exception
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 330, in _advance_tasklet
    yielded = self.generator.throw(type(error), error, traceback)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/query.py", line 2324, in fetch_page_async
    while (yield iterator.has_next_async()):
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 330, in _advance_tasklet
    yielded = self.generator.throw(type(error), error, traceback)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/_datastore_query.py", line 340, in has_next_async
    yield self._next_batch()  # First time
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 330, in _advance_tasklet
    yielded = self.generator.throw(type(error), error, traceback)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/_datastore_query.py", line 370, in _next_batch
    response = yield _datastore_run_query(query)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 330, in _advance_tasklet
    yielded = self.generator.throw(type(error), error, traceback)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/_datastore_query.py", line 1018, in _datastore_run_query
    response = yield _datastore_api.make_call(
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 330, in _advance_tasklet
    yielded = self.generator.throw(type(error), error, traceback)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/_retry.py", line 96, in retry_wrapper
    raise error
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/_retry.py", line 82, in retry_wrapper
    result = yield result
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/tasklets.py", line 330, in _advance_tasklet
    yielded = self.generator.throw(type(error), error, traceback)
  File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/ndb/_datastore_api.py", line 99, in rpc_call
    raise error
google.api_core.exceptions.InvalidArgument: 400 Offset must be non-negative. 

[msaad7777]

def main_flow():
    for i in range(9, 11):
        inp = f"2022-01-0{i} 01:00:00+0000"
        prefect.get_run_logger().info(inp)
        page = max(0, page)  # Ensure page is non-negative
        sub_flow(inp, return_state=True)

Above is an example
It seems that there is an issue with the page offset, which is resulting in noise in the logs. The logs indicate that negative page offsets are being fed, which is causing the problem. The error message shows that the offset must be non-negative.

To fix this issue, you should check the code to ensure that the page offset is not negative. You may need to modify the code to prevent negative page offsets from being generated and fed into the query. Once the code is fixed, you should test it to ensure that the logs no longer show any noise related to negative page offsets.

[Unknownsentinel193]

Hi, can I work on this? Can you please assign this to me.

[ZhangChen199102]

Do we know in which case this will happen? I'm not quite sure how to replicate this error on local.

[another-rex]

It needs to be a turbo request to trigger it, so if you replay a "Load More" request, but change the page query to a negative number, you'll get this error.

Example

curl 'http://127.0.0.1:8000/list?page=-4' \
  -H 'Accept: text/html, application/xhtml+xml' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Referer: http://127.0.0.1:8000/list?page=3' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'Turbo-Frame: vulnerability-table-page4'