cloudflare-python icon indicating copy to clipboard operation
cloudflare-python copied to clipboard

Pagination on zero_trust.devices.list only returns 1 page

Open ajcollett opened this issue 10 months ago • 9 comments

Confirm this is a Python library issue and not an underlying Cloudflare API issue.

  • [x] This is an issue with the Python library

Describe the bug

When using either the automatic pagination or the "has_next_page" manual pagination on the client.zero_trust.devices.list( endpoint, only 1 page is returned (20 devices).

The "has_next_page" function returns False.

This applies to async and non-async calls.

I can confirm that I have much more than 20 devices on the account, and the old API used to fetch them all.

This has been present in the lib for a while (months). I tried this lib out a while ago and found a few issues, but didn't get to reporting them then. I thought I'd mention that in case it helps narrow the issue down.

To Reproduce

  1. Try fetch all devices, either by automatic pagination or manual.
  2. Append to list
  3. Check length of list to find it's too short.

Code snippets

from cloudflare import Cloudflare

client = Cloudflare()

all_devices = []

for device in client.zero_trust.devices.list(account_id=***):
    all_devices.append(device)

len(all_devices)
# In my case, outputs 20 instead of 91

OS

Ubuntu 24.04

Python version

3.12.3

Library version

v4.1.0

ajcollett avatar Mar 27 '25 10:03 ajcollett

this is due to the internal teams schema not correctly populating pagination information and it renders a SyncSinglePage instead of looping through the available items.

jacobbednarz avatar Mar 28 '25 00:03 jacobbednarz

@jacobbednarz thanks for the response. So, what do we do from here? Or are you handling it internally?

ajcollett avatar Mar 28 '25 10:03 ajcollett

Hi, this error is happening because your Python file is named cloudflare.py.

When you write import cloudflare, Python ends up importing your own file instead of the actual Cloudflare library. That’s why it can’t find the CloudFlare class and throws an AttributeError.

The fix is simple — just rename your file to something else, like:

cloudflare.py ❌
cf_test.py ✅
main.py ✅

Then try running it again — it should work fine 🙂

Hope this helps!

codewithdhruba01 avatar May 08 '25 04:05 codewithdhruba01

@codewithdhruba01 are you claiming that I have a file called cloudflare.py that is interfearing? That's not the case.

ajcollett avatar May 08 '25 11:05 ajcollett

The issue at hand is not an AttributeError at all, so that diagnosis is a mile off course. I guess it's just AI slop to brush up their GitHub profile's activity.

BjoernPetersen avatar May 08 '25 12:05 BjoernPetersen

So, it seems like the problem is related to how pagination is being handled in the Python library for zero_trust.devices.list().

Even though there are more than 20 devices, only the first page is returned and has_next_page is incorrectly showing False. This might indicate a bug in the internal pagination logic of the SDK, not in the API itself.

Could this be related to the way pagination tokens or result info are being handled in the SDK? If needed, I’d be happy to help look into it further or test a patch.

codewithdhruba01 avatar May 08 '25 13:05 codewithdhruba01

Noticed the same for pages.projects.deployments.list. Even on async mode, it's only returning 1 page worth of results.

MrSaints avatar Jun 05 '25 00:06 MrSaints

Likewise for zero_trust.access.users.list. Believe some of the other clients like the golang one has the same issue.

My temp workaround:

cf_users = []
cf_users_page = await cf_client.zero_trust.access.users.list(account_id='...')
cf_users += cf_users_page.result
result_info = cf_users_page.model_extra['result_info']
while result_info['page'] < result_info['total_pages']:
    cf_users_page = await cf_client.zero_trust.access.users.list(account_id='...', extra_query={'page': result_info['page'] + 1})
    result_info = cf_users_page.model_extra['result_info']
    cf_users += cf_users_page.result

yndai avatar Jun 17 '25 20:06 yndai

Looks like the same issue as #2584 ?

jhutchings1 avatar Jun 30 '25 21:06 jhutchings1