A Way to get all list values for a CollectionResponse?

[jussihi]

Is your feature request related to a problem? Please describe the problem.

Is there already a way to get all items for a CollectionResponse? Currently the API/SDK returns only a page of results, and then we need to fetch more by using odata_next_link.

Is there a way to get all results in a single request?

I have made a helper function which I use, is this the best way;

# Takes in any CollectionResponse
async def get_complete_paginated_list(
    request_builder_func: Any,
    request_configuration: Optional[Any] = None
) -> List[Any]:
    ret: List[Any] = []
    collection_response: Any = await request_builder_func.get()

    # Nothing to return here
    if collection_response is None or collection_response.value is None:
        return ret

    def add_response_items(response: Optional[Any], items: List[Any]) -> None:
        if response is not None and response.value is not None:
            for item in response.value:
                items.append(item)

    # Add initial results
    add_response_items(collection_response, ret)

    # Iterate pages
    while collection_response is not None and collection_response.odata_next_link is not None:
        collection_response: Optional[Any] = await request_builder_func.with_url(collection_response.odata_next_link).get(request_configuration = request_configuration)
        if collection_response is not None:
            add_response_items(collection_response, ret)

    return ret

It can then be used like this; mailfolders: List[MailFolder] = await get_complete_paginated_list(graph_client.users.by_user_id(user_id).mail_folders)

Describe the solution you'd like.

something like what I've made

Additional context?

No response

[shemogumbe]

Hello @jussihi thanks for using the SDK and for raising this.

Imagine a situation where you have a request that gives thousands or even millions of records, getting all of these in one API call becomes not feasile, to solve for this, graph allows for painated responses, with next pages being obtained using odata_next_link.

You can loop through these and extract page contents storing them in memory for your operation if it solves for your specific need. However, building a feature that gets all items in one go would not be a good idea for situations where an entity has a very high numr of records.

Another way to get content of pages is the page Iterator task in https://github.com/microsoftgraph/msgraph-sdk-python-core/blob/main/src/msgraph_core/tasks/page_iterator.py

[jussihi]

@shemogumbe

Ah! So such feature exists :) Could you give an example on how to use the PageIterator ? This would be a good addition to docs, if I knew about its existence, I would've used that, I think.

[shemogumbe]

Check out samples here, also doing a samples PR for easy discovery of the feature and how to use it