Getting Error while try to fetch records using SharePoint Rest API.

[visualscrapy]

What type of issue is this?

Question

What SharePoint development model, framework, SDK or API is this about?

SharePoint REST API

Target SharePoint environment

SharePoint Online

What browser(s) / client(s) have you tested

• [ ] 💥 Internet Explorer
• [ ] 💥 Microsoft Edge
• [X] 💥 Google Chrome
• [ ] 💥 FireFox
• [ ] 💥 Safari
• [ ] mobile (iOS/iPadOS)
• [ ] mobile (Android)
• [ ] not applicable
• [X] other (enter in the "Additional environment details" area below)

Additional environment details

• browser version
• SPFx version
• Node.js version
• etc PostMan

Issue description

In My site, I had 50,000+ records so I am trying to fetch records using SharePoint rest API call. Ex: https://tenent.sharepoint.com/sites/.../_api/lists/getbytitle('Documents')/files$skiptoken=Paged=TRUE&p_ID=39000&$top=1000 once it's reached to specific records it's throwing error. Column '_mpSPSiteId' does not exist.

• I am using next_page to iterate data['d']['__next']. It will take you to the next page.
• Note: If it's a server-side issue how I can resolve it?
• Even I am trying to implement using skiptoken is it the correct approach?
• Goal: Fetch all records without missing any records from the Site using SPO rest api.

[ghost]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[visualscrapy]

Fetch all records from the SharePoint site using SharePoint rest API.

[visualscrapy]

using rest api call.

[hack-tom]

I'm having the same issue - this is specifically an error occurring from following the __next variable in the paginated response, please see below and ignore the debugging print statements.

nextUrl = "http://" + tenant  + "/sites/" + siteName + "/_api/files"
reqCount = 0
while nextAvailable == True:
    l = requests.get(nextUrl, headers=headers)

    if l.text[:25] != """{"d":{"results":[{"__meta""":
        print("({}) Request Failed".format(reqCount), nextUrl.replace("----", "[TENANT]").replace("----", "[SITE]"))
        print(l.text)
    else:
        print("({}) Request Successful".format(reqCount), nextUrl.replace("----", "[TENANT]").replace("----", "[SITE]"))
    
    items = json.loads(l.text)
    for item in items['d']['results']:
        if item["__metadata"]['type'] == "MS.FileServices.File":
            files.append(item)

    if "__next" in items['d']:
        nextUrl = items['d']["__next"]
    else: 
        nextAvailable = False
    reqCount += 1

Precise error message is:

{"error":{"code":"-2147024809, System.ArgumentException","message":{"lang":"en-US","value":"Column '_mpSPSiteId' does not exist. It may have been deleted by another user."}}}

As far as I am aware, no columns have been renamed or deleted - it is highly unlikely that anyone else has modified either. This error is reliably repeatable, occurring at the same point (12th page request). If this was genuinely a columns issue I would expect the first 12 page requests to fail too.

Worth noting that this issue did not occur on four previous sites that this code has ran against, I have 13 further SharePoint sites to process excluding this failing one and will update again if I find more examples of this issue.

UPDATE: this has occured on three further sites - I have confirmed that the issue is not new by verifying the same code still works on one of the original (successful) sites. The depth of pagination does not seem to be a factor, with the sites taking different (but consistent & repeatable) numbers of next page requests to fail.

[jerodj-cdata]

Our (CData Software) customers have recently started encountering this issue as well.

The problem is identical to what @hack-tom describes: several requests to the files endpoint are successful until one of the "next" links returned from SharePoint gives the "'_mpSPSiteId' does not exist" error.

In our case, we see this even with an explicit "$select=Id%2cUrl" on the request.

[hack-tom]

Apologies for the late return - I managed to get in touch with the product team and after a lengthy investigation they have found and resolved the issue on their side, which I believe has been pushed to all sharepoint online customers. In my own testing, the issue has been resolved.