socrata-py icon indicating copy to clipboard operation
socrata-py copied to clipboard

Unable to create an external dataset, using Python

Open DLakin01 opened this issue 6 years ago • 1 comments

I'm working on automating more of USAID's uploads to its Development Data Library via the Socrata API, and I'm getting stuck when trying to create one or more external datasets (with a link to data on another site, no actual data uploaded). For a given dataset entity, I'm trying to set the external url in accessPoints and metadata:additionalAccessPoints:urls, but the link does not appear after creation in Socrata.

Code:

(ok, revision) = Socrata(self.auth_obj).new({
                "name": dataset["name"],
                "description": dataset["description"],
                "accessPoints": {
                    dataset["type"]: dataset["url"]
                }
            })

................

if "url" in dataset:
            metadata["metadata"]["additionalAccessPoints"] = [{
                "urls": {
                    dataset["type"]: dataset["url"]
                }
            }]

  r = push_metadata(ds_fourfour, self.link, self.username, self.password, metadata, '1')

  r = apply_revision(ds_fourfour, self.link, self.username, self.password, '1')

Any tips?

DLakin01 avatar Aug 12 '19 21:08 DLakin01

I am running into the same problem, but I am starting to doubt the functionality is available with this existing codebase. If you look at the code for Revisions.new:

@staticmethod
def new(auth, metadata):
    path = 'https://{domain}/api/publishing/v1/revision'.format(
        domain = auth.domain,
    )

    response = post(
        path,
        auth = auth,
        data = json.dumps({
            'action': {
                'type': 'update'
            },
            'metadata': metadata
        })
    )
    return Revision(auth, response)

The revision's JSON data has two top-level attributes, action and metadata. However, if you look at Socrata's Publishing api (which socrata-py wraps around), you'll see that in order to add an external URL, you need a third top-level attribute, href (pictured below).

Socrata Publishing API

I added a small amount of code to implement external datasets. Check out the Pull Request here

evantayloryates avatar Oct 28 '20 00:10 evantayloryates