sickle
sickle copied to clipboard
mloesch
Sickle: OAI-PMH for Humans
Iteration with next() gets very slow when the OAIItemIterator is "empty" but StopIteration has not been raised yet. It takes several minutes. Example: `from sickle import Sickle oai_end = 'http://ws.pangaea.de/oai/provider'...
I am very grateful for this OAI-PMH python package plus the good documentation! But: Code from [tutorial](https://sickle.readthedocs.io/en/latest/): sickle = Sickle('http://elis.da.ulcc.ac.uk/cgi/oai2') records = sickle.ListRecords(metadataPrefix='oai_dc') yields following error: HTTPError: 502 Server Error:...
https://data.csiro.au/dap/ws/v2/collections is an example of a server that sends back JSON rather than XML to a sickle request by default. Adding headers={'Accept': 'application/xml'} to the Sickle() constructor fixes that, but...
Hello, and thanks for this great project. I'm running `Sickle` on Python 3.8.5, but this issue seems to be valid for Python 3.6 upwards due to a change in `re`...
This is a minor PR adding support for sessions while using `requests`. By using sessions, `requests` can reuse the same TCP connection while making requests to the same host, which...
Per #43, this is my take at replacing the custom retry code with the tried and tested urllib3 Retry class. Benefits include that it also handles `Timeout` and `ConnectionError`, and...
I'm harvesting from a server which frequently times out (`requests.exceptions.Timeout`). Then, the request is not retried even though I set `max_retries`, since the retry functionality only covers the case where...
It looks like this project is semantically versioned, but still on major release zero. I see encouraging recent activity, but just wanted to check if the versioning means the project...
as the title suggests, the http spec says that a retry-after header can be a date of format... or an int. in the wild I am getting date values returned,...
Without `huge_tree=True`, lxml parsing apparently fails on certain, even slightly largish responses (apparently of more than 9.5MB). Because also `recover=True`, from the viewpoint of Sickle, this happens silently. I only...
