scrapi Document Type Support

Adds preliminary support for a documentType field for crossref & plos harvesters

Apr 14 '16 17:04 wearpants

Coverage increased (+0.2%) to 93.642% when pulling 536cd97b4c62507e9e19b3bc0d00c063d6d7ce2b on wearpants:feature/document-type into 05282578751f52e3c955d0b190c5628dd221389a on CenterForOpenScience:develop.

Apr 14 '16 18:04 coveralls

Pull request with initial support for a documentType field for crossref and plos. See SHARE-294 in Jira. /cc @chrisseto It supports the following values: "article", "abstract", "dataset", "book", "book-chapter", "dissertation", "correction", "preprint", "source-code", "clinical-trial", "reference-entry", "monograph"

To update production, we'll need to renormalize after merging: invoke migrate renormalize -s 'crossref,plos' --no-dry. I estimate this will take about 23 machine hours - may be able to get some speedup by using more celery tasks, but not 100% sure where the bottleneck is. Be sure to merge branch on all harvester machines before running this migration.

@alexschiller if you'd like to start adding frontend support, checkout the feature/document-type branch from my fork: https://github.com/wearpants/scrapi/ ; You can either renormalize an existing database or re-run the crossref & plos harvesters to generate new data. The OSF web server should pass through documentType field in share_search API results.

Apr 14 '16 22:04 wearpants

Coverage increased (+0.2%) to 93.642% when pulling f0941bd28685b105c7455fda25600cc9351011aa on wearpants:feature/document-type into 05282578751f52e3c955d0b190c5628dd221389a on CenterForOpenScience:develop.

Apr 15 '16 15:04 coveralls