ckanext-cloudstorage icon indicating copy to clipboard operation
ckanext-cloudstorage copied to clipboard

resource_create API request returning 504 timeout error

Open hammadhasandogar opened this issue 6 years ago • 2 comments

api/action/resource_create works fine with local storage ckan.storage_path = /var/lib/ckan/default but after enabling cloudstorage it gives timeout error while making request from UI or Postman

am i missing anything here?

hammadhasandogar avatar Jul 31 '19 14:07 hammadhasandogar

I was able to resolve the timeout error, now metadata get saved but not the file, neither on S3 nor on local. After diving into code, I think below is what I got to do:

  1. Call resource_create API - it saves the metadata
curl -H'Authorization: <api key>' \
'http://<instance ip>/api/action/resource_create' \
--form [email protected] --form package_id=public --form name="curl file"
  1. Then cloudstorage_initiate_multipart - it gives back upload key
curl -H'Authorization: <api key>' \
'http://<instance ip>/api/action/cloudstorage_initiate_multipart' \
-d '{"id":"e3876b08-27d2-42ba-b088-ee4789091b83","name":"metadata.sql","size":3610 }'
  1. Then cloudstorage_upload_multipart - not working, gives error(AttributeError: SpooledTemporaryFile instance has no attribute 'file') on ckanext/cloudstorage/logic/action/multipart.py", line 155, in upload_multipart
curl -H'Authorization: <api key>' \
'http://<instance ip>/api/action/cloudstorage_upload_multipart' \
 --form "id=e3876b08-27d2-42ba-b088-ee4789091b83" --form "[email protected]" --form "partNumber=1" --form "uploadId=XXXXXXXXX.qqXAQPCxZFij0U1pVpsbVjpzkO3H6Rtang1x2FVS.cJgWkQPjbKvCNzoPQZ2S09dsmG4FcQI_qyRWmRUz3sHWLWgh9L3lEckOAJteeaIHBvlC6tniLh1gu"
  1. Finally cloudstorage_finish_multipart - not working, gives error (Error in multipart commit: The XML you provided was not well-formed or did not validate against our published schema (MalformedXML)) on ckanext/cloudstorage/logic/action/multipart.py", line 198, in finish_multipart, because the option 3 wasn't complete i guess
curl -H'Authorization: <api key>' \
'http://<instance ip>/api/action/cloudstorage_finish_multipart' \
-d '{"save_action":"go-metadata","id":"e3876b08-27d2-42ba-b088-ee4789091b83","uploadId":"XXXXXXXXX.qqXAQPCxZFij0U1pVpsbVjpzkO3H6Rtang1x2FVS.cJgWkQPjbKvCNzoPQZ2S09dsmG4FcQI_qyRWmRUz3sHWLWgh9L3lEckOAJteeaIHBvlC6tniLh1gu"}'

Can anyone confirm please?

hammadhasandogar avatar Aug 04 '19 08:08 hammadhasandogar

Were you ever able to resolve this?

TkTech avatar Nov 06 '19 12:11 TkTech