Allow storing outputs in database

[janpisl]

Implement functionality for storing output data from run processes in a PostgreSQL or SQLite database

Overview

This extension allows output data from run processes to be stored in a PostgreSQL or SQLite database. This extension can be enabled in the configuration (section "server", option "store_type"; when set to "db", the extension is active). Credentials for connecting to the database are found in the (new) section "db".

Contribution Agreement

(as per https://github.com/geopython/pywps/blob/master/CONTRIBUTING.rst#contributions-and-licensing)

I'd like to contribute extension Database Output Storage to PyWPS. I confirm that my contributions to PyWPS will be compatible with the PyWPS license guidelines at the time of contribution.

[cehbrecht]

@janpisl Please check the travis build. There is an import error:

 File "/home/travis/build/geopython/pywps/pywps/inout/storage/db/sqlite.py", line 11, in <module>
    from . import DbStorage
ImportError: cannot import name 'DbStorage'

... and pep8 is failing.

[coveralls]

Coverage decreased (-3.1%) to 69.317% when pulling 43f90421ad06137cc252faee8e711886f70a35f1 on janpisl:db into fbd5b3a39f2f7b6da735b94bd57caad12c267a8c on geopython:master.

[janpisl]

Hi, @cehbrecht , thank you for pointing this out. I have edited my code so it passes the test.

On Mon, Aug 13, 2018 at 3:31 PM, MacPingu [email protected] wrote:

@janpisl https://github.com/janpisl Please check the travis build. There is an import error:

File "/home/travis/build/geopython/pywps/pywps/inout/storage/db/sqlite.py", line 11, in from . import DbStorage ImportError: cannot import name 'DbStorage'

... and pep8 is failing.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/geopython/pywps/pull/371#issuecomment-412518819, or mute the thread https://github.com/notifications/unsubscribe-auth/AexTMxKt0LP24khknXZezia_5kmk6ziJks5uQX_AgaJpZM4V6DwB .

[cehbrecht]

@janpisl I have tested this PR with my Emu processes and I run into an exception (test with Python 3.6):

Traceback (most recent call last):
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/response/execute.py", line 72, in update_status_file                                                       
    self.doc = self._construct_doc()
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/response/execute.py", line 189, in _construct_doc                                                          
    doc = template.render(**self.json)
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/response/execute.py", line 145, in json                                                                    
    data["process"] = self.process.json
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/app/Process.py", line 91, in json                                                                          
    'outputs': [o.json for o in self.outputs],
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/app/Process.py", line 91, in <listcomp>                                                                    
    'outputs': [o.json for o in self.outputs],
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/inout/outputs.py", line 92, in json                                                                        
    data = self._json_reference(data)
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/inout/outputs.py", line 104, in _json_reference                                                            
    data["href"] = self.get_url()
  File "/Users/pingu/Documents/GitHub/birdhouse/pywps/pywps/inout/basic.py", line 711, in get_url                                                                      
    (outtype, storage, url) = self.storage.store(self)
AttributeError: 'NoneType' object has no attribute 'store'

I tried this with the Emu chomsky process. But this exception happens with every process having a complex output.

I have used as config: store_type = file

Currently you have set store_type = db as default. I would keep the file-storage as default.

[janpisl]

@cehbrecht thank you. i will look into it.

[cehbrecht]

@janpisl Please have also a look on the codacy check (see link above next to travis). The complains on the tests can be skipped.