oeplatform icon indicating copy to clipboard operation
oeplatform copied to clipboard

manage.py mirror does not create '_ prefixed schemas

Open pajot opened this issue 3 years ago • 0 comments

I am trying to configure a new schema for table creation.

I have an existing schema containing tables with data that I want to use for this purpose.

I have renamed the schema in the backend with

ALTER SCHEMA <oldname> RENAME TO <newname>;

I modify schema_whitelist in dataedit.views accordingly, and change DEFAULT_SCHEMA to <newname>.

If I now run python manage.py mirror, it will output the table names with the new schema <newname>, but the expected _<newname> is not created.

If I try to test this with oep-client:

python -m oep_client.cli --host mdbt:8000 --token 8984bcef4077b4a6a88f5f709728c614f1b0c6aa -l debug test <newname> the test fails with the error

Internal Server Error: /api/v0/schema/<newname>/tables/test_table_65483024/
Traceback (most recent call last):
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.InvalidSchemaName: schema "_<newname>" does not exist
LINE 1: CREATE TABLE "_<newname>"."_test_table_65483024_edit" (LIKE "up...
                     ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/pajot/oep-website/./api/views.py", line 178, in wrapper
    return f(*args, **kwargs)
  File "/home/pajot/oep-website/./api/views.py", line 409, in put
    self.__create_table(
  File "/home/pajot/oep-website/./api/views.py", line 89, in wrapper
    result = f(*args, **kwargs)
  File "/home/pajot/oep-website/./api/views.py", line 450, in __create_table
    actions.table_create(
  File "/home/pajot/oep-website/./api/actions.py", line 897, in table_create
    get_edit_table_name(schema, table)
  File "/home/pajot/oep-website/./api/actions.py", line 1951, in get_edit_table_name
    create_edit_table(schema, table)
  File "/home/pajot/oep-website/./api/actions.py", line 2004, in create_edit_table
    create_meta_table(schema, table, meta_table, meta_schema)
  File "/home/pajot/oep-website/./api/actions.py", line 1994, in create_meta_table
    engine.execute(query)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2235, in execute
    return connection.execute(statement, *multiparams, **params)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1003, in execute
    return self._execute_text(object_, multiparams, params)
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1172, in _execute_text
    ret = self._execute_context(
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/home/pajot/oep-website/venv-mondb/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidSchemaName) schema "_<newname>" does not exist
LINE 1: CREATE TABLE "_<newname>"."_test_table_65483024_edit" (LIKE "up...
                     ^

[SQL: CREATE TABLE "_<newname>"."_test_table_65483024_edit" (LIKE "<newname>"."test_table_65483024"INCLUDING ALL EXCLUDING INDEXES, PRIMARY KEY (_id) ) INHERITS (_edit_base);]
(Background on this error at: http://sqlalche.me/e/13/f405)

The schema exists in the dataedit.schemas table in oep_django, so this mirror operation must be doing something, but not finishing.

pajot avatar Nov 03 '22 17:11 pajot