migrate icon indicating copy to clipboard operation
migrate copied to clipboard

Metastore Export does not respect column constraints.

Open IceRhymers opened this issue 4 years ago • 2 comments

Issue description

When exporting databases from the metastore, the table definitions generated do not have constraints such as 'is not null' defined for columns that have them in the original.

Steps to reproduce the issue

  1. In Workspace A define a table with a column constraint, such as 'is not null'.
  2. Export the database from workspace A.
  3. Attempt to import the database to Workspace B.

What's the expected result?

  • The database and table will be created.

What's the actual result?

  • An error, explaining that the schema expected does not match schema that already exist, because the existing schema has a constraint and the target schema does not.

Additional details / screenshot

  • Screen Shot 2021-08-10 at 4 30 33 PM
  • Screen Shot 2021-08-10 at 4 21 08 PM

IceRhymers avatar Aug 11 '21 18:08 IceRhymers

Thanks for reporting this. I don't see a method to get this information from the metastore yet. I've looked at desc extended <table_name> and show create table <table_name>, but they don't produce the restrictions here. I'll continue looking to see if this is exposed in any manner.

mrchristine avatar Aug 11 '21 20:08 mrchristine

@IceRhymers I've discussed this w/ our eng team and they've identified this as a gap w/ Databricks Delta. They will be filing a ticket to resolve this, but there's no timeline yet. I'll keep this open for now until it's resolved. Appreciate your help on this!

mrchristine avatar Aug 12 '21 23:08 mrchristine