[EPIC] Multi-workspace installations

[nkvuong]

Make UCX work across multiple workspaces

Relevant issues:

• [x] https://github.com/databrickslabs/ucx/issues/749
• [x] https://github.com/databrickslabs/ucx/issues/1469
• [x] https://github.com/databrickslabs/ucx/issues/1194
• [x] https://github.com/databrickslabs/ucx/issues/1783
• [x] https://github.com/databrickslabs/ucx/issues/1782
• [ ] https://github.com/databrickslabs/ucx/issues/335
• [x] https://github.com/databrickslabs/ucx/issues/673
• [x] https://github.com/databrickslabs/ucx/issues/918
• [x] https://github.com/databrickslabs/ucx/issues/919
• [x] https://github.com/databrickslabs/ucx/issues/917
• [x] https://github.com/databrickslabs/ucx/issues/344
• [x] https://github.com/databrickslabs/ucx/issues/343
• [x] https://github.com/databrickslabs/ucx/issues/1784

Problem statement

Customers with many workspaces will need abilities to run assessment & migration for multiple workspaces at once.

Capabilities:

• join existing workspace collection (& re-sync the relevant info from it). There are 3 entry points

  • Account-level installation https://github.com/databrickslabs/ucx/issues/920
  • New installation https://github.com/databrickslabs/ucx/issues/918
  • Existing installation https://github.com/databrickslabs/ucx/issues/919

• dedupe conflict

  • check ws info is in sync, prompt if not and forcefully sync
  • https://github.com/databrickslabs/ucx/issues/335
  • https://github.com/databrickslabs/ucx/issues/917

Related issues:

• https://github.com/databrickslabs/ucx/issues/673
• https://github.com/databrickslabs/ucx/issues/335
• https://github.com/databrickslabs/ucx/issues/17
• https://github.com/databrickslabs/ucx/issues/917
• https://github.com/databrickslabs/ucx/issues/803
• https://github.com/databrickslabs/ucx/issues/890

Proposed Solution

ucx should discover all workspaces that users have access to, and offer configuration to group those into "collections". This can either be done via prefix, tags or manually, e.g. yaml config

Collection configuration would then be stored in all workspaces installation

Also need to handle duplications in mapping tables

Additional Context

No response