Refactoring of package management

[geritwagner]

The package management is refactored to make sure that colrev packages are installed as full Python packages with their own dependencies, and that package discovery uses importlib functionality (plugin endpoints associated with the "colrev" group). This means that fully independent colrev packages could be developed and used in the colrev workflow. It would also allow us to reduce dependencies in the core by moving them to the colrev packages.

While the package management is set up for external packages, most colrev packages will remain in the colrev repository. Such a monorepo approach should allow us to refactor the codebase more efficiently, and to ensure consistency across colrev packages. Once the development of the core matures, colrev packages may be extracted to separate repositories (standalone packages).

The plugin management systems of pluggy and llm serve as an inspiration.

TODO:

• [ ] Handling colrev-specific package metadata (dev_status, links to the documentation or readme) and extracting it for the documentation (colrev.package_manager.doc_registry_manager)
  • Package metadata should ideally be added to the pyproject.toml
  • When packages are installed, custom metadata in the pyproject.toml is not made available (e.g., for the doc_registry_manager)
• [ ] The package development documentation must be updated accordingly