python pkg_resources performance issues

[tgbugs]

What version of protobuf and what language are you using? Version: master Language: Python What operating system (Linux, Windows, ...) and version? Gentoo linux, but the problem is present on all platform.

What runtime / compiler are you using (e.g., python version or gcc version) Version independent, but, pypy3, python3.6, python3.7. What did you do? Steps to reproduce the behavior:

1. Have many (hundreds of?) python packages present on your system.
2. In your favourite shell run python
3. import google.protobuf or even just import google

What did you expect to see Not waiting 300ms for google or google.protobuf to improt. What did you see instead? Waiting 300ms for import to finish because pkg_resources scans all installed packages.

Anything else we should know about your project / environment This is a result of https://github.com/pypa/setuptools/issues/510. Suggested fix is to remove all top level imports of pkg_resources. For 3.7 and 3.8 it could be replaced with importlib.resources or importlib.metadata and apparently there are backports as well (see https://github.com/hgrecco/pint/issues/1008#issuecomment-610451542). Here is the list of bad import statements. https://github.com/protocolbuffers/protobuf/blob/97f5f649a419279368fd10036dca48c6d8ed0f1f/python/google/init.py#L2 https://github.com/protocolbuffers/protobuf/blob/97f5f649a419279368fd10036dca48c6d8ed0f1f/python/google/protobuf/init.py#L37 https://github.com/protocolbuffers/protobuf/blob/97f5f649a419279368fd10036dca48c6d8ed0f1f/python/google/protobuf/pyext/init.py#L2

[haberman]

Can you send a PR? It's not exactly clear to me what we should change this to.

We currently require Python 3.8 or higher.

Can we just delete our __init__.py files entirely, per PEP 420?

[tgbugs]

Will see if I can dig back in on this and figure out what the current behavior is vs what removing some of the __init__.py files is. I think the main test case is that various import statements continue to work after the change.

[haberman]

Sounds good. I'm unassigning myself since I'm not working on this myself. But I'd be happy to see a PR.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.

This issue was closed and archived because there has been no new activity in the 14 days since the inactive label was added.

[cwegener]

I just ran into this legacy namespace problem myself on Ubuntu 22.04 when trying to use python3-protobuf

I don't even understand why namespacing is used in the first place here?

Isn't there just ONE package that is importable anyway? (google.protobuf)

Why would namespacing be needed with just ONE single package? Does anybody know?