[BUG] mysqlclient (MySQLdb) import problem 3007.2

[jblac]

Description The Salt mysql module fails to load on a minion when using the bundled Python 3.10 environment (/opt/saltstack/salt/bin/python3.10) due to an outdated libstdc++.so.6 in /opt/saltstack/salt/lib. The module’s virtual function returns False with "No python mysql client installed," even though mysqlclient 1.4.6 is installed and MySQLdb is importable in the system Python 3.12.

** Steps to Reproduce **

1. Install Salt 3007.2 on Ubuntu 24.04.2 LTS with /opt/saltstack as the install path.
2. Install mysqlclient 1.4.6 in the system Python (/usr/bin/python3).
3. Verify MySQLdb imports in system Python:

/usr/bin/python3 -c "import MySQLdb"

4. Attempt to import MySQLdb in Salt's Python:

/opt/saltstack/salt/bin/python3.10 -c "import MySQLdb"

This fails with:

ImportError: /opt/saltstack/salt/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /lib/x86_64-linux-gnu/libmysqlclient.so.21)

5. Run:

salt-call sys.doc mysql --local

Returns empty output due to __virtual__ returning False.

** Expected Behavior **

The mysql module should load if MySQLdb is installed, using the system’s libstdc++.so.6 (which supports GLIBCXX_3.4.32) instead of Salt’s outdated libstdc++.so.6 (max GLIBCXX_3.4.29).

** Actual Behavior **

The mysql module fails to load because Salt’s Python environment uses an outdated libstdc++.so.6, causing MySQLdb import to fail.

** Environment **

• Salt Version: 3007.2
• OS: Ubuntu 24.04.2 LTS
• Python: System Python 3.1.2, Salt bundled Python 3.10.17
• MySQL Client: mysqlclient 1.4.6
• MySQL Server: MySQL 8.0
• Install Path: /opt/saltstack

** Workaround **

1. Rename Salt's libstdc++.so.6:

   mv /opt/saltstack/salt/lib/libstdc++.so.6 /opt/saltstack/salt/lib/libstdc++.so.6.bak
   

2. Clear cache and restart:

   rm -rf /var/cache/salt/minion/*
   systemctl restart salt-minion
   

** Suggested Fix **

• Update the bundled libstdc++.so.6 in /opt/saltstack/salt/lib to support GLIBCXX_3.4.32
• Prioritize system libraries over bundled ones
• Improve __virtual__ error messages to include import failure details.

Additional notes

The system's libstdc++.so.6 supports GLIBCXX_3.4.32. This affects states like mysql_user.present. I had a state file for the workaround, but it failed for various reasons so I removed it and replaced it by modifying my bootstrap script.

[welcome[bot]]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Salt’s Contributor Guide
• Join our Community Discord
• Salt Project YouTube channel
• Community Wiki

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[dehnert]

I'm seeing essentially the same issue, but with the virt module.

I get a slightly different import error, but looks like basically the same story -- a relevant system library doesn't like the bundled libstdc++.so.6:

# /opt/saltstack/salt/bin/python3.10
Python 3.10.17 (main, May 11 2025, 04:07:13) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libvirt
Traceback (most recent call last):
  File "/opt/saltstack/salt/extras-3.10/libvirt.py", line 16, in <module>
    import cygvirtmod as libvirtmod  # type: ignore
ModuleNotFoundError: No module named 'cygvirtmod'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/saltstack/salt/extras-3.10/libvirt.py", line 19, in <module>
    raise lib_e
  File "/opt/saltstack/salt/extras-3.10/libvirt.py", line 13, in <module>
    import libvirtmod  # type: ignore
ImportError: /opt/saltstack/salt/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libicuuc.so.74)

It looks like 3007.1 didn't bundle libstdc++:

# dpkg -L salt-common | grep std
/opt/saltstack/salt/include/readline/rlstdc.h
/opt/saltstack/salt/lib/python3.10/site-packages/pydantic/_internal/_std_types_schema.py
/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/listdiffer.py

whereas 3007.3 did:

# dpkg -L salt-common | grep std
/opt/saltstack/salt/include/readline/rlstdc.h
/opt/saltstack/salt/lib/libstdc++.so.6
/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/listdiffer.py

I briefly looked at the 3007.2 release notes and the git log and diff between 3007.1 and 3007.2, and I don't obvious see an intent or mechanism to start bundling libstdc++. Given that simply moving the file aside seems fix the issue, can whatever about the build process(?) results in bundling the file just get reverted?

Version report:

# salt-call -V
Salt Version:
          Salt: 3007.3
 
Python Version:
        Python: 3.10.17 (main, May 11 2025, 04:07:13) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.16.0
      cherrypy: 18.8.0
  cryptography: 42.0.5
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.6
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 24.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.19.2
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.4.2
           ZMQ: 4.3.4
 
Salt Package Information:
  Package Type: onedir
 
System Versions:
          dist: ubuntu 24.04.2 noble
        locale: utf-8
       machine: x86_64
       release: 6.8.0-58-generic
        system: Linux
       version: Ubuntu 24.04.2 noble

[dehnert]

I haven't tried installing it yet, but it looks like 3007.4 also has the problematic file (I haven't checked if it's a more-compatible version).

[rsolano-isb]

I am confirming that this bug is indeed still present on 3007.4 (Chlorine).

[anurag-bug]

It's still present on 3006.11.