MHKiT-Software
MHKiT with MATLAB R2022b , Anaconda, Python 3.9, on Ubuntu 22.04 LTS
I'm trying to get MHKiT working with MATLAB R2022b and Anaconda with Python 3.9 on Ubuntu 22.04 LTS, but I'm having some problems.
I cloned both MHKiT-Python and MHKiT-MATLAB and then installed them both (i.e., mhkit and mhkit-python-utils) into my Anaconda environment using pip install -e .
Then I start MATLAB R2022b and installed the MHKiT toolbox which reports success, however, the following does not work in MATLAB:
[x,y]=circular(30)
Error using _linprog_highs><module>
Python Error: ImportError: /opt/MATLAB/R2022b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
~/anaconda3/lib/python3.9/site-packages/scipy/optimize/_highs/_highs_wrapper.cpython-39-x86_64-linux-gnu.so)
I have GLIBCXX_3.4.29 in ~/anaconda3/lib/libstdc++.so.6 but GLIBCXX_3.4.28 is highest version in
/opt/MATLAB/R2022b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6
If I try to tell MATLAB to use ~/anaconda3/lib/libstdc++.so.6 by doing
LD_PRELOAD=/home/ps/anaconda3/lib/libstdc++.so.6 /opt/MATLAB/R2022b/bin/matlab
MATLAB crashes when I run the circular(30) command.
I also tried to make the pip install build against
/opt/MATLAB/R2022b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6
by uninstalling mhkit, mhkit-python-utils and the MHKit toolbox and repeating the whole process above after first setting
export LD_PRELOAD=/opt/MATLAB/R2022b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6
However, this made no difference to the error message above.
I'd welcome any suggestions on how to fix the problem or what to try next, thanks.
Hi @pstansell, thank you for the detailed issue report.
You did everything right. We have seen similar failures with our unit tests and the root cause seems to be issues between the MATLAB, Python, and system path.
Could you try the following troubleshooting steps:
- Close MATLAB
- Create a new anaconda environment
- In this environment:
- Install C based python packages with
conda-
conda install numpy cython pip pytest hdf5 libnetcdf cftime netcdf4
-
- Install the previous version of MHKiT-Python
-
pip install mhkit==0.7.0
-
- Install the latest version of
mhkit_python_utils-
pip install -e .inside of the MHKiT-MATLAB folder
-
- Verify that mhkit-python is working:
-
python -c "import mhkit; print(mhkit.__version__)" - Output should
v0.7.0
- Install the MHKiT-MATLAB add-on
- Restart MATLAB
- Double click on
mhkit.mltbx - Verify the version is 0.4.1 in your MATLAB add-ons
- Find your conda environment python executable:
-
python -c "import sys; print(sys.executable)
- Setup MATLAB python to use python out of process
- In MATLAB Command Window:
-
pyenv(Version="<path to python executable>", ExecutionMode="OutOfProcess")
-
- Test MHKiT-MATLAB
- In MATLAB Command Window:
-
[x, y] = circular(30)
-
Expected output is (or similar, not sure why my MATLAB is using exponential notation here)
>> [x, y] = circular(30)
x =
30
y =
7.068583470577034e+02
In MATLAB using Python OutOfProcess seems to fix some MATLAB <-> Python issues related to library access.
Let us know if this is helpful!
Hello @simmsa,
Thanks for your detailed instructions on what to try. I followed them carefully, but it didn't seem to make any difference to the results I had before. Following is a summary of what I did and the outputs and errors.
Created and activated a new (Python 3.9) Anaconda environment with
bash Anaconda3-2022.10-Linux-x86_64.sh
eval "$(/home/ps/anaconda3/bin/conda shell.bash hook)"
Installed packages with
conda install numpy cython pip pytest hdf5 libnetcdf cftime netcdf4
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/ps/anaconda3
added / updated specs:
- cftime
- cython
- hdf5
- libnetcdf
- netcdf4
- numpy
- pip
- pytest
The following packages will be downloaded:
package | build
---------------------------|-----------------
cftime-1.6.2 | py39h7deecbd_0 263 KB
hdf4-4.2.13 | h3ca952b_2 714 KB
libnetcdf-4.8.1 | h42ceab0_1 1.0 MB
libzip-1.8.0 | h5cef20c_0 102 KB
netcdf4-1.6.2 | py39h89d13dc_0 505 KB
------------------------------------------------------------
The following NEW packages will be INSTALLED:
cftime pkgs/main/linux-64::cftime-1.6.2-py39h7deecbd_0 None
hdf4 pkgs/main/linux-64::hdf4-4.2.13-h3ca952b_2 None
libnetcdf pkgs/main/linux-64::libnetcdf-4.8.1-h42ceab0_1 None
libzip pkgs/main/linux-64::libzip-1.8.0-h5cef20c_0 None
netcdf4 pkgs/main/linux-64::netcdf4-1.6.2-py39h89d13dc_0 None
Proceed ([y]/n)? y
Installed MHKit with
pip install mhkit==0.7.0
Successfully installed NREL-rex-0.2.79 adal-1.2.7 azure-core-1.30.1 cachetools-4.2.4 fatpack-0.7.7 google-api-core-2.10.2 google-api-python-client-2.129.0 google-auth-1.35.0 google-auth-httplib2-0.2.0 google-auth-oauthlib-0.5.3 googleapis-common-protos-1.63.0 h5pyd-0.10.3 httplib2-0.22.0 isodate-0.6.1 mhkit-0.7.0 msrest-0.7.1 msrestazure-0.6.4.post1 netCDF4-1.5.8 oauthlib-3.2.2 pecos-0.3.1 protobuf-4.25.3 requests-oauthlib-2.0.0 requests-unixsocket-0.3.0 rsa-4.9 typing-extensions-4.11.0 uritemplate-4.1.1
Installed mhkit-python-utils with
cd ~/git_clones/MHKiT-MATLAB
pip install -e .
Successfully installed mhkit-python-utils-0.2.0
Checked MHKiT version and path with
python -c "import mhkit; print(mhkit.__version__)"
v0.7.0
python -c "import sys; print(sys.executable)"
/home/ps/anaconda3/bin/python
Started MATLAB with
/opt/MATLAB/R2022b/bin/matlab
I reinstalled the MTKiT toolbox and then I ran MATLAB commands
pyenv(Version="/home/ps/anaconda3/bin/python", ExecutionMode="OutOfProcess")
ans =
PythonEnvironment with properties:
Version: "3.9"
Executable: "/home/ps/anaconda3/bin/python"
Library: "/home/ps/anaconda3/lib/libpython3.9.so"
Home: "/home/ps/anaconda3"
Status: NotLoaded
ExecutionMode: OutOfProcess
>> [x, y] = circular(30)
Error using _linprog_highs><module>
Python Error: ImportError:
/opt/MATLAB/R2022b/interprocess/bin/glnxa64/pycli/../../../../sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/ps/anaconda3/lib/python3.9/site-packages/scipy/optimize/_highs/_highs_wrapper.cpython-39-x86_64-linux-gnu.so)
@pstansell, thank you for the detailed output. Looking further into your specific error it could be due to an older version of pandas that we are using. This issue suggests that running pip install --upgrade pandas in your mhkit conda environment may fix your issue. If that does not work I suggest trying the fixes suggested here next.
@pstansell, this Answer from Mathworks also looks like a promising solution. These types of MATLAB/Python errors are frustrating, but IMO you are very close to getting this working.
We do have tests on ubuntu verifying that you should be able to install MHKiT-MATLAB. They are based on this GitHub Actions workflow file. There may be some commands there that may help you troubleshoot this further.
@simmsa, based on the various suggestions, I've found a solution that works with MATLAB R2022b (but not with the latest MATLAB version R2024a). Here is a summary.
Install Anaconda with
bash Anaconda3-2022.10-Linux-x86_64.sh
Activate the Anaconda environment and install some packages with
conda install cftime hdf4 libnetcdf libzip netcdf4
Install MHKiT with
pip install mhkit==0.7.0
Install MHKiT-MATLAB with
git clone https://github.com/MHKiT-Software/MHKiT-MATLAB
cd MHKiT-MATLAB
pip install -e .
Change a MALTAB library link with
cd /opt/MATLAB/R2022b/sys/os/glnxa64
mv libstdc++.so.6 libstdc++.so.6_original
ln -s /home/ps/anaconda3/lib/libstdc++.so.6.0.29 libstdc++.so.6
Start MATLAB with
/opt/MATLAB/R2022b/bin/matlab
In MATLAB, install the MHKiT toolbox MHKiT-MATLAB//mhkit.mltbx and then run the MHKiT test command
>> [x, y] = circular(30)
/home/ps/anaconda3/lib/python3.9/site-packages/mhkit/tidal/resource.py:96: SyntaxWarning: assertion is always true, perhaps remove parentheses?
assert(all(directions>=0) and all(directions<=360),
x =
30
y =
706.8583
I'd previously tried something very similar with MATLAB R2024a, but it didn't work. I tried again with exactly the same steps as above but MATLAB R2024a crashed when I ran the circular with the following error
>> [x,y]=circular(30)
/home/ps/anaconda3/lib/python3.9/site-packages/h5py/__init__.py:36: UserWarning: h5py is running against HDF5 1.8.12 when it was built against 1.10.6, this may cause problems
_warn(("h5py is running against HDF5 {0} when it was built against {1}, "
@pstansell, nice work finding a solution! We appreciate your detailed troubleshooting efforts, and we are planning to incorporate your findings into an updated version of the MHKiT-MATLAB installation instructions.
We appreciate you bring this issue to our attention and working through it. To ensure the accuracy of MHKiT's output, we rely on MHKiT-Python as the reference for all computations, calling its functions from within MATLAB. As a result, to use MHKiT-MATLAB users must install MHKiT-Python and understand how to correctly configure MATLAB to use Python. This can be a barrier to entry for using MHKiT-MATLAB. If you have any thoughts or suggestions on how we can streamline this process, we would greatly value your feedback.
A more robust potential solution is manually adding the Anaconda executable directory to the first position MATLAB path. Internally, it seems like MATLAB adjusts the path to include MATLAB-specific binaries. This solution adjusts the path to put the Anaconda binaries first, allowing Python to correctly find the prebuilt binaries. Although you're not obligated to try this, as the first Linux user to report an issue, your input could help us devise a robust solution for all Linux users.
Below are the steps to add the Anaconda Python binaries to the MATLAB path:
-
Close MATLAB.
-
Locate the Anaconda environment bin directory:
Inside your MHKiT-Python Anaconda environment, in the terminal, determine the location of the Python binary directory by running:
python -c "import sys; import os; print(os.path.dirname(sys.executable))" -
Restart MATLAB and add the output directory from the previous step to the first position in the MATLAB path:
setenv('path', ['<path to Anaconda dir>', getenv('path')]) -
Reset the MATLAB Python Environment:
Inside your MHKiT-Python Anaconda environment, in the terminal, determine the location of the Python executable by running:
python -c "import sys; print(sys.executable)"In the MATLAB Command Window, execute:
pyenv(Version="<path to Anaconda Python executable>", ExecutionMode="OutOfProcess") -
Run the circular example again.