ecmwf
Launch metview in Jupyter when installed in different environments
I have Jupyter Notebooks installed in my base environment. When creating/using separate environments, I use ipykernel to make them accessible as Jupyter kernels. I recently installed metview into an existing conda environment (entitled "monsoon-pr") following:
conda install 'metview-batch>=5.16.0' -c conda-forge
When launching a new notebook, activating the environment where metview is installed (via the Kernel dropdown), and attempting to import metview following import metview as mv, I get the following error:
Could not run the Metview executable ('metview'); check that the binaries for Metview (version 5 at least) are installed and are in the PATH.
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
Cell In[1], line 1
----> 1 import metview as mv
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/__init__.py:44
42 except Exception as exp:
43 if "METVIEW_PYTHON_ONLY" not in os.environ:
---> 44 raise exp
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/__init__.py:28
25 if len(sys.argv) != 2 or sys.argv[0] != "-m" or sys.argv[1] != "selfcheck":
27 try:
---> 28 from . import bindings as _bindings
30 _bindings.bind_functions(globals(), module_name=__name__)
32 # Remove "_bindings" from the public API.
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/bindings.py:196
192 signal.signal(signal.SIGTERM, self.sigterm)
193 signal.signal(signal.SIGALRM, self.sigalarm)
--> 196 mi = MetviewInvoker()
198 try:
199 ffi = cffi.FFI()
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/bindings.py:114, in MetviewInvoker.__init__(self)
106 if "METVIEW_PYTHON_ONLY" not in os.environ:
107 print(
108 "Could not run the Metview executable ('"
109 + metview_startup_cmd
(...)
112 "and are in the PATH."
113 )
--> 114 raise exp
116 # wait for Metview to respond...
117 wait_start = time.time()
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/bindings.py:104, in MetviewInvoker.__init__(self)
101 print(metview_flags)
103 try:
--> 104 subprocess.Popen(metview_flags)
105 except Exception as exp: # pragma: no cover
106 if "METVIEW_PYTHON_ONLY" not in os.environ:
File ~/Miniconda/envs/metview/lib/python3.9/subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
947 if self.text_mode:
948 self.stderr = io.TextIOWrapper(self.stderr,
949 encoding=encoding, errors=errors)
--> 951 self._execute_child(args, executable, preexec_fn, close_fds,
952 pass_fds, cwd, env,
953 startupinfo, creationflags, shell,
954 p2cread, p2cwrite,
955 c2pread, c2pwrite,
956 errread, errwrite,
957 restore_signals,
958 gid, gids, uid, umask,
959 start_new_session)
960 except:
961 # Cleanup if the child failed starting.
962 for f in filter(None, (self.stdin, self.stdout, self.stderr)):
File ~/Miniconda/envs/metview/lib/python3.9/subprocess.py:1821, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session)
1819 if errno_num != 0:
1820 err_msg = os.strerror(errno_num)
-> 1821 raise child_exception_type(errno_num, err_msg, err_filename)
1822 raise child_exception_type(err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'metview'
I'm assuming this error is because metview is operating out of a different environment than Jupyter. In my Jupyter notebook, os.environ['PATH'] gives my Jupyter path as:
'/jet/home/sferrett/.local/bin:/jet/home/sferrett/bin:/opt/packages/psc.allocations.user/bin:/opt/packages/allocations/bin:/jet/home/sferrett/Miniconda/bin:/jet/home/sferrett/Miniconda/condabin:/jet/home/sferrett/.local/bin:/jet/home/sferrett/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/packages/interact/bin:/opt/puppetlabs/bin'
While in my conda environment with metview (monsoon-pr) in Terminal, my path is:
/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin:/jet/home/sferrett/Miniconda/condabin:/jet/home/sferrett/.local/bin:/jet/home/sferrett/bin:/opt/packages/psc.allocations.user/bin:/opt/packages/allocations/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/packages/interact/bin:/opt/puppetlabs/bin
Is there a way to set the notebook PATH so that it can retrieve metview from the conda environment? I don't want to have to install Jupyter within my conda environment.
Hi @savannahferretti,
I'm surprised it can't find Metview, but one thing you can do is, from a terminal where Metview can be found, type which metview. This will give the full path to the Metview startup script. You can then set environment variable METVIEW_PYTHON_START_CMD to this path (including the 'metview' at the end), either on the command line before starting Jupyter, or you can set this env var from the Python code in your Jupyter notebook before you import metview.
Good luck!
Cheers, Iain
Thanks for the quick response! I am stilling having an issue loading the package. I am unsure if it has to do with the fact that I'm SSH tunneling into Jupyter Notebooks from an HPC. Here is what I did:
In my first Terminal window, I SSHed into the HPC, was allocated a compute node, and launched Jupyter Notebooks. I opened a second Terminal window, mapped the port to my local, then launched Jupyter in my browser. Also from this second window, I activated the environment where metview is installed and executed which metview, which returned:
/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin/metview
I deactivated the environment, and from my base environment (where Jupyter Notebooks is installed), I executed:
export METVIEW_PYTHON_START_CMD=/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin/metview
Executing printenv METVIEW_PYTHON_START_CMD returned the same path.
After launching my notebook and running import metview as mv, I still get an error:
ModuleNotFoundError: No module named 'metview'
I tried relaunching Jupyter, but still the same error.
Hi,
OK, this should help: first, to make sure the environment inside your notebook has the right path set, you can set the env var before importing metview, and to also get more debug output:
import os
os.environ["METVIEW_PYTHON_START_CMD "] = "/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin/metview"
os.environ["METVIEW_PYTHON_DEBUG"] = "1"
import metview
Hi,
Thanks for the response. Trying the suggestion above still gives the same error (i.e., it cannot find metview):
import os
os.environ['METVIEW_PYTHON_START_CMD'] = '/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin/metview'
os.environ['METVIEW_PYTHON_DEBUG'] = '1'
import metview
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[2], line 4
2 os.environ['METVIEW_PYTHON_START_CMD'] = '/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin/metview'
3 os.environ['METVIEW_PYTHON_DEBUG'] = '1'
----> 4 import metview
ModuleNotFoundError: No module named 'metview'
Hi Savannah,
Looks like this one dropped off the radar...! Did you manage to fix this issue? It looks like you were in an environment where Metview's Python interface was not installed.
Cheers, Iain