How to use Python kernel in Polyglot Notebook?

[Regenhardt]

The package and version I'm asking about:

.NET Interactive

© 2020 Microsoft Corporation
Version: 1.0.461101+09b2630eed4bf45ad5c843caa3420ad14fbcd596
Library version: 1.0.0-beta.23611.1+09b2630eed4bf45ad5c843caa3420ad14fbcd596
Build date: 2023-12-19T16:00:34.1381822Z
https://github.com/dotnet/interactive

Question

I love the dib format for being great with git. The readme says Python even supports variable sharing.
How do I use python in a dib file? The language list in the bottom right corner of the cell doesn't contain python. I can only use an ipynb file with the python kernel, but the .NET Interactive kernel does not offer python to me.

I currently have .NET 8 and Python 3.11.4 installed.

Cell kernel list:

[anigo333]

I followed https://github.com/dotnet/interactive/blob/HEAD/docs/jupyter-in-polyglot-notebooks.md and it worked for me. If you have already tried that, may be try the following in the .dip file and see if then python shows up as a code option.

#!connect jupyter --kernel-name pythonkernel --kernel-spec python3 (do this in csharp)

[Regenhardt]

This indeed helped me fix an adjacent problem, terminal not recognizing python. Had to disable the python app execution aliases in windows settings.

Unfortunately this did not fix vscode not finding python.

Terminal:

> python --version
Python 3.11.7

VSCode dib file:

> #!connect jupyter --kernel-name pythonkernel --kernel-spec python3

The #!connect jupyter feature is in preview. Please report any feedback or issues at https://github.com/dotnet/interactive/issues/new/choose.

Error: System.ArgumentException: KernelSpec python3 not found
at Microsoft.DotNet.Interactive.Jupyter.ZMQ.JupyterConnection.CreateKernelConnectionAsync(String kernelSpecName) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive.Jupyter\ZMQ\JupyterConnection.cs:line 47
at Microsoft.DotNet.Interactive.Jupyter.JupyterKernelConnector.CreateKernelAsync(String kernelName) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive.Jupyter\JupyterKernelConnector.cs:line 27
at Microsoft.DotNet.Interactive.Jupyter.ConnectJupyterKernelCommand.ConnectKernelsAsync(KernelInvocationContext context, InvocationContext commandLineContext) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive.Jupyter\ConnectJupyterKernelCommand.cs:line 72
at Microsoft.DotNet.Interactive.CompositeKernel.<>c__DisplayClass21_0.<<AddKernelConnector>b__0>d.MoveNext() in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive\CompositeKernel.cs:line 271
--- End of stack trace from previous location ---
at System.CommandLine.NamingConventionBinder.CommandHandler.GetExitCodeAsync(Object returnValue, InvocationContext context)
at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass15_0.<<AddMiddleware>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass12_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass19_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Invocation.InvocationPipeline.<InvokeAsync>g__FullInvocationChainAsync|2_0(InvocationContext context)
at System.CommandLine.Parsing.ParseResultExtensions.InvokeAsync(ParseResult parseResult, IConsole console)
at Microsoft.DotNet.Interactive.Commands.DirectiveCommand.InvokeAsync(KernelInvocationContext context) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive\Commands\DirectiveCommand.cs:line 33
at Microsoft.DotNet.Interactive.Kernel.HandleAsync(KernelCommand command, KernelInvocationContext context) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive\Kernel.cs:line 330
at Microsoft.DotNet.Interactive.KernelCommandPipeline.<BuildPipeline>b__6_0(KernelCommand command, KernelInvocationContext context, KernelPipelineContinuation _) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 60
at Microsoft.DotNet.Interactive.KernelCommandPipeline.SendAsync(KernelCommand command, KernelInvocationContext context) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive\KernelCommandPipeline.cs:line 41

ms-toolsai.jupyter: v2023.11.1100101639

[colombod]

Are you using anaconda?

[Regenhardt]

No, no virtual environments. Haven't installed anaconda here.

[teodimache]

I have the same issue and I'm using miniconda which comes with a Python version 3.11.5. Also I don't have conda or python in the PATH env variable.

[colombod]

Doesn't mini onda come with default environment as root or base?

[teodimache]

it does yes, it's called "base", and it's not complaining about that, it's complaining about the KernelSpec: "KernelSpec python3 not found"

[colombod]

can you run jupyter kernelspec list and see what are the kernel specs and share the paths?

[teodimache]

Good question, you made me go down the rabbit hole and figured out i was missing the iPython package. Thanks! :) All is well now, I'm able to run Python code from the .NET Interactive kernel.

[colombod]

@Regenhardt what do you see if you run the jupyter kernelspec list command?

[Regenhardt]

I get jupyter : The term 'jupyter' is not recognized as the name of a cmdlet, function, script file, or operable program.[...].

I installed/upgraded the notebook package, pip list includes notebook 7.0.7 (previously it was 7.0.1).
python/pip work, ipynb works too. PATH contains both Python\ and Python\Scripts\. What am I missing?

[colombod]

I get jupyter : The term 'jupyter' is not recognized as the name of a cmdlet, function, script file, or operable program.[...].

I installed/upgraded the notebook package, pip list includes notebook 7.0.7 (previously it was 7.0.1). python/pip work, ipynb works too. PATH contains both Python\ and Python\Scripts\. What am I missing?

I usually install Jupyter using pip3 install jupyterlab can you try that? if the Jupyter command is not resolved it means Jupyter server is not installed and you don't have kernel spec available

[Regenhardt]

Doing that results in a whole lot of "Requirement already satisfied" messages, the first being:

Requirement already satisfied: jupyterlab in e:\programs\python\lib\site-packages (4.0.10)

[gabor-konstanzer]

I'm not sure of what I'm doing wrong here. I have the same issue. I am using VSCode. I have both the Jupyter and the Polyglot extensions. I have dotnet-interactive running with with .Net 8 Runtime and SDK

1. I installed miniconda3
2. I created a conda env called "ai_course"
3. I installed jupyter, ipykernel, IPython
4. I installed ipykernel for --user with --name=ai_course And when I try to connect like this in the Notebook:

#!connect jupyter --kernel-name pythonkernel --conda-env ai_course --kernel-spec ai_course

I get this with a long stacktrace

Error: System.ArgumentException: KernelSpec ai_course not found
at Microsoft.DotNet.Interactive.Jupyter.ZMQ.JupyterConnection.CreateKernelConnectionAsync(String kernelSpecName) in D:\a\_work\1\s\src\Microsoft.DotNet.Interactive.Jupyter\ZMQ\JupyterConnection.cs:line 47

On a conda info I get the following:

     active environment : ai_course
    active env location : /Users/gabor.konstanzer/miniconda3/envs/ai_course
            shell level : 1
       user config file : /Users/gabor.konstanzer/.condarc
 populated config files :
          conda version : 24.1.2
    conda-build version : not installed
         python version : 3.12.1.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.1.2=0
                          __osx=13.5.2=0
                          __unix=0=0
       base environment : /Users/gabor.konstanzer/miniconda3  (writable)
      conda av data dir : /Users/gabor.konstanzer/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/gabor.konstanzer/miniconda3/pkgs
                          /Users/gabor.konstanzer/.conda/pkgs
       envs directories : /Users/gabor.konstanzer/miniconda3/envs
                          /Users/gabor.konstanzer/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.1.2 requests/2.31.0 CPython/3.12.1 Darwin/22.6.0 OSX/13.5.2 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.3
                UID:GID : 503:20
             netrc file : None
           offline mode : False

On jupyter kernelspec list I get the following: (I ran the dotnet-interactive jupyter install once)

Available kernels:
  python3            /Users/gabor.konstanzer/miniconda3/envs/ai_course/share/jupyter/kernels/python3
  .net-csharp        /Users/gabor.konstanzer/Library/Jupyter/kernels/.net-csharp
  .net-fsharp        /Users/gabor.konstanzer/Library/Jupyter/kernels/.net-fsharp
  .net-powershell    /Users/gabor.konstanzer/Library/Jupyter/kernels/.net-powershell
  ai_course          /Users/gabor.konstanzer/Library/Jupyter/kernels/ai_course

VSCode finds the ai_course kernel. I can run python with it in another notebook, It just doesn't appear as a language when set to .NET Interactive kernel, nor can it connect with the magic connect command. Any Idea what could be missing?

[webteckie]

I just went thru solving this error...for me I had to ensure the following:

• if you have installed jupyter but it doesn't run in the terminal then you're almost there...all I had to do was:

1. ensure the PATH points to the Scripts directory, e.g., %localappdata%\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts
2. ensure the PATHEXT accounts for the .PY extension -- add ;.PY

• if you have not installed jupyter and jupyterlab then ensure that's done first. You can tell if you do not have these installed by doing something like:

1. pip show jupyter
2. pip show jupyterlab

• to install these you can use something like:

1. pip install jupyter
2. pip install jupyterlab

• it's possible that other things may be needed as well (e.g., I installed R but not sure I needed that)

Hope that helps!

[colombod]

@webteckie thank you for such a detailed fix! This will be useful for other users too.

[Regenhardt]

It works now! Not sure what changed, but I did get a new machine and installed everything from scratch. Did the same as before though as ar as I know: Install vscode extensions, install python, uninstall python 3.12 because I need 3.11, install python 3.11 so I can do my ai stuff. PATH and PATHEXT were properly set by the python installer.

Ran #!connect jupyter --kernel-name pythonkernel --kernel-spec python3 and it works.

Just don't have variable sharing between C# and Python now though, any idea what's going on there?