Python virtual environment

[gswifort]

Hi, I have a question: at what point does PyRx decide on the choice of a specific python interpreter? During installation or when loading into AutoCAD? I would like to run Autocad from a python virtual environment with a separate interpreter (e.g. .\venv\Scripts\python.exe), but PyRx in Autocad is always set to the global interpreter.

[CEXT-Dan]

Hi,

Currently, nothing is implemented to handle virtual environments. Each instance of AutoCAD fires up a new interpreter by calling Py_Initialize(), The module loads python312.dll into the AutoCAD process, python.exe is never called

I’m not entirely sure how to handle virtual environments in an embedded context, but I will look into this

[gswifort]

Thanks for the answer. Yet on another topic. I haven't found an example of how to open a new empty document, add elements to it and save it. On various attempts I received:

RuntimeError:
Exception! (eWrongDatabase) in function appendAcDbEntity ,Line 2322, File PyDbSymbolTableRecord.cpp

[CEXT-Dan]

Hi do you mean a new .DWG, as in a side database? Have a look at this sample

import traceback
import PyRx as Rx
import PyGe as Ge
import PyGi as Gi
import PyDb as Db
import PyAp as Ap
import PyEd as Ed

#scope so everything is closed
def processDb(db : Db.Database):
    line = Db.Line(Ge.Point3d(0,0,0),Ge.Point3d(100,100,0))
    line.setDatabaseDefaults(db)
    model = Db.BlockTableRecord(db.modelSpaceId(), Db.OpenMode.kForWrite)
    model.appendAcDbEntity(line)
    
def PyRxCmd_doit() -> None:
    try:
        sideDb = Db.Database()
        processDb(sideDb)
        sideDb.saveAs("e:\\newdwg.dwg")
    except Exception as err:
        traceback.print_exception(err)

[gswifort]

Yes, that was it, thank you!

Coming back to the virtual environment, python312.dll can always be the same, so it all comes down to loading <venv>\Lib\site-packages into sys.path instead of %localappdata%\programs\python\python312\Lib\site-packages or at least site-packages from a virtual environment before global ones

[gswifort]

I think I found the solution, just set PYTHONPATH to <venv>\Lib\site-packages

[CEXT-Dan]

Great!

Note: I’m not happy with the current installer, I don’t want to have to write anything to the registry. So, I plan on writing a loader module that sets AutoCAD’s local ENV to the DLL paths, then loads the python wrapper module.

This loader module will expose PyConfig, https://docs.python.org/3/c-api/init_config.html#init-path-config , I’m not sure the format yet, maybe an .INI or XML configuration file that sits in the same folder.

This is some time off though, as I’m still writing wrappers

[gswifort]

Yes, this should also solve the virtual environment issue (take into account the VIRTUAL_ENV environment variable). Also keep in mind that AutoCAD can be launched using the COM interface (win32com.client.Dispatch), and so far I have encountered a problem where AutoCAD does not see the modules located in the PyRxStubs folder (I also solved this by adding to PYTHONPATH, but this is a bit of a monkey-patching solution)

[schoeller]

@gswifort I have set up venv with launch.json as below:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Remote Attach",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "127.0.0.1",
            "justMyCode": false,
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "."
                }
            ],
            "env": {
                "PYTHONPATH": "$PYTHONPATH:C:/ProgramData/Autodesk/ApplicationPlugins/PyRx.bundle/Contents/PyRxStubs" 
            },
        }
    ]
}

VS Code states that env property is not allowed. I have tried adding PYTHONPATH="C:/ProgramData/Autodesk/ApplicationPlugins/PyRx.bundle/Contents/PyRxStubs" to the .env file. I still receive below error

Any hints?

Best regards

Sebastian

[gswifort]

I have the following debugging settings:

launch.json

{
    "name": "Autocad attach",
    "type": "debugpy",
    "request": "attach",
    "connect": {
        "host": "127.0.0.1",
        "port": 5678
    },
    "justMyCode": false
}

settings.json

{
    "python.envFile": "${workspaceFolder}/.env"
}

.env

PYTHONPATH = c:\ProgramData\Autodesk\ApplicationPlugins\PyRx.bundle\Contents\PyRxStubs

However, the error you show will not be resolved by setting the PYTHONPATH to the "stubs" folder. The "stubs" folder contains only *.pyi interfaces and Pyxx modules (containing the source code) are only available in the AutoCAD environment.

[CEXT-Dan]

Not quite sure how to deal with that vscode warning. The problem is PyLance can’t see python embedded inside C++ Those modules PyRx, PyGe etc.. are inside the .ARX file I’ve semi-resolved the issue by hiding it, see issue #3

[gswifort]

If the problem is only the display of errors, you can simply ignore them, as follows:

import PyRx as Rx  # type:ignore
import PyGe as Ge  # type:ignore
import PyGi as Gi  # type:ignore
import PyDb as Db  # type:ignore
import PyAp as Ap  # type:ignore
import PyEd as Ed  # type:ignore

If you use isort then also:

import PyRx as Rx  # isort:skip # type:ignore
import PyGe as Ge  # isort:skip # type:ignore
import PyGi as Gi  # isort:skip # type:ignore
import PyDb as Db  # isort:skip # type:ignore
import PyAp as Ap  # isort:skip # type:ignore
import PyEd as Ed  # isort:skip # type:ignore

[schoeller]

Thanks for the work. My current environment foresees the following working steps

1#Start BCAD and VS Code in parallel 2#Load script in BCAD manually via _PYLOAD 3#Start pydebug and call the remote debugger in VS Code 4#Alter and save script and restart from 2# via _PYLOAD (hopefully not killing BCAD)

Thus all modules must be installed accesible to the interpreter in BCAD (which is system-wide Python). VDEV is only available to VS Code. Is there any chance that I may have missed something and may directly launch and debug from within VS Code or reorganize my Python installation?

Best

Sebastian

[gswifort]

For BricsCad to see the virtual environment (installed packages), the directory <virtualenv>\Lib\site-packages must be added to sys.path To do this you have two options:

1. Set PYTHONPATH:

# <virtualenv>\Scripts\activate.bat
...
set PYTHONPATH=%VIRTUAL_ENV%\Lib\site-packages;%PYTHONPATH%

and run BricsCad from the console with the virtual environment activated (for Autocad it is acad, for BricsCad probably bcad). For me, when I run Autocad from a virtual environment, the embedded Python does not see the PyRxStubs directory, so it also needs to be added to PYTHONPATH (You probably won't need this if you have the global PYTHONPATH variable set correctly - it is set during installation):

# <virtualenv>\Scripts\activate.bat
...
set PYTHONPATH=%VIRTUAL_ENV%\Lib\site-packages;C:\ProgramData\Bricscad\ApplicationPlugins\PyRx.bundle\Contents\PyRxStubs;%PYTHONPATH% 
# check if the path is correct, I only replaced Autocad → Bricscad.

2. Add a path from the BCAD console:

>>> PYCMDPROMPT
>>> import sys
>>> sys.path.insert(0, ".\<virtualenv>\Lib\site-packages")

[CEXT-Dan]

Some ideas:

-create loader modules for each host application. -The loader module will replace registry entries by adding env paths at runtime. -The loader module will be able to read a human editable configuration file so users can customize it.

-the installer will install main bulk of the package, i.e. wrappers, stubs, samples Users*username*\AppData\Local\Programs\PyRx

-Only the AutoCAD loader modules and PackageContents.xml will be installed in ApplicationPlugins -The loader modules for the clones will be installed in the PyRx folder where they can be added to a startup suite

• or maybe this can be a separate thing

[gswifort]

What do you mean by loader module?

[CEXT-Dan]

Another .ARX module that loads into Autocad, I'll need to hook into Autocads DLL paths and add the necessary paths

[CEXT-Dan]

Hi, Can you test if v1.3.002 is enough to solve this task?

I’ve added these functions: PyPreConfig_InitPythonConfig PyConfig_InitPythonConfig

But I don’t quite understand how they are used, but the goal is to be able to add configurations to the INI, that I can pass along. But I don’t know which ones you might need

[gswifort]

Hi, Wednesday at the earliest.

[gswifort]

Hi, I don't know how to use what you did. From what I see, I can use PyRx.ini, but only globally, this file is not searched in the current directory (virtual environment). For the virtual environment to work properly, it is necessary to adopt a different installation path scheme, as python.exe does in the virtual environment (python3x.dll does not set paths in the same way as python.exe, it has to be done manually). issue22213 and PEP 587 should be helpful.

[CEXT-Dan]

Yeah, It clear I don’t understand how all this works.

“From what I see, I can use PyRx.ini, but only globally, this file is not searched in the current directory (virtual environment).”

It’s not meant to be, the design is to tell AutoCAD the path to the DLLs. And to pass information to Py_PreInitialize and Py_InitializeFromConfig, I just don’t have a clue what parameters are needed

How about we start with: PYTHONISOLATED = 1 PYTHONEXECUTABLE = C:\path\to\venv\Scripts\python.exe

I’ll pass these to Py_InitializeFromConfig

[gswifort]

Yes, it is possible that this will be enough. However, you should probably pay attention to one more thing - which python3x.dll file to use. When creating a virtual environment, a pyvenv.cfg file is created in the %VIRTUAL_ENV% directory, containing, among others:

home = C:\Users\gswi\AppData\Local\Programs\Python\Python312

use the file located in this directory (we assume that the user may have different versions of python installed).

[schoeller]

@gswifort is this design platform-independant?

[gswifort]

I don't have access to macOS to test, but it seems so. PEP 587

[gswifort]

Analyzing the python venv module also shows that it is platform-independent

[gswifort]

And to determine whether to use an "isolated" configuration, I would check if the VIRTUAL_ENV environment variable exists.

[CEXT-Dan]

I’m preparing a build now; my thought is you must explicitly set PYTHONISOLATED in the INI I don’t want this to automatic, at least not yet

[gswifort]

I installed the latest version (1.3.006), but I don't know how to configure the .ini file. I tested in various ways, but it does not see the virtual environment. This is what the python.exe configuration looks like in the virtual environment:

(venv) K:\Programy\demos\pyrx_demo>python -m sysconfig
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "venv"

Paths:
        data = "K:\Programy\demos\pyrx_demo\venv"
        include = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
        platinclude = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
        platlib = "K:\Programy\demos\pyrx_demo\venv\Lib\site-packages"
        platstdlib = "K:\Programy\demos\pyrx_demo\venv\Lib"
        purelib = "K:\Programy\demos\pyrx_demo\venv\Lib\site-packages"
        scripts = "K:\Programy\demos\pyrx_demo\venv\Scripts"
        stdlib = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Lib"

Variables:
        BINDIR = "\\pc-135\Users\GSWI\Documents\Programy\demos\pyrx_demo\venv\Scripts"
        BINLIBDEST = "K:\Programy\demos\pyrx_demo\venv\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".cp312-win_amd64.pyd"
        INCLUDEPY = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Include"
        LIBDEST = "C:\Users\gswi\AppData\Local\Programs\Python\Python312\Lib"
        TZPATH = ""
        VERSION = "312"
        VPATH = "..\.."
        abiflags = ""
        base = "K:\Programy\demos\pyrx_demo\venv"
        exec_prefix = "K:\Programy\demos\pyrx_demo\venv"
        installed_base = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        installed_platbase = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        platbase = "K:\Programy\demos\pyrx_demo\venv"
        platlibdir = "DLLs"
        prefix = "K:\Programy\demos\pyrx_demo\venv"
        projectbase = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        py_version = "3.12.2"
        py_version_nodot = "312"
        py_version_nodot_plat = "312"
        py_version_short = "3.12"
        srcdir = "C:\Users\gswi\AppData\Local\Programs\Python\Python312"
        userbase = "C:\Users\gswi\AppData\Roaming\Python"

and like this in autocad:

>>>: import sysconfig
>>>: sysconfig._main()
Platform: "win-amd64"
Python version: "3.12"
Current installation scheme: "nt"

Paths:
 data = "c:\users\gswi\appdata\local\programs\python\python312"
 include = "c:\users\gswi\appdata\local\programs\python\python312\Include"
 platinclude = "c:\users\gswi\appdata\local\programs\python\python312\Include"
 platlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib\site-packages"
 platstdlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
 purelib = "c:\users\gswi\appdata\local\programs\python\python312\Lib\site-packages"
 scripts = "c:\users\gswi\appdata\local\programs\python\python312\Scripts"
 stdlib = "c:\users\gswi\appdata\local\programs\python\python312\Lib"

Variables:
 BINDIR = "C:\Program Files\Autodesk\AutoCAD 2022"
 BINLIBDEST = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
 EXE = ".exe"
 EXT_SUFFIX = ".cp312-win_amd64.pyd"
 INCLUDEPY = "c:\users\gswi\appdata\local\programs\python\python312\Include"
 LIBDEST = "c:\users\gswi\appdata\local\programs\python\python312\Lib"
 TZPATH = ""
 VERSION = "312"
 VPATH = "..\.."
 abiflags = ""
 base = "c:\users\gswi\appdata\local\programs\python\python312"
 exec_prefix = "c:\users\gswi\appdata\local\programs\python\python312"
 installed_base = "c:\users\gswi\appdata\local\programs\python\python312"
 installed_platbase = "c:\users\gswi\appdata\local\programs\python\python312"
 platbase = "c:\users\gswi\appdata\local\programs\python\python312"
 platlibdir = "DLLs"
 prefix = "c:\users\gswi\appdata\local\programs\python\python312"
 projectbase = "C:\Program Files\Autodesk\AutoCAD 2022"
 py_version = "3.12.2"
 py_version_nodot = "312"
 py_version_nodot_plat = "312"
 py_version_short = "3.12"
 srcdir = "C:\Program Files\Autodesk\AutoCAD 2022"
 userbase = "C:\Users\gswi\AppData\Roaming\Python"

[CEXT-Dan]

Just of thought Instead of using the pycomand, try using the pyrx_onload.py in the bin directory

it loads early, so you might have to use a command or

import sysconfig

def OnPyLoadDwg(): print(sysconfig._main())

[CEXT-Dan]

I'll try to set one up over the next week and try

[gswifort]

Calling from the command level changes nothing.

The expected behavior is:

• create and activate a virtual environment:

> python -m venv venv
> venv\Scripts\activate.bat

• run AutoCAD from the virtual environment:

> acad

• load the module:

import PyRx as Rx # isort:skip
import PyGe as Ge # isort:skip
import PyGi as Gi # isort:skip
import PyDb as Db # isort:skip
import PyAp as Ap # isort:skip
import PyEd as Ed # isort:skip


def PyRxCmd_doit():
     import sys
     print(*sys.path, sep="\n")

you should get:

...\AppData\Local\Programs\Python\Python312\python312.zip
...\AppData\Local\Programs\Python\Python312\DLLs
...\AppData\Local\Programs\Python\Python312\Lib
...\AppData\Local\Programs\Python\Python312
...\venv                    # !!!
...\venv\Lib\site-packages  # !!!

Note that one .ini file is not a good solution because we want to have different directories in sys.path depending on the environment we run autocad from.