Feature Request: publish package on a private pypi server?

[deven-gqc]

Hello, my organization uses nbdev extensively. To use the packages in Colab, we need to clone and install the repo each time we wish to use the package that we've developed in Colab. nbdev_release_pypi by default publishes the package to the public pypi server, is there any plan to add a feature to publish the package to a private pypi server?

maybe additional params can be added in the settings.ini to make this possible? Any thoughts on this? @hamelsmu @seeM ?

[hamelsmu]

I would love to help on this, but at the moment I have a bit too much on my plate

[seeM]

Is it not possible to pip install directly from GitHub? Like so (replacing org and repo):

pip install git+https://github.com/org/repo

I believe this syntax even works in requirements files. You can also pin to a specific commit, branch, or tag (see here).

[deven-gqc]

@seeM This solution becomes problematic if the repo is private. If the repo is private, I can use

pip install git+ssh://..

but it has the assumption that the SSH keys are setup. I've searched a lot to see if its possible to clone a repo with a PAT and https, I couldn't find anything that would work.

We use a lot of Colab and it is not feasible to add SSH keys each time we start a nb, as an alternative to that I have the following workflow

1. mount drive (this assumes that I've cloned that repo on my drive already)
2. cd to the repo
3. pip install -e .
4. use the repo as expected

These steps would get reduced to directly a single step, if I could download the package directly from a private pypi server.

[Elijas]

Option 1. Doing it without nbdev

How to publish a package to a private pypi server:

test -f setup.py && rm -rf dist build *.egg-info .eggs
python setup.py sdist bdist_wheel
twine upload --repository-url <URL> --username <PYPI_USERNAME> --password <PYPI_PASSWORD> dist/*

How to then install the package from a private pypi server:

pip install --extra-index-url "http://<PYPI_USERNAME>:<PYPI_PASSWORD>@123.134.56.7:8080" private_package

Change the placeholders:

• <URL>: The URL of your private PyPI repository.
• <PYPI_USERNAME>: The username for accessing your private PyPI repository.
• <PYPI_PASSWORD>: The password for accessing your private PyPI repository.
• 123.134.56.7: The placeholder IP address of your private PyPI repository.
  • Note: it's better to use a domain like my-pip-server.com instead of an IP address so that you can set up an SSL connection, otherwise your credentials are passed in plain text over the internet. Instructions how to do that are available here and here.
• 8080: The placeholder port number for your private PyPI repository.
• private_package: The private package you want to install.

Option 2. Using nbdev to do it

nbdev itself doesn't do anything different, as seen in their def release_pypi function in https://github.com/fastai/nbdev/blob/3f0266328c2537a35487767288bc4283308f7048/nbs/api/18_release.ipynb#L748

In fact, you probably can just create a ~/.pypirc file like

[distutils]
index-servers =
    private

[private]
repository: https://pypi.example.com
username: user
password: PUT_PASSWORD_HERE

and just pass release_pypi(repository="private") and you will be able to upload to your private PyPI server.

Existing GitHub issues related to this topic

• https://forums.fast.ai/t/nbdev-for-private-pypi-repositories-and-poetry-package-management/100523/4
• https://github.com/fastai/nbdev/issues/1146

[Elijas]

Is it not possible to pip install directly from GitHub? Like so (replacing org and repo):

pip install git+https://github.com/org/repo

I believe this syntax even works in requirements files. You can also pin to a specific commit, branch, or tag (see here).

But you won't be able to select a version range like this. This becomes especially problematic if you'd try to pin a specific commit, branch, or tag in settings.ini (which is used by setup.py), as that directly will lead to a dependency hell (Read more: setup.py vs requirements.txt)

[Elijas]

I've searched a lot to see if its possible to clone a repo with a PAT and https, I couldn't find anything that would work.

For the Classic PAT, execute this command once, and you'll be able to clone any private repo: git config --global url."https://<USERNAME>:<CLASSIC_PAT_TOKEN>@github.com/".insteadOf "https://github.com/"

For the fine-grained PAT, use this command instead: git config --global url."https://oauth2:<FINEGRAINED_PAT_TOKEN>@github.com/".insteadOf "https://github.com/"