Support a "command_prefix" option to customize how Pulumi gets invoked

[punkdata]

This is the error produced in the deploy_to_gcp: job of my config when calling the pulumi/update: orb function

#!/bin/bash -eo pipefail
pulumi update --stack dev --cwd ${HOME}/project/pulumi/gcp/orb-demo/ 
Previewing update (dev):

    pulumi:pulumi:Stack orb-demo-dev  Traceback (most recent call last):
    pulumi:pulumi:Stack orb-demo-dev    File "/home/circleci/.pulumi/bin/pulumi-language-python-exec", line 14, in <module>
    pulumi:pulumi:Stack orb-demo-dev      import pulumi
    pulumi:pulumi:Stack orb-demo-dev  ModuleNotFoundError: No module named 'pulumi'
    pulumi:pulumi:Stack orb-demo-dev  It looks like the Pulumi SDK has not been installed. Have you run pip install?
    pulumi:pulumi:Stack orb-demo-dev  If you are running in a virtualenv, you must run pip install -r requirements.txt from inside the virtualenv.
    pulumi:pulumi:Stack orb-demo-dev  error: an unhandled error occurred: Program exited with non-zero exit code: 1
    pulumi:pulumi:Stack orb-demo-dev  1 error; 6 messages
 
Diagnostics:
  pulumi:pulumi:Stack (orb-demo-dev):
    Traceback (most recent call last):
      File "/home/circleci/.pulumi/bin/pulumi-language-python-exec", line 14, in <module>
        import pulumi
    ModuleNotFoundError: No module named 'pulumi'
    It looks like the Pulumi SDK has not been installed. Have you run pip install?
    If you are running in a virtualenv, you must run pip install -r requirements.txt from inside the virtualenv.
 
    error: an unhandled error occurred: Program exited with non-zero exit code: 1
 
error: an error occurred while advancing the preview
Exited with code 255

I'm using pipenv and it looks like the pulumi/update function is kicking off outside of the pipenv environment which caused the installed deps to be hidden from the pulumi/update orb call. I'm unable to get the pulumi/update call to complete successfully when using pipenv to install deps.

Here is the config that I'm currently using. Some Jobs are comment out to save time.

version: 2.1

orbs:
  pulumi: pulumi/[email protected]

jobs:
  build_test:
    docker:
      - image: circleci/python:3.7.2
        environment:
          PIPENV_VENV_IN_PROJECT: 'true'
    steps:
      - checkout
      - run:
          name: Install Python Dependencies
          command: |
            pipenv install
      - run:
          name: Run Tests
          command: |
            pipenv run pytest
  build_push_image:
    docker:
      - image: circleci/python:3.7.2
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: false
      - run:
          name: Build and push Docker image
          command: |       
            pipenv install
            pipenv run pyinstaller -F hello_world.py
            echo 'export TAG=0.1.${CIRCLE_BUILD_NUM}' >> $BASH_ENV
            echo 'export IMAGE_NAME=python-cicd-workshop' >> $BASH_ENV
            source $BASH_ENV
            docker build -t $DOCKER_LOGIN/$IMAGE_NAME -t $DOCKER_LOGIN/$IMAGE_NAME:$TAG .
            echo $DOCKER_PWD | docker login -u $DOCKER_LOGIN --password-stdin
            docker push $DOCKER_LOGIN/$IMAGE_NAME
  deploy_to_gcp:
    docker:
      - image: circleci/python:3.7.2
    steps:
      - checkout
      - run:
          name: Generate GCP Credentials
          command: |
            echo ${GOOGLE_CLOUD_KEYS} | base64 --decode --ignore-garbage > ${HOME}/project/pulumi/gcp/orb-demo/cicd_demo_gcp_creds.json
      - pulumi/login:
          access-token: ${PULUMI_ACCESS_TOKEN}
      - run:
          name: Install dependecies 
          command: pipenv install pulumi pulumi-gcp         
      - pulumi/update:
          stack: dev
          working_directory: ${HOME}/project/pulumi/gcp/orb-demo/
workflows:
  build_test_deploy:
    jobs:
      - build_test
      # - build_push_image:
      #     requires:
      #       - build_test
      - deploy_to_gcp:
          requires:
          # - build_push_image
          - build_test # kill this line once the deploy issue is fixed

[punkdata]

So I tried a straight dep install using:

      - run:
          name: Install dependecies 
          command: sudo pip install --upgrade pip==18.0 && pip install --user pulumi pulumi-gcp

This actually works but obviously doesn't use the pipenv cmd. I just want to confirm that the issue was isolated to pipenv.

[chrsmith]

So the behavior you are seeing is generally by design. In order to pick up the packages managed by pipenv, any commands need to be in a processed launched by pipenv. Though we can repurpose this issue to track supporting this scenario.

If we were to update the pulumi/{update, preview} orbs to have a "command prefix" or something. If we had an escape hatch where you could specify some aribrary text before the pulumi update command, that should allow you to run the pulumi update command within pipenv. Not the best design, but possibly useful for some other scenarios too.

For posterity, here are some other approaches:

• Install packages system wide by just using sudo pip install. (Like you mentioned above.) Obviously this defeats the purpose of using pipenv.

• Create a pipenv CircleCI orb that will allow nesting of other orbs, and have that take care of the "execute within pipenv environment". That would be pretty slick, but might not actually be possible based on the way Orbs get executed and how that interacts with pipenv.

[josegonzalez]

This also impacts usage of the orb with poetry as dependencies there are installed in a virtualenv as well.

My workaround is something like this (assuming you are using the python orb to install poetry deps):

      - python/install-packages:
          pkg-manager: poetry
      - run:
          name: "Add poetry venv to environment"
          command: |
            echo "export PATH=$(poetry env info --path)/bin:$PATH" >> "$BASH_ENV"
            echo "export VIRTUAL_ENV=$(poetry env info --path)" >> "$BASH_ENV"
      # pulumi commands here