serverless-python-requirements
serverless-python-requirements copied to clipboard
serverless
AWS lambda deploy fails using a private python repo
Hello,
I am trying to deploy a python lambda function which depends on a package into a private repo, but I get the error below. A GitLab runner executes the sls deploy using poetry to install the dependencies and packagr to get the private package.
serverless.yml
service: premani-rg270-dukane-iqtc-adt
custom:
region: us-west-2
stage: stage
duration_threshold: 20
pythonRequirements:
slim: true
alerts:
dashboards: false
topics:
alarm:
topic: arn:xxx
notifications:
- protocol: email
endpoint: [email protected]
definitions:
functionErrors:
treatMissingData: notBreaching
functionDuration:
treatMissingData: notBreaching
alarms:
- functionErrors
- functionDuration
provider:
name: aws
runtime: python3.8
region: ${self:custom.region}
stage: ${self:custom.stage}
memorySize: 512
timeout: 30
tracing:
lambda: true
iamRoleStatements:
- Effect: "Allow"
Action:
- "ses:SendEmail"
- "ses:SendRawEmail"
- "sagemaker:InvokeEndpoint"
- "sagemaker:DescribeEndpoint"
Resource: "*"
plugins:
- serverless-python-requirements
- serverless-plugin-aws-alerts
functions:
premani-rg270-dukane-iqtc-adt:
handler: lambda_function.lambda_handler
name: premani-rg270-dukane-iqtc-adt-${self:custom.stage}
description: Trigger anomaly detection for RG270 ultrasound generator Dukane IQT C-series a dis.400-2256 June 2013 version in ${self:custom.stage} env.
alarms:
- name: functionErrors
- name: functionDuration
type: anomalyDetection
comparisonOperator: GreaterThanUpperThreshold
threshold: ${self:custom.duration_threshold}
events:
- schedule:
rate: cron(5 * * * ? *)
sls deploy output
Serverless: Generating requirements.txt from pyproject.toml...
Serverless: Parsed requirements.txt from pyproject.toml in /builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/.serverless/requirements.txt...
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/bcd1f00a0181384da43ae6dd50e0d4ed234ff7c1e88ac99f66060683a2e854e4_slspyc/requirements.txt ...
Serverless: Using download cache directory /root/.cache/serverless-python-requirements/downloadCacheslspyc
Serverless: Running ...
Error --------------------------------------------------
Error: STDOUT: Looking in indexes: https://pypi.org/simple, https://api.packagr.app/Wk3Sg2IP1
Collecting boto3==1.16.20
Downloading boto3-1.16.20-py2.py3-none-any.whl (129 kB)
Collecting botocore==1.19.20
Downloading botocore-1.19.20-py2.py3-none-any.whl (6.8 MB)
Collecting certifi==2020.11.8
Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)
Collecting chardet==3.0.4
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting cycler==0.10.0
Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
User for api.packagr.app:
STDERR: ERROR: Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
status = self.run(options, args)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
return func(self, options, args)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 323, in run
requirement_set = resolver.resolve(
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 339, in _get_abstract_dist_for
self._populate_link(req)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 305, in _populate_link
req.link = self._find_requirement_link(req)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 270, in _find_requirement_link
best_candidate = self.finder.find_requirement(req, upgrade)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 900, in find_requirement
best_candidate_result = self.find_best_candidate(
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 883, in find_best_candidate
candidates = self.find_all_candidates(project_name)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 827, in find_all_candidates
package_links = self.process_project_url(
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 791, in process_project_url
html_page = self._link_collector.fetch_page(project_url)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 629, in fetch_page
return _get_html_page(location, session=self.session)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 441, in _get_html_page
resp = _get_html_response(url, session=session)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 138, in _get_html_response
resp = session.get(
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/session.py", line 421, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 650, in send
r = dispatch_hook('response', hooks, r, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
_hook_data = hook(hook_data, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/auth.py", line 256, in handle_401
username, password, save = self._prompt_for_password(parsed.netloc)
File "/usr/local/lib/python3.8/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password
username = ask_input("User for {}: ".format(netloc))
File "/usr/local/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 259, in ask_input
return input(message)
EOFError: EOF when reading a line
at /builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/serverless-python-requirements/lib/pip.js:325:13
at Array.forEach (<anonymous>)
at installRequirements (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/serverless-python-requirements/lib/pip.js:312:28)
at installRequirementsIfNeeded (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/serverless-python-requirements/lib/pip.js:556:3)
at ServerlessPythonRequirements.installAllRequirements (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/serverless-python-requirements/lib/pip.js:635:29)
at ServerlessPythonRequirements.tryCatcher (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/promise.js:729:18)
at _drainQueueStep (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/builds/galdi/premani/premani-rg270-dukane-iqtc-anomaly-detection-trigger/node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:461:21)
Installed version
Operating System: linux
Node Version: 12.19.1
Framework Version: 2.11.1
Plugin Version: 4.1.2
SDK Version: 2.3.2
Components Version: 3.3.1
This is likely caused by an issue with the poetry export command in the current released version not using the --with-credentials parameter when it generates your requirements.txt file from the pyproject.toml file. This is fixed in master but since there hasn't been a release since February you likely aren't running with this update. You have a few options to resolve your issue.
Options:
- change to reference GitHub when you install serverless-python-requirements in your package.json
- generate the requirements.txt file yourself before running serverless and change your serverless config so
usePoetry = false. The command to generate the requirements file ispoetry export -f requirements.txt -o requirements.txt --without-hashes --with-credentials.
v5.1.1 seems to now have the --with-credentials so this issue might be resolved.
