pyemd
pyemd copied to clipboard
wmayner
pyemd in databricks could not be installed
pyemd in databricks could not be installed in Databrick Runtime version 4.2 (no workers). The error is below:
%sh
lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Stepping: 1
CPU MHz: 2294.685
BogoMIPS: 4589.37
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 51200K
NUMA node0 CPU(s): 0-3
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt
Python:
sys.version_info
Out[25]: sys.version_info(major=3, minor=5, micro=2, releaselevel='final', serial=0)
Pip:
%sh
/databricks/python3/bin/python3 -m pip freeze
ansi2html==1.1.1
azure-common==1.1.18
azure-nspkg==3.0.2
azure-storage-blob==1.1.0
azure-storage-common==1.1.0
azure-storage-nspkg==3.1.0
backports.shutil-get-terminal-size==1.0.0
boto==2.42.0
boto3==1.4.1
botocore==1.4.70
brewer2mpl==1.4.1
certifi==2016.2.28
cffi==1.7.0
chardet==2.3.0
colorama==0.3.7
configobj==5.0.6
cryptography==1.5
cycler==0.10.0
Cython==0.24.1
decorator==4.0.10
docutils==0.14
enum34==1.1.6
et-xmlfile==1.0.1
freetype-py==1.0.2
funcsigs==1.0.2
fusepy==2.0.4
ggplot==0.6.8
html5lib==0.999
idna==2.1
ipaddress==1.0.16
ipython==2.2.0
ipython-genutils==0.1.0
jdcal==1.2
Jinja2==2.8
jmespath==0.9.0
llvmlite==0.13.0
lxml==3.6.4
MarkupSafe==0.23
matplotlib==1.5.3
mpld3==0.2
msgpack-python==0.4.7
ndg-httpsclient==0.3.3
numba==0.28.1
numpy==1.16.1
openpyxl==2.3.2
pandas==0.24.1
pathlib2==2.1.0
patsy==0.4.1
pexpect==4.0.1
pickleshare==0.7.4
Pillow==3.3.1
pkg-resources==0.0.0
ply==3.9
prompt-toolkit==1.0.7
psycopg2==2.6.2
ptyprocess==0.5.1
py4j==0.10.3
pyarrow==0.8.0
pyasn1==0.1.9
pycparser==2.14
pycurl==7.43.0
Pygments==2.1.3
pygobject==3.20.0
pyOpenSSL==16.0.0
pyparsing==2.2.0
pypng==0.0.18
python-apt==1.1.0b1+ubuntu0.16.4.1
python-dateutil==2.5.3
python-geohash==0.8.5
pytz==2016.6.1
requests==2.11.1
s3transfer==0.1.9
scikit-learn==0.18.1
scipy==0.18.1
scour==0.32
seaborn==0.7.1
simplejson==3.8.2
singledispatch==3.4.0.3
six==1.10.0
ssh-import-id==5.5
statsmodels==0.8.0
traitlets==4.3.0
unattended-upgrades==0.1
urllib3==1.19.1
virtualenv==15.0.1
wcwidth==0.1.7
Error:
%sh
/databricks/python3/bin/python3 -m pip install pyemd
Downloading https://files.pythonhosted.org/packages/c0/c5/7fea8e7a71cd026b30ed3c40e4c5ea13a173e28f8855da17e25271e8f545/pyemd-0.5.1.tar.gz (91kB)
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/pyemd.egg-info
writing requirements to pip-egg-info/pyemd.egg-info/requires.txt
writing pip-egg-info/pyemd.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pyemd.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pyemd.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/pyemd.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-eii9t9ra/pyemd/setup.py", line 121, in <module>
'Programming Language :: Python :: 3.6',
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 278, in run
self.find_sources()
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 293, in find_sources
mm.run()
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 524, in run
self.add_defaults()
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/command/egg_info.py", line 560, in add_defaults
sdist.add_defaults(self)
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
self._add_defaults_ext()
File "/[REDACTED]/python3/lib/python3.5/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/usr/lib/python3.5/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3.5/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/tmp/pip-install-eii9t9ra/pyemd/setup.py", line 77, in finalize_options
import numpy
ImportError: No module named 'numpy'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-eii9t9ra/pyemd/
Might it be that the path to python in /usr/lib or similar is hardcoded somewhere there?
The problem is that NumPy needs to be imported during the setup process so that its C headers can be included, and for some reason it cannot be imported.
You could try reinstalling NumPy directly before installing PyEMD.
It is still not working, I tried installing first Numpy and then PyEMD but it doesn't work. Has anyone tried to install it successfully? Thank you very much.
