Fix docker image
By running docker run -it --rm srcd/ml --help I was not able to run docker image with the following error messages:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 570, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 888, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 779, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (bblfsh 2.9.14 (/usr/local/lib/python3.4/dist-packages), Requirement.parse('bblfsh==2.9.13'), {'sourced-jgit-spark-connector'})
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/srcml", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 3095, in <module>
@_call_aside
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 3079, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 3108, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 572, in _build_master
return cls._build_from_requirements(__requires__)
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 585, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 779, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (bblfsh 2.9.14 (/usr/local/lib/python3.4/dist-packages), Requirement.parse('bblfsh==2.9.13'), {'sourced-jgit-spark-connector'})
jgit-spark-connector requires bblfsh with the version of 2.9.13.
After this change received the following error message:
/usr/local/lib/python3.4/dist-packages/sourced/ml/cmd/id2vec_preprocess.py:10: UserWarning: Tensorflow is not installed, dependent functionality is unavailable.
warnings.warn("Tensorflow is not installed, dependent functionality is unavailable.")
/usr/local/lib/python3.4/dist-packages/sourced/ml/cmd/run_swivel.py:21: UserWarning: Tensorflow is not installed, dependent functionality is unavailable.
warnings.warn("Tensorflow is not installed, dependent functionality is unavailable.")
Traceback (most recent call last):
File "/usr/local/bin/srcml", line 11, in <module>
load_entry_point('sourced-ml==0.8.2', 'console_scripts', 'srcml')()
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 476, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2700, in load_entry_point
return ep.load()
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2318, in load
return self.resolve()
File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2324, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/local/lib/python3.4/dist-packages/sourced/ml/__main__.py", line 8, in <module>
from sourced.ml import cmd
File "/usr/local/lib/python3.4/dist-packages/sourced/ml/cmd/__init__.py", line 22, in <module>
from sourced.ml.cmd.id2role_eval import id2role_eval
File "/usr/local/lib/python3.4/dist-packages/sourced/ml/cmd/id2role_eval.py", line 6, in <module>
import pandas
ImportError: No module named 'pandas'
Move on to the setup.py pandas should be not in extras_require, but in install_requires group and as of python 3.4 usage pandas<0.21 should be used.
With these changes, I was able to build an image and run a container.
Thank you for testing the docker build! I am going to include it to the CI so that we never break it again.
This is going to be resolved after we split this project into the library and the framework, stay tuned.