marvin-python-toolbox
marvin-python-toolbox copied to clipboard
Adding support to build docker images with the engine-httpserver running as default
Now we have a CLI to build docker images.
This fixes issue #35
This first version is able to build two type of docker images:
- base - This is the base docker image that should be extended by any other marvin image.
- spark - This is a docker image that uses marvin base and adds spark on top of it. Ideally should be used by engines that are spark-dependent.
Requirements:
- The host machine should have docker installed.
- The user running marvin CLI should be part of docker group (to avoid running with sudo). See https://docs.docker.com/engine/installation/linux/linux-postinstall/
How to test:
- Clone this branch
- Generate a new project: marvin engine-generate
- Customize with your code (optional)
- Build the docker image: marvin engine-dockerbuild (run --help and play around with parameters)
- Run the image just build: docker run -it marvinai/marvin-spark
- Call the REST pipeline endpoint and check that everything is working fine.
Custom spark configuration:
- If you want to use custom spark files, replace the base files in <engine_home>/docker/marvin-spark-docker/spark-conf/* and build the image.
Codecov Report
Merging #37 into master will increase coverage by
0.75%. The diff coverage is23.07%.
@@ Coverage Diff @@
## master #37 +/- ##
==========================================
+ Coverage 49.57% 50.33% +0.75%
==========================================
Files 28 25 -3
Lines 2104 2110 +6
Branches 210 211 +1
==========================================
+ Hits 1043 1062 +19
+ Misses 1057 1044 -13
Partials 4 4
| Impacted Files | Coverage Δ | |
|---|---|---|
| marvin_python_toolbox/management/engine.py | 27.52% <23.07%> (-0.32%) |
:arrow_down: |
| ...es/python-engine/project_package/_compatibility.py | ||
| ...emplates/python-engine/project_package/_logging.py | ||
| ...on-engine/project_package/data_handler/__init__.py | ||
| marvin_python_toolbox/common/config.py | 96.96% <0%> (+71.96%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update f1c170f...658b23c. Read the comment docs.