Using AutoMLStep requires azureml.train.automl.runtime

[ross-p-smith]

I am attempting to have an AutomMLStep in a new pipeline based off this sample. In order to add this step into my prepare_train.py script I need to use the following

from azureml.train.automl.runtime import AutoMLStep

This is not included in azureml.train.automl.client which is already installed in the mlops container. This 'runtime' library cannot be pip installed using the mcr.microsoft.com/mlops/python:latest container as it has conflicts with numpy and pandas. So I have attempted to build the container myself following the docker-image-pipeline.yml example. However, when I build my own image I am having problems with psutil

creating build/temp.linux-x86_64-3.7/psutil gcc -pthread -B /usr/local/envs/mlopspython_ci/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/usr/local/envs/mlopspython_ci/include/python3.7m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.7/psutil/_psutil_common.o unable to execute 'gcc': No such file or directory error: command 'gcc' failed with exit status 1

My issue is how to AutomMLStep in the mlops pipeline?

Everything works fine in a Notebook!

[algattik]

Perhaps adding psutil and/or gcc conda package to your env should work.

name: mlopspython_ci
dependencies:
  - gcc

[ross-p-smith]

Thanks Alexandre, I did try that, but you cant install gcc in a container (apparently!). We have reverted back to our own ci container but instead of 1.0.85 we are 1.0.83 and we can now use AutoMLStep

• azureml-sdk==1.0.83
• azureml-train-automl==1.0.83
• azureml-train-automl-runtime==1.0.83.1

Unsure whether this issue should be left open for 1.0.85 of azureml-train-automl