AttributeError: 'LocalSagemakerClient' object has no attribute 'create_pipeline'

[neilmcguigan]

Describe the bug SageMaker Python SDK model building pipelines does not work in Local Mode

To reproduce

sagemaker_session = LocalSession()
sagemaker_session.config = {'local': {'local_code': True}}

processor = SKLearnProcessor(
    base_job_name = "processingjob",
    role = cfg.role_arn,
    framework_version = cfg.sklearn_version,
    instance_count = cfg.instance_count,
    instance_type = "local",
    sagemaker_session=sagemaker_session
)

processing_step = ProcessingStep(
    name="processing",
    processor=processor,
    code = "code/preprocessor.py"
)

pipeline = Pipeline(
    name = "mypipeline",
    steps = [processing_step],
    sagemaker_session = sagemaker_session
)

pipeline.upsert(role_arn = cfg.role_arn)
pipeline.start()

Expected behavior SageMaker model building pipelines should work in Local Mode for faster development

Screenshots or logs AttributeError: 'LocalSagemakerClient' object has no attribute 'create_pipeline'

System information A description of your system. Please provide:

• SageMaker Python SDK version: 2.78.0
• Python version: 3.8
• CPU or GPU: CPU
• Custom Docker image (Y/N): N

[staubhp]

Hi @neilmcguigan, you're correct that local mode jobs won't work today in a pipeline. For that to happen, pipelines needs its own version of local mode that recreates the server-side orchestration logic. This is on the team's radar.

[ycrouin]

Hello, the cold-start time of SageMaker pipelines is a real pain when developing and doing small experiments, and can account for most of the pipeline execution time. This feature would be great to tackle this issue! Is there any ETA?

[knoam]

This would be very useful.

[jerrypeng7773]

we are diligently working on this. =)

[cirofdo]

Good to know, for now I'll keep testing locally each step at a time.

[cirofdo]

It seems to be avaiable now https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-local-mode.html

[ShwetaSingh801]

Closing this tkt since the requested feature is now available in SageMaker Python SDK.