Add image builder user defined

[arbaobao]

Tracking issue

Why are the changes needed?

In some situations, there isn't internet to build image from pulling uv and micromamba images from the net.

What changes were proposed in this pull request?

I add an ImageBuilderConfig to detect if there is user defined image_builder in config.yaml.

How was this patch tested?

This patch can be tested by adding image_builder to ~/.flyte/config.yaml

image_builder:
  uv_image: arbaobao/flytekit:uv
  micromamba_image: arbaobao/flyte-test-images:micromamba-amd64

if we don't specify the image, the default images are ghcr.io/astral-sh/uv:0.5.1 and mambaorg/micromamba:2.0.3-debian12-slim.

Setup process

1. Run a flyte sandbox and add image_builder to the config.yaml.
2. if you are using image_spec to run the workflow, it will use the image that you specified to build the task image.

Screenshots

Check all the applicable boxes

• [ ] I updated the documentation accordingly.
• [ ] All new and existing tests passed.
• [X] All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This PR adds support for user-defined image builder configuration to enhance flexibility in offline or air-gapped environments. It refactors configuration classes to better separate default from user-defined settings, replacing legacy configuration with new classes including ImageBuilderConfig and updated DataConfig. Docker file templates are revised to allow users to override default images through configuration files.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 2

[flyte-bot]

Code Review Agent Run #4675f8

Actionable Suggestions - 2

• flytekit/image_spec/default_builder.py - 1

  • Consider defining default values for template variables · Line 96-97
• flytekit/configuration/__init__.py - 1

  • Function call in dataclass default · Line 673-673

Review Details

• Files reviewed - 3 · Commit Range: 44c7cc0..e10a65c

  • flytekit/configuration/__init__.py
  • flytekit/configuration/internal.py
  • flytekit/image_spec/default_builder.py
• Files skipped - 0
• Tools

  • Whispers (Secret Scanner) - ✔︎ Successful
  • Detect-secrets (Secret Scanner) - ✔︎ Successful
  • MyPy (Static Code Analysis) - ✔︎ Successful
  • Astral Ruff (Static Code Analysis) - ✔︎ Successful

---

AI Code Review powered by

[flyte-bot]

Changelist by Bito

This pull request implements the following key changes.

Key Change	Files Impacted
New Feature - User-defined Image Builder Configuration	- __init__.py - Introduces new dataclass-based configuration classes (DefaultConfig and ImageBuilderConfig) and their auto methods to load user-defined image builder settings. - internal.py - Adds new configuration entries (IMAGE_BUILDER_NAME, UV_IMAGE, and MICROMAMBA_IMAGE) to support the image builder feature.
Feature Improvement - Docker Builder Template Parameterization	- default_builder.py - Replaces hardcoded image sources with configurable variables in the Docker template and updates the docker context creation to utilize the user-specified image values.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 77.22%. Comparing base (2e12f43) to head (a84654b). Report is 13 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3180      +/-   ##
==========================================
- Coverage   83.58%   77.22%   -6.37%     
==========================================
  Files           3      213     +210     
  Lines         195    22275   +22080     
  Branches        0     2901    +2901     
==========================================
+ Hits          163    17201   +17038     
- Misses         32     4227    +4195     
- Partials        0      847     +847     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[flyte-bot]

Code Review Agent Run #df86ab

Actionable Suggestions - 1

• flytekit/configuration/__init__.py - 1

  • Function call in dataclass default · Line 668-668

Review Details

• Files reviewed - 3 · Commit Range: e10a65c..a84654b

  • flytekit/configuration/__init__.py
  • flytekit/configuration/internal.py
  • flytekit/image_spec/default_builder.py
• Files skipped - 0
• Tools

  • Whispers (Secret Scanner) - ✔︎ Successful
  • Detect-secrets (Secret Scanner) - ✔︎ Successful
  • MyPy (Static Code Analysis) - ✔︎ Successful
  • Astral Ruff (Static Code Analysis) - ✔︎ Successful

---

AI Code Review powered by

[pingsutw]

@thomasjpfan I see your point, but if users want to use the UV image from a private registry, they must update every image spec

[pingsutw]

or we can probably show the builder config in the error message, will that be helpful for debugging

[thomasjpfan]

I see your point, but if users want to use the UV image from a private registry, they must update every image spec

I think it is reasonable for users to change all their ImageSpecs if their uv image is in a private registry. I consider considering the uv image to be on the same level as ImageSpec.base_image.

[arbaobao]

@pingsutw @thomasjpfan
According to the discuss above, I think I will implement this config in image_spec, but user will have to add this config at every image_spec(xxx) repeatedly.

Is there anything still need to discuss?

[pingsutw]

got it, I'm okay with it

ImageSpec(builder_config={"uv_image": "..."})