[Serve] Track user-configured options in Serve deployments

[shrekris-anyscale]

Signed-off-by: Shreyas Krishnaswamy [email protected]

Why are these changes needed?

Background Serve sets deployment config defaults directly via keyword arguments in the @serve.deployment decorator. This makes it it makes it impossible to track whether the config options were set by default or by the user. Distinguishing between these would let serve build print only the user-configured options, instead of all options (including defaults).

Changes This change adds a layer of indirection in the @serve.deployment decorator. It sets all the decorator config options to an enum element, DEFAULT.VALUE, by default. This element indicates that the config setting was set by default– not by the user. The Serve DeploymentSchema is also updated to use DEFAULT.VALUE as default.

This change also introduces a new Set called user_configured_options to the DeploymentConfig, which tracks the group of options configured by the user. When Serve is schematized or serialized, it can use the values in this set to indicate which values are default and which ones are user-configured.

serve build now prints only user-configured values (as well as deployment names) to the Serve config file.

None is no longer the default value for deployment options. The only options that can be set to None (or null in the Serve config) are:

• route_prefix: None means the deployment should not be exposed over HTTP.
• num_replicas: None is allowed if an autoscaling_config is provided.
• autoscaling_config: None is allowed if a num_replicas is provided.
• user_config: None indicates that there's no user_config to pass into reconfigure.

Follow-up Changes

• serve build should offer a flag that lets users print all options, including the defaults. This is especially useful for users that want to explicitly track all options settings. It may also be useful to always print the route_prefixes for Serve deployments if they're not None to easily see which deployment is the driver.
• Plumb the Serve codebase to actually make certain options non-optional (e.g. ray_actor_options). This change only modifies the thin wrappers over the Deployment object (i.e. the @serve.deployment decorator and the Serve schemas) but not the Deployment object itself.

Related issue number

N/A

Checks

• [X] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• [X] I've run scripts/format.sh to lint the changes in this PR.
• [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
• [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • [ ] Unit tests
  • [ ] Release tests
  • [ ] This PR is not tested :(