DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Remove bf16 from inference config dtye enum

Open molly-smith opened this issue 2 years ago • 0 comments

Remove bf16 from inference config dtye enum because not it is not supported. Users should now see pydantic error with supported types vs. vague CUDA error.

pydantic.error_wrappers.ValidationError: 1 validation error for DeepSpeedInferenceConfig
dtype
  value is not a valid enumeration member; permitted: torch.float16, torch.float32, torch.int8 (type=type_error.enum; enum_values=[<DtypeEnum.fp16: torch.float16, 'torch.float16', 'fp16', 'float16', 'half'>, <DtypeEnum.fp32: torch.float32, 'torch.float32', 'fp32', 'float32', 'float'>, <DtypeEnum.int8: torch.int8, 'torch.int8', 'int8'>])

https://github.com/microsoft/DeepSpeed/issues/2954

molly-smith avatar Mar 13 '23 23:03 molly-smith