DeepSpeed
DeepSpeed copied to clipboard
Remove bf16 from inference config dtye enum
Remove bf16 from inference config dtye enum because not it is not supported. Users should now see pydantic error with supported types vs. vague CUDA error.
pydantic.error_wrappers.ValidationError: 1 validation error for DeepSpeedInferenceConfig
dtype
value is not a valid enumeration member; permitted: torch.float16, torch.float32, torch.int8 (type=type_error.enum; enum_values=[<DtypeEnum.fp16: torch.float16, 'torch.float16', 'fp16', 'float16', 'half'>, <DtypeEnum.fp32: torch.float32, 'torch.float32', 'fp32', 'float32', 'float'>, <DtypeEnum.int8: torch.int8, 'torch.int8', 'int8'>])
https://github.com/microsoft/DeepSpeed/issues/2954