DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Fix Bloom logits mismatch

Open molly-smith opened this issue 2 years ago • 0 comments

Bloom with kernel injection was showing significant logits mismatch compared to Transformer's baseline as reported by issue https://github.com/microsoft/DeepSpeed/issues/2730.

Softmax input_mask is float32, not int64, and needs to be converted to half.

@RezaYazdaniAminabadi

molly-smith avatar Feb 18 '23 01:02 molly-smith