Megatron-DeepSpeed icon indicating copy to clipboard operation
Megatron-DeepSpeed copied to clipboard
verified publisher icon bigscience-workshop

[checkpoints] replace bf16 with fp32 checkpoint weights

Open stas00 opened this issue 3 years ago • 3 comments

as requested by @thomasw21 this is a little hack script to to replace half-precision weights with fp32 weights in the existing HF transformers checkpoint seeded from the universal checkpoint, see the script for the steps.

stas00 avatar Aug 10 '22 03:08 stas00

@thomasw21, at your convenience - no rush at all - we can merge this when it meets your needs.

stas00 avatar Aug 11 '22 19:08 stas00

Hey sorry for delaying this, I haven't had the chance to play around with it yet. Let me know if it's blocking and we'll merge then.

thomasw21 avatar Aug 16 '22 08:08 thomasw21

Absolutely no rush, you're the only one who asked for it. So it's totally up to you Thomas.

stas00 avatar Aug 16 '22 16:08 stas00