TransformerEngine icon indicating copy to clipboard operation
TransformerEngine copied to clipboard

Support simulating FP8 on older hardware

Open zplizzi opened this issue 2 years ago • 1 comments

It would be great if this library supported simulating FP8 on eg Ampere hardware, as you did in the FP8 whitepaper. I'm sure a lot of people are interested in seeing if their models will work well in FP8 before investing a lot of money in H100s, let alone the fact that they're barely available yet.

I see https://github.com/IntelLabs/FP8-Emulation-Toolkit, but it's poorly documented and it's not clear if it implements the same tensor scaling algorithms that you have here.

zplizzi avatar Feb 10 '23 23:02 zplizzi

Have you ever integrated the "FP8-Emulation-Toolkit" into TransformerEngine, and run a simple network ?

shoveller86 avatar Jun 20 '23 09:06 shoveller86