NVIDIA
[FastPitch1.1/pytorch] A script to convert the model trained on GPU to the one that can be run on CPU
Related to FastPitch1.1/pytorch pytorch 1.10.2+cu111
Is your feature request related to a problem? Please describe.
I trained FastPitch1.1 on GPU, with AMP setup before I got that Half is not fully supported on pytorch CPU mode, when I run inference with GPU checkpoints on CPU, got exceptions.
fastpitch/transformer.py, line 32, in forward pos_emb = torch.cat([sinusoid_inp.sin(), sinusoid_inp.cos()], dim=1) RuntimeError: "sin" "_vml_cpu" not implemented for 'Half'
Describe the solution you'd like I requires a new script, to convert the GPU trained FastPitch model into the one can be run on CPU. no matter what are the training setup.
Describe alternatives you've considered The FastPitch implementation can support AMP model on CPU.
Additional context Add any other context or screenshots about the feature request here.
