Does text_encoder work in float16?

[elephantpanda]

I recently converted stable diffusion to float16 onnx with the convert tool.

Everything works great. Except the text_encoder.onnx doesn't work (as in it doesn't produce any images just random patterns). So I am still using the float32 version of the text_econder.

My assumption is that because although the input is in int32, internally a float16 can only represent consecutive integers up to 2048. So perhaps something is going wrong there?

I am doing something wrong or should the text_encoder work with float16?

[patrickvonplaten]

cc @anton-l for ONNX.

[patrickvonplaten]

cc @anton-l @echarlaix here

[anton-l]

@pauldog for a bit of context here, have you used the convert_stable_diffusion_checkpoint_to_onnx.py script with --fp16, running on CUDA? Is it a Stable Diffusion 1.x or a 2.x model? Any chance you could link the source checkpoint for debugging?

[elephantpanda]

@pauldog for a bit of context here, have you used the convert_stable_diffusion_checkpoint_to_onnx.py script with --fp16, running on CUDA? Is it a Stable Diffusion 1.x or a 2.x model? Any chance you could link the source checkpoint for debugging?

Yes I used that script. The original problem was for SD 1.4. It seemed to work ok when I redid it for SD 2.1. So maybe I did something wrong the first time? Although the text encoders are different as SD1.4 uses vectors of length 768 while SD 2.1 uses vectors of length 1024.

My setup is Nvidia P5000 DirectML. Using Onnxruntime on Windows 10. c#

I'll have to check it again to make sure.

(Maybe this is an issue for onnxruntime or directml rather than huggingface.)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.