XCL

Thanks for the question! We are migrating the code and model to the diffusers library and hopefully we can finish it this week.

Hi I did the following things: 1. cleaned the pipeline/transformer codes according to Sayak's suggestions. 2. switched to yiyi's refactored attention. 3. new test code in `test_hunyuan_dit.py`. based on yiyi's...

I made some improvements according to Sayak and Yiyi's suggestions. Several additional problems : 1. Our BertModel cannot work in FP16. So to make our pipeline to FP16, we have...

> > Our BertModel cannot work in FP16. So to make our pipeline to FP16, we have to make only the transformer to FP16 instead of just pipe = HunyuanDiTPipeline.from_pretrained("XCLiu/HunyuanDiT-0523",...

I pushed a new version. In this version: 1. Checked and Merged Yiyi's refactor in https://github.com/huggingface/diffusers/commit/b0e0da28c4d0f057824faacb23da6b06dd43a786 (PR https://github.com/huggingface/diffusers/pull/8310) 2. Fixed FP16 and `to_empty` problem. 3. polished the whole codebase following...

Note: (1) Fix FP16: update the `transformers` to `4.41.1` or later. (2) `to_empty`: https://github.com/huggingface/diffusers/pull/8240/files#r1617108976 ```py from transformers import BertModel bert_model = BertModel.from_pretrained("XCLiu/HunyuanDiT-0523", add_pooling_layer=True, subfolder="text_encoder") pipe = HunyuanDiTPipeline.from_pretrained("XCLiu/HunyuanDiT-0523", text_encoder=bert_model, transformer=model, torch_dtype=torch.float32)...

> thanks!! I think we can merge this soon > > We have tests and doc left - doc can be added in a separate PR if you need more...

I changed XCLiu/HunayunDiT-0523 and simplified `test_hunyuan_dit.py`

@yiyixuxu Thanks for merging! Reply to your TODOs: 1. Doc added here: https://moon-ci-docs.huggingface.co/docs/diffusers/pr_8383/en/api/pipelines/hunyuandit 2. I manually tested and verified the results. The current version has the same output as the...

Thanks for the comments! I'll update the commit. My question is: if we don't add `cross_attention_kwargs`, how do we control `lora scale`? I will provide a doc update after all...