[WIP] #7535 Update FloatTensor type hints to Tensor

[vanakema]

What does this PR do?

Updates all references to torch.FloatTensor to torch.Tensor. This is because FloatTensor is essentially deprecated and all Tensor types are actually just Tensors now. FloatTensor was kept to be backwards compatible as noted here.

This allows type hints to not yell when a user uses the canonical Tensor type

Fixes #7535

Before submitting

• [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [x] Did you read the contributor guideline?
• [x] Did you read our philosophy doc (important for complex PRs)?
• [x] Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• [x] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
• [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

[vanakema]

@DN6 Can you approve this for a pipeline run? Due to the number of files this touches, I was trying to run the full suite on my laptop but it was taking over an hour, and I see it takes way less than an hr (around 20m) for people's tests to run in the CI pipeline.

[yiyixuxu]

cc @standardAI here :)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yiyixuxu]

I think we need to manually update this too (a test currently fails) https://github.com/huggingface/diffusers/blob/8edaf3b79c564a56fbffb003ee74ac92e9162a73/tests/others/test_check_copies.py#L31C1-L31C15

[vanakema]

Yep! On it. Thanks for the review!

[vanakema]

Do you think it makes sense to update the examples too? Just realized I missed those

[vanakema]

I really meant all of them that reference or use FloatTensor. I decided to update them.

I'm running the tests on my server right now instead of my laptop, but they're still going pretty slow. Do one of you mind approving the workflow again?

[vanakema]

How long should the full suite of tests take to run? I started a full run last night on my home server and as of this morning, they're still running. I'm curious to hear what's the typical dev flow for situations like this for those of you at HF. It seems super weird for it to take this long. Maybe the 32 worker threads it spun up was too many and my box is getting bottlenecked somewhere? Would it run faster if I passed my GPU to it? On my Mac, the pipelines never were this slow (but they were only running 8 threads since there were only 8 "cores"), but on my much more powerful server it is typical to see numbers like "159s/it".

[vanakema]

An update on the test that failed: I believe the one that caused the failed pipeline was a fluke as it passes consistently on my box. I think it just needs a retry.

Looking forward at other tests that have yet to run in the pipeline, I am consistently having issues with one of the determinism tests: tests/models/unets/test_models_unet_spatiotemporal.py::UNetSpatioTemporalConditionModelTests::test_set_attn_processor_for_determinism

I see that elsewhere the precision required for the determinism test was reduced to 10^2 down from 10^3. When I apply this to the base_precision in the ModelTesterMixin in test_modeling_common.py, the determinism test that was regularly failing on my box now passes. I am curious to hear y'all's thoughts on this, and why you think this might be happening. Every (almost every) change in this PR is only changes to the type hints and documentation. I trawled through nearly every file's changes (I did skip a batch of the pipeline and schedulers, and all the deprecated schedulers), and I could not find any instance of actual object instantiation that would affect this. It's strange though because I don't understand how typehints would change the behavior of the determinism test.

Admittedly determinsim is definitely something that's a little out of my wheelhouse, so it's totally possible I missed something, and I would love to hear any ideas y'all may have. In the meantime, I'll continue investigating tomorrow in my free time. I'm totally down to hop on a call to talk about it if any of you at HF are open to it! But I know you guys are probably really busy, so I also completely understand if that's not something of interest. I'll hop on the Discord though

Thanks in advance!

[tolgacangoz]

Thanks for updating type hints Mark! The failed pipeline test can be passed on my local PC too. test_set_attn_processor_for_determinism can also be passed on my local PC. This is my setup:

❯ diffusers-cli env
An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- 🤗 Diffusers version: 0.28.0.dev0
- Platform: Ubuntu 24.04 LTS - Linux-6.8.0-31-generic-x86_64-with-glibc2.39
- Running on a notebook?: No
- Running on Google Colab?: No
- Python version: 3.10.14
- PyTorch version (GPU?): 2.3.0+cu121 (True)
- Flax version (CPU?/GPU?/TPU?): 0.8.3 (cpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26
- Huggingface_hub version: 0.23.0
- Transformers version: 4.40.2
- Accelerate version: 0.30.0
- PEFT version: 0.10.0
- Bitsandbytes version: 0.43.1
- Safetensors version: 0.4.3
- xFormers version: not installed
- Accelerator: NVIDIA GeForce GTX 1650, 4096 MiB VRAM
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

And, I used this command:

python -m pytest -n auto --dist=loadfile -s -v examples/text_to_image/test_text_to_image.py::TextToImage::test_text_to_image

[vanakema]

Oh awesome, thanks for testing that out Tolga! Ok maybe it just needs a retry then? I'll rebase on main and then can @yiyixuxu or @stevhliu approve the pipeline run?

[vanakema]

Pipeline passed, woohoo, let's go! Does someone with permissions mind merging?

[yiyixuxu]

thank you! thank @standardAI for your help too :)

[vanakema]

Amazing, thank you all! Super stoked to have my first contribution merged in :)