leonary

> Just thought to add my experience: On the latest pull of the SD3 branch, I can train on DevDedistilled. It's detected as a Schnell model because of the missing...

Thanks for the excellent multi-GPU training feature—it works perfectly! I was wondering if you could also enable multi-GPU support for latent caching? The current single-GPU process is very slow and...

Your loss is equal to nan in the initial stage of training. This should be caused by fp16 precision. Set mixed_precision=bf16, and then do not declare t5xxl_dtype.

I totally agree. Since SD3 may not be able to fit a slightly larger dataset due to model problems (scripts include SimpleTuner, SD-scripts, OneTrainer), it is recommended to stop developing...

> I strongly disagree. While the SD3 Medium model has certain drawbacks, it possesses a crucial advantage that FLUX lacks: its weights are publicly available. In contrast, FLUX only provides...

> Stabilityai promised to release the 3.1 model soon. They promised to fix this problem in it. You've been too quick to educate yourself If SD3.1 could achieve the performance...

> Now sd3 branch supports FLUX.1 dev LoRA training experimentally :) https://github.com/kohya-ss/sd-scripts/tree/sd3 Thank you for your excellent work. The fine-tuning effect of sd-scripts with Flux has completely met my expectations,...

Found a repository that encodes with SDXL series VAE and decodes with SD15 series, but it has problems with artifacts and color shift. https://github.com/city96/SD-Latent-Interposer

> Applying T5 mask seems to increase memory usage by about 1GB, but 10GB is too much. Please let me know the versions of PyTorch and CUDA. Upgrading PyTorch to...

> For some reason, FlashAttention for PyTorch's scale_dot_product_attention may be disabled when masking. This may depend on the CUDA version, GPU or mixed precision dtype etc. > > If you...