stable-diffusion.cpp icon indicating copy to clipboard operation
stable-diffusion.cpp copied to clipboard
Published 3 weeks ago • verified publisher icon leejet

VAE tiling seems to skip the last several tiles during processing

Open lostdisc opened this issue 1 year ago • 3 comments

With both SD 1.5 and SDXL, running with VAE tiling seems to skip the last several tiles during processing.

With SD 1.5, VAE tiling has 16 tiles, but skips to the end after doing 8. The relevant part of the verbose output looks like this:

[DEBUG] ggml_extend.hpp:549  - tile work buffer size: 0.77 MB
[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
[INFO ] ggml_extend.hpp:563  - processing 16 tiles
  |===>                                              | 1/16 - 0.00it/s[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 1/16 - 2.18s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |======>                                           | 2/16 - 2.20s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=========>                                        | 3/16 - 2.25s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |============>                                     | 4/16 - 2.17s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |===============>                                  | 5/16 - 2.17s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |==================>                               | 6/16 - 2.13s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 7/16 - 2.14s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 8/16 - 2.22s/it[DEBUG] ggml_extend.hpp:1062 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================================| 16/16 - 2.22s/it
[DEBUG] stable-diffusion.cpp:1080 - computing vae [mode: DECODE] graph completed, taking 21.88s
[INFO ] stable-diffusion.cpp:1487 - latent 1 decoded, taking 21.88s
[INFO ] stable-diffusion.cpp:1491 - decode_first_stage completed, taking 21.88s
[INFO ] stable-diffusion.cpp:1614 - txt2img completed in 57.88s
save result image to 'output.png'

With SDXL, VAE tiling has 64 tiles, but skips to the end after doing 48. The relevant part of the verbose output looks like this:

[DEBUG] ggml_extend.hpp:513  - tile work buffer size: 0.77 MB
[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
[INFO ] ggml_extend.hpp:527  - processing 64 tiles
  |>                                                 | 1/64 - 0.00it/s[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |>                                                 | 1/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=>                                                | 2/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==>                                               | 3/64 - 1.75s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 4/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===>                                              | 5/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====>                                             | 6/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====>                                            | 7/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |======>                                           | 8/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======>                                          | 9/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======>                                          | 10/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |========>                                         | 11/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========>                                        | 12/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========>                                       | 13/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========>                                       | 14/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===========>                                      | 15/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============>                                     | 16/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=============>                                    | 17/64 - 1.72s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============>                                   | 18/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============>                                   | 19/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===============>                                  | 20/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================>                                 | 21/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================>                                | 22/64 - 1.68s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================>                                | 23/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================>                               | 24/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================>                              | 25/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====================>                             | 26/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 27/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================>                            | 28/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |======================>                           | 29/64 - 1.73s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=======================>                          | 30/64 - 1.74s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |========================>                         | 31/64 - 1.73s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 32/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=========================>                        | 33/64 - 1.58s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==========================>                       | 34/64 - 1.69s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===========================>                      | 35/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============================>                     | 36/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |============================>                     | 37/64 - 1.63s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=============================>                    | 38/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==============================>                   | 39/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===============================>                  | 40/64 - 1.64s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================================>                 | 41/64 - 1.61s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |================================>                 | 42/64 - 1.67s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=================================>                | 43/64 - 1.71s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================>               | 44/64 - 1.70s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================================>              | 45/64 - 1.65s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |===================================>              | 46/64 - 1.72s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |====================================>             | 47/64 - 1.66s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |=====================================>            | 48/64 - 1.74s/it[DEBUG] ggml_extend.hpp:1026 - vae compute buffer size: 416.00 MB(VRAM)
  |==================================================| 64/64 - 1.74s/it
[DEBUG] stable-diffusion.cpp:1045 - computing vae [mode: DECODE] graph completed, taking 83.93s
[INFO ] stable-diffusion.cpp:1416 - latent 1 decoded, taking 83.93s
[INFO ] stable-diffusion.cpp:1420 - decode_first_stage completed, taking 83.93s
[INFO ] stable-diffusion.cpp:1539 - txt2img completed in 346.14s
save result image to 'output.png'

And for visual comparison... First, here is an SD 1.5 image generated with non-tiled VAE: sd-cpp-vulkan_cat_output

And here is how it comes out with tiled VAE: sd-cpp-vulkan_vae-tiled_cat_output

The colors look a little different if you flip between the two, but I don't see an obvious boundary in the tiled version, FWIW.

lostdisc avatar Jan 10 '25 03:01 lostdisc

I also noticed that, but have not yet found bad result : my images are correct. May be it is only a simple error in % calculation ? (because of a bad total ?)

olivbrau avatar Jan 10 '25 16:01 olivbrau

I'm pretty sure https://github.com/leejet/stable-diffusion.cpp/pull/484 fixes that. This was not the initial goal of the PR, but I can't reproduce this issue with these changes merged in.

stduhpf avatar Jan 10 '25 16:01 stduhpf

Neat, thanks!

lostdisc avatar Jan 10 '25 16:01 lostdisc