AUTOMATIC1111
[Bug]: Error during generation when gen size lower than mask or padding size
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
What happened?
An error occurs near the end of any inpainting generation where the masked area + padding exceeds generation size despite a seemingly successful inpainting stage, as seen with preview images.
The pertinent variables:
- Img2img; mask mode set to inpaint masked; inpaint mode set to inpaint only masked area
- Generation size (width/height) and its ratio to the image
- Masked area size
- Only masked padding pixels, as they get added to the final generation size
Steps to reproduce the problem
- In img2img tab, set to inpaint any sufficiently sized image
- Set to inpaint only masked
- Mask an area larger than set width/height OR set padded area to size larger than either the width or size of the masked area OR ensure mask size + padded area exceeds generation size
- Generate (with preview image)
- Receive vector size mismatch error during last VAE stage of generation.
What should have happened?
A successful inpainting generation. If it's intended behavior, I would at least expect a more helpful error ("Mask too large, increase parameters") instead of a vector mismatch.
Commit where the problem happens
https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/3e0f9a75438fa815429b5530261bcf7d80f3f101
What platforms do you use to access the UI ?
Windows
What browsers do you use to access the UI ?
Mozilla Firefox
Command Line Arguments
Tested without.
List of extensions
Tested without.
Console logs
Error completing request█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:03<00:00, 4.61it/s]
Arguments: ('task(fsrob0flr3w0zb2)', 2, 'masterpiece, best quality, a', 'a', [], None, None, {'image': <PIL.Image.Image image mode=RGBA size=1059x1029 at 0x23B3BFDA50>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=1059x1029 at 0x23B3BFFB50>}, None, None, None, None, 20, 10, 4, 0, 1, False, False, 1, 1, 14, 1.5, 0.75, 4149673344.0, -1.0, 0, 0, 0, False, 512, 128, 0, 1, 32, 0, '', '', '', [], 0, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True,'', '', True, 50, True, 1, 0, False, 4, 1, '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', 0, '', 0, '', True, False, False, False) {}
Traceback (most recent call last):
File "J:\AI\stable-diffusion-webui\modules\call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "J:\AI\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "J:\AI\stable-diffusion-webui\modules\img2img.py", line 169, in img2img
processed = process_images(p)
File "J:\AI\stable-diffusion-webui\modules\processing.py", line 486, in process_images
res = process_images_inner(p)
File "J:\AI\stable-diffusion-webui\modules\processing.py", line 673, in process_images_inner
image = apply_overlay(image, p.paste_to, i, p.overlay_images)
File "J:\AI\stable-diffusion-webui\modules\processing.py", line 69, in apply_overlay
image = images.resize_image(1, image, w, h)
File "J:\AI\stable-diffusion-webui\modules\images.py", line 283, in resize_image
resized = resize(im, src_w, src_h)
File "J:\AI\stable-diffusion-webui\modules\images.py", line 266, in resize
im = upscaler.scaler.upscale(im, scale, upscaler.data_path)
File "J:\AI\stable-diffusion-webui\modules\upscaler.py", line 63, in upscale
img = self.do_upscale(img, selected_model)
File "J:\AI\stable-diffusion-webui\modules\esrgan_model.py", line 154, in do_upscale
img = esrgan_upscale(model, img)
File "J:\AI\stable-diffusion-webui\modules\esrgan_model.py", line 225, in esrgan_upscale
output = upscale_without_tiling(model, tile)
File "J:\AI\stable-diffusion-webui\modules\esrgan_model.py", line 204, in upscale_without_tiling
output = model(img)
File "J:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "J:\AI\stable-diffusion-webui\modules\esrgan_model_arch.py", line 61, in forward
return self.model(feat)
File "J:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "J:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\container.py", line 204, in forward
input = module(input)
File "J:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "J:\AI\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 182, in lora_Conv2d_forward
return lora_forward(self, input, torch.nn.Conv2d_forward_before_lora(self, input))
File "J:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "J:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4 channels instead
Additional information
No response
I'm also getting this problem. I never used to get this when I was doing inpainting a while ago (back then it was called "inpaint at full resolution" instead of "only masked") has something changed since then? This makes it annoying to inpaint high resolution pictures because the mask can only cover a very small area.
I've done a git bisect and found that it seems like the error was introduced in this commit: 2e8b5418e3cd4e9212f2fcdb36305d7a40f97916
Commits before that I can successfully inpaint larger areas.
It works on latest commit if you comment out that blendLayers line
I can confirm that this commented line allowed me to inpaint with larger masks.
Should be closed by https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12480
