Interactive exploration using Stylegan2 model config-e

[samuelpietri]

Hi, I'm trying to test the interactive.py script with different stylegan2 models. The one I'm using is coming from the original config-e training and I managed to convert it for Pytorch using the modified version of the convert_weight script implemented here https://github.com/rosinality/stylegan2-pytorch/blob/master/convert_weight.py . When I try to use the converted model I get mismatch error for the tensor sizes: ... size mismatch for to_rgbs.7.conv.modulation.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([32]). If I force the channel_multiplier=1 when the model generator is loaded in wrappers.py inside the Stylegan2 class : self.model = stylegan2.Generator(self.resolution, 512, 8, channel_multiplier=1).to(self.device) I get a different error RuntimeError: Error(s) in loading state_dict for Generator: Unexpected key(s) in state_dict: "noises.noise_0", "noises.noise_1", "noises.noise_2", "noises.noise_3", "noises.noise_4", "noises.noise_5", "noises.noise_6", "noises.noise_7", "noises.noise_8", "noises.noise_9", "noises.noise_10", "noises.noise_11", "noises.noise_12", "noises.noise_13", "noises.noise_14", "noises.noise_15", "noises.noise_16". Is there any way to implement this use case?

[harskish]

Hi Samuel,

The fork of stylegan2 that's used by the project (https://github.com/harskish/stylegan2-pytorch) is slightly older than Rosinality's official version (https://github.com/rosinality/stylegan2-pytorch) and doesn't handle the noise inputs that you're seeing in the error. You could either update the stylegan2 fork to match Rosinality, or re-convert the checkpoint using the fork instead of the official repo.

I haven't tried to use other than the f-configs of StyleGAN2, so if you get it working please let me know. I'd be happy to accept a pull request on any necessary code changes.

[harskish]

The readme should make the mismatch explicit. I'll look into updating the readme (and potentially the SG2 fork).

[samuelpietri]

Thank you Erik for the response, I'll try both solutions and I'll let you know what comes out.

[woctezuma]

I follow this notebook: https://colab.research.google.com/drive/1g-ShMzkRWDMHPyjom_p-5kqkn2f-GwBi

I try to use a PyTorch 256x256 config-e model, which I have converted with the flag --channel_multiplier=1 and put here:

/content/ganspace/models/checkpoints/stylegan2/stylegan2_steam_256.pt

I have implemented the fixes mentioned above in my fork.

I then define a few variables before I try to run the important parts of the notebook:

model_name = 'StyleGAN2' 
model_class = 'steam'
num_components = 80

Then I run this:

!python visualize.py --model $model_name --class $model_class --use_w

and encounter this error:

[21.09 15:16] StyleGAN2, g_mapping, ipca
Layer 'g_mapping' not found in model!
Available layers: 
style
style.0
style.1
style.2
[...]
strided_style.11
strided_style.12
strided_style.13
Traceback (most recent call last):
  File "visualize.py", line 152, in <module>
    inst = get_instrumented_model(args.model, args.output_class, layer_key, device, use_w=args.use_w)
  File "/usr/lib/python3.6/functools.py", line 807, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/content/ganspace/models/wrappers.py", line 715, in get_instrumented_model
    raise RuntimeError(f"Unknown layer '{layer_name}''")
RuntimeError: Unknown layer 'g_mapping''

However, this works:

!python visualize.py --model $model_name --class $model_class --use_w --layer=style -c $num_components

Any idea how to fix the error? Although it seems I can use the rest of the code just fine, I would like to remove every error.

I think that is because:

• there is an issue with layer g_mapping,
• the first command omits the --layer flag, which makes the program try to list all the layers, including the faulty one:

  --layer      layer at which to perform PCA; leave empty to list options

[harskish]

@woctezuma a few comments:

• g_mapping is just the default value for the layer parameter. I guess making the default None would be more in line with the documentation
• The first error isn't really an error - it just tells you that you need to specify a correct layer to analyze, since the default (g_mapping) is invalid for StyleGAN2.
• The notebook you linked isn't the official one (accessed via the readme), although I think the official notebook is basically identical.
• Justin Pinkney's fork (https://github.com/justinpinkney/ganspace/) has a few config-e related changes, if you run into more issues you can look there for inspiration!

[woctezuma]

Thank you for your reply.

It was really not obvious to me that the default value (g_mapping) was a dummy name, especially as it appears also here:

https://github.com/harskish/ganspace/blob/2480494ab07e7dd9ca1adc260c6208686504136c/visualize.py#L224-L228

I have been using this unofficial notebook, because I encounter an OpenGL issue with interactive.py on Google Colab, and this unofficial notebook only makes use of visualize.py, plus some code in a notebook cell to work around interactive.py.

So far so good, I get results with your method which are similar (but more manageable) than with "closed-form factorization".

I will look at Justin Pinkney's fork, just in case.

[harskish]

g_mapping is not a dummy value per se, but rather a default layer name that exists in StyleGAN1 (but not StyleGAN2). As for the Colab notebook, I was referring to https://colab.research.google.com/github/harskish/ganspace/blob/master/notebooks/Ganspace_colab.ipynb (which is pretty much identical to the one you've been using)

[woctezuma]

Right. Thanks. I did not realize it was the same!