expose ff_multiplier argument of Transformer

[kashif]

This PR exposes the ff_multiplier option of the Attention so that the UNet is able to set it.

cc @sayakpaul

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[kashif]

I believe the wrapper function will register all the arguments to the config and since now ff_multiplier is there it should also register it.

[kashif]

@patrickvonplaten no this was to make smaller student unets for the distillation experiments

[williamberman]

@kashif is the intention to only add the config to the feedforward of the middle block of the unet or to all down and up blocks that use a transformer for their cross attention mechanism?

As is, I see we also add the config to a few other block in the unetblocks2d but because the ff_multiplier isn't passed through get_{down,up}_block, they won't receive the config.

[kashif]

for now, we wanted it for all attention feedforward blocks... but yes middle ones would also work i suppose. let me fix the get_ helpers...

[patrickvonplaten]

I'd say we only merge this PR if we have a powerful distilled SD model - otherwise no need yet to expose more parameters IMO

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[williamberman]

@kashif should we close this for now? I'm ok with revisiting in the future

[kashif]

yes please! i only needed it for my experiments. closing!