Chameleon image generation low quality.

[Mr-Loevan]

System Info

transformers 4.43.0 dev

Who can help?

@zucchini-nlp

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

Using the model.generate function within the transformers framework to generate an image, in comparison to the inference pipeline from https://github.com/GAIR-NLP/anole, results in a relatively lower quality image.

Besides I take a test on some image generation benchmark, and get consistent results.

Expected behavior

1. Could you please provide a script to convert Hugging Face weights to PyTorch weights, so that we can use them with the Meta inference pipeline?
2. Could you kindly outline the best sampling parameters in the doc for generating high-quality images?

[zucchini-nlp]

Hey @Mr-Loevan ,

Chameleon currently doesn't support image generation as it wasn't releaseed by Meta. As you mentioned, Anole has tuned the model for image generation, but it is still in progress and will be added to the library soon. You can track the progress here (https://github.com/huggingface/transformers/pull/32013)

Regarding the conversion of weights, we don't have a script to convert HF-format to Meta-format. I am not sure why you would need that tbh, afaik meta pipeline neither supports image generation