ControlNet icon indicating copy to clipboard operation
ControlNet copied to clipboard

How to train without text prompt?

Open aartykov opened this issue 3 years ago • 8 comments

Hey, thanks for sharing such an excellent work. How can I train without text prompt?

aartykov avatar Feb 24 '23 15:02 aartykov

I have the same question here!

chokevin8 avatar Feb 24 '23 20:02 chokevin8

I saw in the paper.The empty string might be an unexpected input for the model, and SD tends to generate random texture maps if no prompt is provided. A better setting is to use meaningless prompts like “an image”, “a nice image”, “aprofessional image”

15539793373 avatar Mar 01 '23 08:03 15539793373

And I would like to know whether the authors trained the models using these meaningless prompts. The paper only mentioned that they used the default prompts to test the model. It seems that we don't know the prompts they used in the training processes.

According to the training tutorial https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md, the prompts used for training actually describe the image content, such as "a xxx color circle in some other color background."

Could the authors provide more details about the prompts used for training? It would be helpful for us to train the ControlNet on our private dataset. Thanks!

yukang123 avatar Mar 03 '23 07:03 yukang123

我想作者使用毫毫无意义的训练。。论文论文只提到提到他们他们他们使用使用默认提示提示来来模型模型。。。似乎似乎我们我们

根据培训教程https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md, 用于培训的提示实践上描述了图像内容,例如“其他颜色背景中的xxx色圈”

作者能否提供有关培训的更多详细信息?这对我们在私有数据集上培训ControlNet很有帮助。谢谢!

I use the “”“” prompt in own private Semantic segmentation dataset to train ControlNet,and I think it look’s very nice than other GAN models.

15539793373 avatar Mar 03 '23 08:03 15539793373

Same question too!

WooXinyi avatar Apr 25 '23 07:04 WooXinyi

I have quite similar question. Do they drop the 50% of the text prompts during the training process?

engrmusawarali avatar Jun 11 '23 10:06 engrmusawarali

all duplicates about "dropping prompts" https://github.com/lllyasviel/ControlNet/issues/93 https://github.com/lllyasviel/ControlNet/issues/160 https://github.com/lllyasviel/ControlNet/issues/246 https://github.com/lllyasviel/ControlNet/issues/422 https://github.com/lllyasviel/ControlNet/issues/506

geroldmeisinger avatar Sep 17 '23 11:09 geroldmeisinger

And I would like to know whether the authors trained the models using these meaningless prompts. The paper only mentioned that they used the default prompts to test the model. It seems that we don't know the prompts they used in the training processes.

According to the training tutorial https://github.com/lllyasviel/ControlNet/blob/main/docs/train.md, the prompts used for training actually describe the image content, such as "a xxx color circle in some other color background."

Could the authors provide more details about the prompts used for training? It would be helpful for us to train the ControlNet on our private dataset. Thanks!

Have you got answer? i am wondering whether they used default prompt or meaningless prompt during training? @yukang123

KhawlahB avatar Dec 13 '23 01:12 KhawlahB