diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

Is it possible to train a model for an anime character?

Open RayZ3R0 opened this issue 3 years ago • 1 comments

I'm not really good at this stuff but is it possible to train a dreambooth model for an anime character? I tried just using the default stuff but the results are horrible

RayZ3R0 avatar Nov 19 '22 07:11 RayZ3R0

There's quite a few things that could go wrong if using the default settings:

Do you have sufficient images? 10-15 should be enough, and they should be varied, but all "good" images. Have you made sure your dataset is all 1:1 resolution? All your images should be square. Center crop is a poor replacement for manually making the images 1:1, because they will be poorly cropped. Is the learning rate too high? I use 3.0e-7 to 6.0e-7 for 4000 steps (on an anime themed base model). Lower learning rate and more steps is typically better, but obviously takes longer.

Also, do make sure you're using an anime themed base model. Base Stable Diffusion has so much of the model not intended for anime, and you'll have to craft much better prompts in order to get even anime style results, let alone good results. Anime models are usually trained on booru tags, so you can just use those in the prompts and get good results.

toomanydev avatar Nov 20 '22 12:11 toomanydev