diffusion-rig icon indicating copy to clipboard operation
diffusion-rig copied to clipboard

Controlling the eye pose

Open atgao opened this issue 2 years ago • 1 comments

Inside the inference script, we can control the pose with here. Is it also possible to control or condition on the eye pose since FLAME has controllable eye joints? I tried to modify the eye pose in the decoding step https://github.com/adobe-research/diffusion-rig/blob/d9b266102e1900032efc62e66a7f6c66dde5ff79/decalib/models/FLAME.py#L189 but it appears that the eye pose is still the same

atgao avatar Oct 26 '23 20:10 atgao

Since we utilize DECA to obtain physical buffers but DECA does not output eye_pose parameters, the model could not learn to control the eye pose. FLAME would use a default eye pose if the parameters are not provided. As you can see in the teaser figure, the eye poses of the physical buffers are neutral regardless of the inputs.

zh-ding avatar Oct 27 '23 21:10 zh-ding