太郎君

我这边处理 Obama和 Obama2，提示 Traceback (most recent call last): File "/home/tailangjun/Documents/AIGenHuman/2DHuman/GeneFace/data_gen/nerf/binarizer.py", line 277, in binarizer.parse(hparams['video_id']) File "/home/tailangjun/Documents/AIGenHuman/2DHuman/GeneFace/data_gen/nerf/binarizer.py", line 267, in parse ret = load_processed_data(processed_dir) File "/home/tailangjun/Documents/AIGenHuman/2DHuman/GeneFace/data_gen/nerf/binarizer.py", line 98, in load_processed_data coeff_dict...

我查了一下代码，vid_coeff.npy是第9步生成的，说明那一步执行失败了 9. calculate 3DMM python data_gen/nerf/extract_3dmm.py --video_id=$VIDEO_ID 我手动再执行一遍，会得到如下的错误 python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/opt/anaconda3/envs/geneface/lib/python3.9/site-packages/torchvision/image.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE'If you don't plan on using image functionality from `torchvision.io`, you...

问题我搞清楚了，本质问题是 torch、torchversion、pytorch3d的版本不匹配 按照 pytorch3d官网的说法，目前仅仅支持这些情况 Linux or macOS or Windows Python 3.8, 3.9 or 3.10 PyTorch 1.10.0, 1.10.1, 1.10.2, 1.11.0, 1.12.0, 1.12.1, 1.13.0, 2.0.0 or 2.0.1. 具体的安装方法可参考 https://github.com/facebookresearch/pytorch3d/issues/1401

我这边可以成功生成 vid_coeff.npy了 $ python data_gen/nerf/extract_3dmm.py --video_id=$VIDEO_ID loading the model from deep_3drecon/checkpoints/facerecon/epoch_20.pth loading video ... extracting 2D facial landmarks ...: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 6455/6455 [02:07

> 不需要了，下载新版本使用 意思是我更新一下代码就可以了，是吗？

更新后，可以下载了，下载还挺快，真不错

将 lm3d_radnerf_torso_sr.yaml中的eye_blink_dim改为2就可以了

感觉是看控制台中的 total_loss | Validation results@18000: {'total_loss': 26.5918741798, 'mse_loss': 0.0003416548, 'sr_mse_loss': 0.0004718732, 'lambda_ambient': 26.5910606384} 03/16 01:51:34 PM Epoch 00000@18000: saving model to checkpoints/motion2video_nerf/Obama_head/model_ckpt_steps_18000.ckpt 03/16 01:51:34 PM Delete ckpt: model_ckpt_steps_16000.ckpt Valid: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|...

另外发现 PM Delete ckpt只保留了最后一个，一不注意最小的那个就被删掉了

这个是mead-512的效果 https://github.com/semchan/HyperLips/assets/12316965/bccbbb61-7b9c-4911-ac72-dbc63666ca6b