UPC-ViRVIG
Could you share the evaluate code for AMASS?
I am currently working with the AMASS dataset and have encountered issues with the pre-trained model not performing as expected. I am seeking assistance to properly evaluate the model and, if possible, obtain any additional evaluation code that might be available.
Here are the results I obtained on HUMAN4D by simulating your source code . Evaluate Loss: 0.1619343272083112 Mean Per Joint Position Error: 0.06029307960557777 Mean End Effector Position Error: 0.10164124760273341
Here's the key code I tried.
def eval_pos_error(result, means, stds, device):
res, offsets, filename = result
gt_rots, gt_pos, gt_parents, gt_offsets = get_info_from_npz(filename)
gt_rots = torch.from_numpy(gt_rots).float().to(device).unsqueeze(0)
gt_pos = torch.zeros(gt_rots.shape[0], gt_rots.shape[1], 3)
gt_offsets = torch.from_numpy(gt_offsets).float().to(device)
res = res.permute(0, 2, 1)
res = res.flatten(0, 1)
res = res.cpu().detach().numpy()
dqs = res
dqs = dqs * stds["dqs"].cpu().numpy() + means["dqs"].cpu().numpy()
dqs = dqs.reshape(dqs.shape[0], -1, 8)
# Get rotations and translations from dual quaternions
_, rots = from_root_dual_quat(dqs, np.array(gt_parents))
rots = torch.from_numpy(rots).float().to(device).unsqueeze(0)
pos = torch.zeros(rots.shape[0], rots.shape[1], 3)
gt_joint_poses, _ = fk(gt_rots, gt_pos, gt_offsets, torch.Tensor(gt_parents).long().to(device))
joint_poses, _ = fk(rots, pos, gt_offsets, torch.Tensor(gt_parents).long().to(device))
# show_3d_vs(joint_poses, gt_joint_poses)
# Error
error = torch.norm(
joint_poses - gt_joint_poses[:, : joint_poses.shape[1], ...], dim=-1
)
sparse_error = error[:, :, param["sparse_joints"][1:]] # Ignore root joint
return torch.mean(error).item(), torch.mean(sparse_error).item()
def get_info_from_npz(filename):
"""
return:
All return values are numpy arrays.
rots: (frames, n_joints, 4) represents the rotation as a unit quaternion.
pos: (frames, 3) represents the root node translation.
parents: The parent-child relationship in the xsens skeleton structure.
offsets: Shape parameters from the SMPL model, distinguishing only between male and female.
"""
s2x_map = np.array(S2X_map) # Mapping relationship between SMPL and xsens skeletons (parents)
motion_data = np.load(filename)
gender = motion_data['gender'].item().upper()
rots = motion_data['poses'].reshape(motion_data['poses'].shape[0], 52, 3)
pos = motion_data['trans']
# Get skeleton graph from the SMPL model file (offsets)
smpl_name = '../npz/SMPL_' + gender + '.npz'
smpl_model = np.load(smpl_name)
root0J = smpl_model["J"] - smpl_model["J"][0]
kintree_table = smpl_model["kintree_table"]
# Calculate offsets
offsets = np.zeros_like(root0J)
for i in range(1, root0J.shape[0]):
parent_idx = kintree_table[0, i]
offsets[i] = root0J[i] - root0J[parent_idx]
# Convert SMPL structure data to the format required for training
offsets = offsets[s2x_map, :]
rots = rots[:, s2x_map, :]
parents = xsens_parents
# Convert rots from Euler angles to quaternions
rot_order = np.tile(['x', 'y', 'z'], (rots.shape[0], len(parents), 1))
rots = quat.unroll(
quat.from_euler(rots, order=rot_order),
axis=0,
)
rots = quat.normalize(rots) # Make sure all quaternions are unit quaternions
return rots, pos, parents, offsets
I am currently working with the AMASS dataset and have encountered issues with the pre-trained model not performing as expected. I am seeking assistance to properly evaluate the model and, if possible, obtain any additional evaluation code that might be available.
Here are the results I obtained on HUMAN4D by simulating your source code . Evaluate Loss: 0.1619343272083112 Mean Per Joint Position Error: 0.06029307960557777 Mean End Effector Position Error: 0.10164124760273341
The outcome is derived from the generator model, specifically the 'dancedb/generator.pt' model paired with the 'data.pt' .
Hi! I appreciate your interest in our project.
One question: are you using the model_xsens or model_dancedb? For testing on HUMAN4D, we trained on DanceDB, which is included in AMASS. The pre-trained model can be found at: https://github.com/UPC-ViRVIG/SparsePoser/tree/main/python/models/model_dancedb
Hi! I appreciate your interest in our project.你好！感谢您对我们项目的关注。
One question: are you using the model_xsens or model_dancedb?一个问题：您是在使用model_xsens还是model_dancedb？ For testing on HUMAN4D, we trained on DanceDB, which is included in AMASS. The pre-trained model can be found at: https://github.com/UPC-ViRVIG/SparsePoser/tree/main/python/models/model_dancedb为了在 HUMAN4D 上进行测试，我们在 AMASS 中包含的 DanceDB 上进行了训练。预训练模型位于： https://github.com/UPC-ViRVIG/SparsePoser/tree/main/python/models/model_dancedb
I have re-downloaded your model and thoroughly reviewed my code, but I'm still getting the same results. I'm quite confused about this. Could you please help me check my code, or provide the code you used to evaluate the AMASS dataset? I would greatly appreciate it! You can reach me at [email protected]. Thank you very much!
As a supplement，here is my main code:
def main(args):
# Set seed
torch.manual_seed(param["seed"])
random.seed(param["seed"])
np.random.seed(param["seed"])
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Using device:", device)
# Additional Info when using cuda
if device.type == "cuda":
print(torch.cuda.get_device_name(0))
# Prepare Data
eval_dir = args.data_path
# check if train and eval directories exist
if not os.path.exists(eval_dir):
raise ValueError("eval directory does not exist")
eval_files = []
for root, dirs, files in os.walk(eval_dir):
for file in files:
# 检查文件是否以.npz结尾，且文件名不是shape.npz
if file.endswith('.npz') and file != 'shape.npz':
full_path = os.path.join(root, file)
eval_files.append(full_path)
eval_dataset = TestMotionData(param, 1, device)
# Eval Files
for filename in eval_files:
if filename[-4:] == ".npz":
rots, pos, parents, offsets = get_info_from_npz(filename)
# Eval Dataset
eval_dataset.add_motion(
offsets,
pos, # only global position
rots,
parents,
filename
)
print(eval_dataset.get_len(), " added to eval_dataset")
# Create Models
train_data = Train_Data(device, param)
generator_model = Generator_Model(device, param, xsens_parents, train_data).to(device)
# Load Model
means, stds = load_model(generator_model, args.model_dir, train_data, device)
eval_dataset.set_means_stds(means, stds)
eval_dataset.normalize()
results = evaluate_generator(generator_model, train_data, eval_dataset)
mpjpe, mpeepe = eval_result(results, means, stds, device)
evaluation_loss = mpjpe + mpeepe
print("Evaluate Loss: {}".format(evaluation_loss))
print("Mean Per Joint Position Error: {}".format(mpjpe))
print("Mean End Effector Position Error: {}".format(mpeepe))
I run it using "python eval.py ../HUMAN4D ../models/model_dancedb/generator.pt "
I quickly checked your code and it looks good to me. However, before having a more in-depth look, I suggest the following:
- Try to evaluate the code on DanceDB since it was trained on that DB (it is also found in the AMASS DB). The error should be quite small. If that's not the case, then there is probably a bug somewhere.