Fixed model transformation doubt

[javierrodenas]

Hello

I have a doubt about how fixed_model_transform works. As far as I could understand, it is applied to center and align the object. This alignment with respect to the camera?

I saw this note in your website : Note: Like the fixed_model_transform, the pose_transform_permuted is actually the transpose of the matrix. Moreover, after transposing, the columns are permuted, and there is a sign flip (due to UE4's use of a lefthand coordinate system). Specifically, if A is the matrix given by pose_transform_permuted, then actual transform is given by A^T * P, where ^T denotes transpose, * denotes matrix multiplication, and the permutation matrix P is given by

    [ 0  0  1]
P = [ 1  0  0]
    [ 0 -1  0]

How does UE4 work? I mean, how is it transforming the object to center and align? Where fixed_model_transform and pose_transform are supposed to invervene? How are they combined?

Thank you in advance,

[mintar]

Just to clarify: This is a question about the JSON dataset format used by the Falling Things (FAT) dataset, NDSS and DOPE (scripts/train.py). Correct?

To answer your questions:

• I've written down a documentation of the JSON format here; is that where you were quoting from?
• Just ignore pose_transform_permuted. I believe it's a leftover from somewhere, but it is never used inside DOPE. The fixed_model_transform and the pose_transform_permuted can be converted into each other using the paragraph you quoted above. There is no extra information in pose_transform_permuted, and fixed_model_transform is much nicer to work with.

Could you give some more context what you are trying to accomplish? What dataset are you using (FAT, self-generated dataset using NDSS, ...)? What problem are you trying to solve using fixed_model_transform?

[javierrodenas]

Thanks for your feedback. Let me explain a little bit my confusion.

First of all, I am using a self-generated dataset using NDDS.

Object settings:

{
	"exported_object_classes": [
		"TESTEE"
	],
	"exported_objects": [
		{
			"class": "TESTEE",
			"segmentation_class_id": 255,
			"segmentation_instance_id": 16777214,
			"fixed_model_transform": [
				[ 0, 0, 1, 0 ],
				[ -1, 0, 0, 0 ],
				[ 0, -1, 0, 0 ],
				[ 0, 15, 0, 1 ]
			],
			"cuboid_dimensions": [ 100, 30, 80 ]
		}
	]
}

This is my original pose:

From this, I generated the folllowing sample:

The sample rotation is (-180º, -90º, 0º) -> (x, y, z)

Quarternion and pose transform:
			"quaternion_xyzw": [ 0, -0.70709997415542603, -0.70709997415542603, 0 ],
			"pose_transform": [
				[ 0, 1, 0, 0 ],
				[ -1, 0, 0, 0 ],
				[ 0, 0, -1, 0 ],
				[ 64.268898010253906, 18.540000915527344, 378.285888671875, 1 ]

My confusion comes when I try to get the original pose using the pose_tranform or fixed_transform_model, As I can see, the two matrices are different and I tried to convert this matrices to degrees and I am not able to get the correct rotation. How is it multiplied? Which of the two matrices are used by NDSS? Another point is how the quaternion are applied.

Thank you in advance!

[TontonTremblay]

We wrote these exported_objects to explain how an original model is transformed, e.g., we changed the center of the YCB objects to make them more tight. I will ask Thang to check out this issue as well.