Which observables are in world frame?

[Soha18]

Hi,

Thanks for opening this repository.

I'm trying to implement lift task on real Panda robot after training a model on the simulator (robosuite), and need to know for sure about the observations and actions coordinates in the simulator. (I'm using OSC-POSE controller)

• Are 'cube_pos', 'cube_quat' ,'eef_pos' and 'eef_quat' in the observations are in world frame? (when we get observations from observations = env.reset() for example)
• The output action vector contains delta position and delta axis angles, and it is used to calculate the target pose of the end-effector, do I need to get this resulting eef pose in base frame to feed it to the controller? or it is already in base frame? it is really confusing, please note this:

from robosuite.models import grippers
import robosuite as suite
import gym
import numpy as np
import os

from robosuite.environments.base import register_env
from robosuite import load_controller_config
from robosuite.wrappers import GymWrapper
import robosuite.utils.transform_utils as T

def make_env():
    controller_config = load_controller_config(default_controller="OSC_POSE")
    env = suite.make(
        env_name="Lift", # try with other tasks like "Stack" and "Door"
        robots="Panda",  # try with other robots like "Sawyer" and "Jaco"
        reward_shaping=True,
        has_renderer=True,
        render_camera = 'agentview',
        has_offscreen_renderer=True,
        controller_configs=controller_config,
        use_camera_obs=False,
        horizon = 100,
        use_object_obs= True
    )
    env = GymWrapper(env)
    return env
env = make_env()
obs = env.reset()

print("eef_in_obs", env._get_observations()['robot0_eef_pos'], env._get_observations()['robot0_eef_quat'])
eef_pos = env._eef_xpos
eef_quat= env._eef_xquat
print("eef in world", eef_pos,eef_quat)
print("hand_pose",T.mat2pose(env.robots[0]._hand_pose)) # supposed to be hand in base frame
print("hand_pose_in_base",T.mat2pose(env.robots[0].pose_in_base_from_name('robot0_right_hand')))

the output is like this:

eef_in_obs [-0.10601769 -0.00338632  0.99875861  0.99835936  0.02188318  0.05285238
  0.00251831]
eef in world [-0.10601769 -0.00338632  0.99875861] [0.6904729  0.7214205  0.03559157 0.03915299]
hand_pose (array([ 0.44373508, -0.00312295,  0.18321546]), array([0.9983594 , 0.02188317, 0.05285237, 0.00251831], dtype=float32))
hand_pose_in_base (array([ 0.44373508, -0.00312295,  0.18321546]), array([0.9983594 , 0.02188317, 0.05285237, 0.00251831], dtype=float32))

Please consider my query.. Thanks a lot and I appreciate any help

[zhuyifengzju]

Hi, the positions and orientations of objects and end effectors are considered in the world frame. For osc controllers, we the default action space is the delta poses. If you do not set is_delta to false, you do not need to consider the transformation of coordinates for the action space. If you are doing motion planning and needs to following a certain trajectory in the world frame, you can first compute the trajectory of waypoints in the world frame, and compute the delta poses, which will be the action for your robot follow.