ml-agents icon indicating copy to clipboard operation
ml-agents copied to clipboard

Retrieving the range of the state space from python.

Open Odalric-Ambrym opened this issue 4 years ago • 1 comments

Is your feature request related to a problem? Please describe.

In the class UnityToGymWrapper(gym.Env), there is a part to build the gym world from unity. Regarding the states, the follwoing is written: if self._get_vec_obs_size() > 0: # vector observation is last high = np.array([np.inf] * self._get_vec_obs_size()) list_spaces.append(spaces.Box(-high, high, dtype=np.float32))

This produces state space with infinite boxes. In Unity, I can specify an environment with delimited state, not an infinite box. And this can be specified using parameters. Also, this is not convenient when encoding a state space that is e.g. a multi-dimensional discrete grid (say 2-dim grid-world). Indeed the only way right now seems to proceed using one-hot encoding, but in that case we cannot recover the dimensions of the grid, only the total number of states.

Describe the solution you'd like How can I retrieve this information ? I would be happy with just passing an addtional vector of parameter from unity to gym. This way, I could specify the appropriate state space, and not the default infinite box.

Describe alternatives you've considered I haven't find a way to get other parameters passed from unity to gym. I was expecting to find such somewhere in the code. Now for discrete grid-world, I have to go through one-hot encoding and back. Again, getting one vector of parameter (such as size of the grid in each dimension) would be ok, but I could not find where ot get such.

Additional context

Odalric-Ambrym avatar Jul 07 '21 09:07 Odalric-Ambrym

Hmm, this isn't something that's currently supported. If I understand correctly, you'd want to constrain the observation in Unity, and have the Box that's returned have those values?

I think it's something worth considering, but to do so in a general way would mean having an explicit range specified on the Unity side. For most of the Sensors (e.g. RayCast) this should be between -1 and 1. We'd have to add something manual to the VectorSensor.

ervteng avatar Jul 22 '21 01:07 ervteng