mbaske
gridsensor for release2?
Hi, it's going to be a few more days... I got the code changes mostly done (it's a pretty major update), now I need to test it, retrain the examples and update the description.
i'm super excited and looking forward to it !! (never used gridsensor so it will be a huge deal for me :))
I've posted the updates to the dev branch https://github.com/mbaske/grid-sensor/tree/dev Still need to write the docs though, as a couple of things have changed. Most notably:
- Settings are grouped into foldouts.
- Debugging and visualization options are now integrated in the "Debug" foldout.
- There are no longer 2D/3D variants of the DetectableGameObject. Instead, its shape scanning options have a "Flatten" toggle for creating 2D points.
- Collider shape scanning uses context sensitive LODs.
- The "Scan at runtime" on-demand option was removed in favour of bulk scanning upfront. If there are a lot of objects that need to be scanned at runtime, like the random asteroids in the "Dogfight" example, this is done at initialization and can take a moment.
- Added a new minimal GridSensorComponent that doesn't do any detection by itself, see "MazeExplorer" example.
hello @mbaske thanks for your update, i worked until 1 am last night and today i took the day off.
i will check your branch tomorrow and figure out how to use grindsensor at all, since i've never used it, hence I cannot create a formed opinion on these points
- There are no longer 2D/3D variants of the DetectableGameObject. Instead, its shape scanning options have a "Flatten" toggle for creating 2D points.
- Collider shape scanning uses context sensitive LODs.
- The "Scan at runtime" on-demand option was removed in favour of bulk scanning upfront. If there are a lot of objects that need to be scanned at runtime, like the random asteroids in the "Dogfight" example, this is done at initialization and can take a moment.
- Added a new minimal GridSensorComponent that doesn't do any detection by itself, see "MazeExplorer" example.
- What are context sensitives LOD?
- So just to understand the pipeline the objects that will be scanned are pre-initialized somehow inside the sensor? (bulk scanning point)
- What do you mean that the minimal GSC doesnt do any detection "by itself"?
I have these cubes around my object (lets call them collectibles), and I added to each cube (voxel) a Detectable Game Object script, but I cannot seem to be able to use scan shape.
- What is the Scan LOD and Gizmo LOD for ? I tried to increase that but it becomes "--".
Update: Solved by putting the Detectable Object script in the GameObject child of my Collectible, that had the Collider component.
Also, for some reason Barracuda is throwing me this error, I will try to solve it
IndexOutOfRangeException: Index was outside the bounds of the array.
Unity.Barracuda.Tensor.set_Item (System.Int32 b, System.Int32 h, System.Int32 w, System.Int32 ch, System.Single value) (at C:/Users/franc/Downloads/unity installed packages/barracuda-release-1.4.0-preview/Barracuda/Runtime/Core/Tensor.cs:2105)
Unity.MLAgents.Sensors.ObservationWriter.set_Item (System.Int32 h, System.Int32 w, System.Int32 ch, System.Single value) (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Sensors/ObservationWriter.cs:126)
MBaske.Sensors.Grid.GridSensor.Write (Unity.MLAgents.Sensors.ObservationWriter writer) (at Assets/MBaske/Scripts/Sensors/Grid/Basic/Sensor/GridSensor.cs:156)
Unity.MLAgents.Inference.ObservationGenerator.Generate (Unity.MLAgents.Inference.TensorProxy tensorProxy, System.Int32 batchSize, System.Collections.Generic.IList`1[T] infos) (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Inference/GeneratorImpl.cs:276)
Unity.MLAgents.Inference.TensorGenerator.GenerateTensors (System.Collections.Generic.IReadOnlyList`1[T] tensors, System.Int32 currentBatchSize, System.Collections.Generic.IList`1[T] infos) (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Inference/TensorGenerator.cs:173)
Unity.MLAgents.Inference.ModelRunner.DecideBatch () (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Inference/ModelRunner.cs:204)
Unity.MLAgents.Policies.BarracudaPolicy.DecideAction () (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Policies/BarracudaPolicy.cs:125)
Unity.MLAgents.Agent.DecideAction () (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Agent.cs:1360)
Unity.MLAgents.Academy.EnvironmentStep () (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Academy.cs:578)
Unity.MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at C:/Users/franc/Downloads/unity installed packages/ml-agents-release_17/com.unity.ml-agents/Runtime/Academy.cs:43)
Update: solved this by removing an old NN Brain I had loaded in the Agent.
Is there a way to visualize the detections of the Gridsensor the way you do it in the Dogfight Video? I don't know how to validate what my agent sees like a raycast would make a gizmo ?
Update: Found this gem you mentioned in the Debug sector, I see that it visualizes the observations in the Inspector instead of in a canvas in the Gameview.
Questions
- How to determine a good value for "Collider buffer size"?
- What is the Episode Reset checkbox?
- How is the sensor's Observation shape determined?
- I am using like in the Driver Scene a 2D snesor and a 3D sensor, what is the intuition behind this? Is it for the Agent to know in which direction to rotate, and the 3D sensor to in a 3D space photo where the objects are and how far?
- What are observations passed to the PPO Agent ? Does it observe a 2D Grid from the 3D observations or also the positions of each detected object?
- I keep reading in the Grid Sensors documentation "one-hot" but this doesn't fit in my understanding of one hot vectors, could you clarify me what is happening ? Specially in the 2D Grid Buffer (debug drawing), it says One-Hot.
- Is there a way to pass the Distance and the DotProduct(transform.forward, target.forward) for each object seen by the sensor? Right now I am passing the DotProduct manually in AddObservations by sampling what I have in front of me (I have a ConeScanner that marks an object as detected and gives the agent a reward, and when the cone detects something it passes the Distance it saw and the DotProduct to the Collectible).
Hi, the updated code and new readme are now posted on the main branch. I hope it clears up some questions.
- The driver has two sensors for looking far ahead (3D) and close range (2D).
- I'm using "one-hot" for observables that can either be 1 or 0.
- Both 2D and 3D sensor components have a public GetDetectedGameObjects(string tag) method that returns a gameobject enumeration. You should be able to calculate distances and dot products from the gameobject 's transforms. However, if you're using the 3D sensor component, distance is the default observable anyway.
hey, thanks for the update, i will look into it tomorrow.
i just made a video to show you a first demo of the prototype of my thesis :) i sucessfully trained last night with the gridsensor for release2, so thanks again.
https://youtu.be/sUjnC6-Wsvs
tomorrow, apart from reading the docs you've updated, i will shift my functionality to be a bit more like the explorer-drone you made. I was reading your code today and I think details of your thought process will improve my drone's behavior, for example you let the drone decide where to scan, with actions indexes 0 and 1, and you gave the rewards on the observation stage. My current drone has the scanner always on and it just has to determine how to move in XYZ+rotation in XZ, and gets a reward everytime it scans an object. I think the way you did it is a bit more of an active process to let the drone scan something itself, not as a result of how it moved with respect to some object.... If you have any ideas about this they're welcome!
Those changes will be like a 60% code change in my project. i estimate that should take a week cause I need to write chapters 1, 2 for monday.
Hey @mbaske thanks for all the updates you've done :)
I have a quick architecture question if you could help me out.
PPO's architecture from my understanding is 2 hidden layers and 2 outputs (shared parameter version)
But in MLAgents we always have a visual encoder in the front, by default simple. This simple encoder is 3 convolutions of
8x8x32, stride=4
4x4x64, stride=2
3x3x64, stride=1
How do the observations get transformed to get fed into the PPO network? Does the gridsensor + vector observations all get reshaped into a NxM grid and fed into the visual encoder ? I dont know where to find this answer...
I am trying to get a diagram of this style, I would highly appreciate the clarification.
Hi, the grid sensor observations are indeed being feed into the visual encoder. I'm not too familiar with the inner workings of the ML-Agents architecture though. Are you aware of Netron? https://netron.app/ You can upload a trained model to it and visualize the structure.
Quick question about obstacle detection. I am trying to make one gridsensor for goals/targets and one for obstacles.
For example for these rocks (obstacles)
(left: gridsensor, right: the little cubes that compose the shape scan)
I only detect lines like this though from the little cubes
The little cubes don't seem to all be scanned or idk what I am configuring wrong.
Also when I am very close to the object the Debug info disappears, even if min distance is set to 0
Same for different configurations of the Modifier
How can I make it so that more "little cubes" are detected? and do they have a better name, sorry.
Here is a more exaggerated example with higher numbers but still it only detects 2-3 per pillar. You can also see the config for the DGO
idk why with one boulder specifically it sees a better shape
What could be happening is that the number of detectable points is reduced too much, because of the distance between sensor and object. If you pause play mode and select the boulder, you should see enough red (currently detectable) points so they still kinda outline the shape. Maybe the sensor needs a higher max. distance value, in order to not reduce point counts as much.
thanks for your response! also i have no clue why my island collider mesh that sets the limits for the whole island, the red cubes collect themselves around the bikes ? (my goals)
these ones are not seen either
Are the bikes maybe on the Boundary layer as well? For concave colliders, the shape scanning uses raycasts - looks like the rays are colliding around the bikes for some reason.
Good point, the bikes were indeed in the boundary layer. i will check what else is in that layer and move it to their respective layer
collectible boundary obstacle
i didnt know/ignored that the layer plays a role in the detection!
how come during play time some cubes are gray? indeed the ones that the sensor detects are the red ones only. first picture shows some gray ones at the borders of the rock . second picture is play mode and shows only a handful red and the rest greyed out.
The grey points are only detectable when the sensor is very close to the object, in order to provide more detail. These are the points you're seeing in the shape settings for the highest LOD. The larger the distance, the less points are detectable/red. When the object is just at the edge of the sensor range, only a single red point should be visible.
aha! i understand now.. is there a way to detect a higher level of detail at LOD 0 for example... ? or is it not required for the agent to understand there is an obstacle "near"?
i am trying to use the gridsensor instead of a raycast to detect "boundaries or obstacles".. and if i put it on "closest point" it does not see the object i think, because the border points are set for a different LOD level, 5 for example
edit: i think closest point doesnt detect anything at all.
is there a way to detect a higher level of detail at LOD 0 for example... ?
No, I'm afraid not. The assumpition was that less points are required the farther away objects are. But yeah, I see how being able to lock a fixed LOD could be useful. Currently, this is only possible if you enable the "flatten" option, but then all points are on the XZ-plane.
As long as you include "Distance" in the observables, the agent should know how far away objects are. Closest Point only detects a single point on the collider bounds. I think the sensor itself has to be outside of those bounds though.
hmm ok. so closest point should probably the way to go to detect obstacles, the same way a raycast would return the distance to hit, if i am not mistaken.
but Debug unfortunately does not visualize closest point "interactively"... which is a downside :/ i dont know if this is because closest point is in trouble with the LOD constraint, that it does not see the borders of an object...
also, Distance does not tell the agent/sensor how wide the object is, only the transform.position (if i understand correctly), which for goals maybe it is good but for obstacles not.
but then i wonder why not / where would Shape be used?
in contrast, i see that in the gridsensor 2D closest point and shape do "work" and are visible in Debug.
exactly what the figure shows in 2D but in 3D for obstacles would have been amazing!
i also just noticed that in general the grindsensor (3D) ignores colliders that are in the way of the field of vision.. in other words it has True Sight/Wall hacks, in difference to raycasts..
but Debug unfortunately does not visualize closest point "interactively"...
Strange, it should display the point. The sensor can observe distance regardless of whether position, closest point or shape is set. Shapes are just inferred by the distribution of points in the grid. But either way, each point has a distance value, being brighter in the debug view the closer it is.
in other words it has True Sight, in difference to raycasts..
Yes, if points overlap though from the perspective of the sensor, the closer point is used and occluded points are ignored.
being able to lock a fixed LOD could be useful.
This should actually be pretty easy to hack; If you go to
https://github.com/mbaske/grid-sensor/blob/master/Assets/Scripts/Sensors/Grid/GameObject/Detection/GameObjectShape.cs#L244
and replace SetSelectedLOD(Mathf.RoundToInt((1 - normDistance) * m_MaxLOD)); with SetSelectedLOD(m_MaxLOD));
the sensor should always see the highest LOD.
Sorry that I edited my older answer to update new knowledge I discovered. I don't want to spam you with emails.
By true sight I meant that if a bike is behind a stone, the GoalGridsensor sees the bike. This might be unproductive for "exploration". Is there a way to not detect "little cubes" that are behind other objects?
I will try this LOD change to see what happens.
I am trying with a 2D Gridsensor since it allows me to show in the Debug the obstacles around (see Figures attached)
Also I noticed that the drag "button" to adjust the size of the gridsensor even when outside of Play mode it does not actually change the values somehow in the corresponding variables, so when we go back into play mode the sensor distance goes back to what it was.
i will update to the newer version from August that you published. I am running an older version
ok i got it to work
it is related to Shape with the above settings and the Cell Arc of the Gridsensor plays a huuuuge role (and i still dont really understand how it behaves 100%).
quick question, should i have a separate gridsensor for goals and one for obstacles/boundaries or does it pass the tag of the object it detects?
quick question, should i have a separate gridsensor for goals and one for obstacles/boundaries or does it pass the tag of the object it detects?
You don't need a separate sensor if the FOV angles, resolution and range are the same. Multiple tags and their observable values are encoded as different channels in the resulting visual observation.
thanks so much! this solves all my questions, except my last one:
- to disable Wallhack on the gridsensor, which line should I modify to prohibit adding observations that are behind objects?