How do you know when training is "done"?

[NextSim]

Problem description

How do we know if training is "complete"? That is, trained so that is it (at least close) to as well as it can drive?

Problem details

I am trying to train using the distributed RL tutorial. I've tried with transfer learning and also training from scratch.

However I can't get it to successfully train so that it can drive through the neighborhood. I know it can take > 7 days to train from scratch (it's currently on day 8). Sometimes it drives ok and you can see that it is adjusting the car around the center of the lane and will maybe make a turn, but more often that not it immediately turns into the bushes.

I know that it is still taking random actions in training, so when I am testing I am taking the latest .json file and running the RunModel notebook on a different machine, and the results are generally poor.

So how do we know when it is optimized well and can be used to successfully and reliably drive, without over training? Or does the training stop when it is "done" and doesn't think it can train it any better, and I haven't let the training run long enough?

I am running on a system with 32 GB of ram, which is usually enough, but sometimes on longer runs it does max out on ram, so that will slow it down a bit, but generally this isn't an issue.

Experiment/Environment details

• Tutorial used: DistributedRL
• Environment used: Neighborhood

[mitchellspryn]

There isn't an easy way to determine when it is done.

For our work, we tracked the average episode length, and once that started reaching 30 seconds consistently (the timeout for our experiment), we called it "done." Note that training longer would actually cause the model to get worse due to overfit.

[Misslindalian]

Hello! How about your trained models? My tested result is not good. When I trained the models, it stopped when the iterator = 45. But in fact, it needs to run 500 iterators. What I mean is that it "stopped early". Thank you for your patience!