Julien Perez

Maybe building upon this remark, it seems that all environments have a step() function returning unconditionally "None" as the third variable where it is supposed to be a "done" indicator...

Dear @ryanjulian, thanks very much for your detailed answer. Maybe as a middle ground, letting the decision of an environment to be considered as a finite-horizon or infinite-horizon manner could...