Sai Bhargav Yalamanchi

Reproducing agent performance in MovementBandits

3

Hi, great work on the paper and code! I am working on a project that builds on top of MLSH. We implemented [our own GPU optimized version](https://github.com/ysaibhargav/mlsh-gpu) of the algorithm...

Terminal states logic for sub-policies

1

https://github.com/openai/mlsh/blob/58f527ab7e3397eeb723a7309852b6d8791d5c24/mlsh_code/rollouts.py#L123 Hi, shouldn't the logic for determining terminal states for sub-policies consider the case where the master action changes? If the action changes, shouldn't we designate the current state as...

Looks like the algorithm to produce the exact result is computationally expensive; so if we can find a method for large N (number of trials) that would yield approximate PDF...