Exponential increase in loss

[hariharan-jayakumar]

Hi @gsurma,

Thank you for the wonderful code and the medium article. I tried implementing your code but found that the loss function in my model shoots off after some time.

These are the hyper-parameters I used:

initialize environment

env = MainGymWrapper.wrap(gym.make('SpaceInvaders-v0')) #env = gym.make('SpaceInvaders-v0')

define hyperparameters

total_step_limit = 5000000 wandb.config.episodes = 1000 GAMMA = 0.99 MEMORY_SIZE = 350000 BATCH_SIZE = 32 TRAINING_FREQUENCY = 4 TARGET_NETWORK_UPDATE_FREQUENCY = 40000 MODEL_PERSISTENCE_UPDATE_FREQUENCY = 10000 REPLAY_START_SIZE = 50000 action_size = env.action_space.n EXPLORATION_MAX = 1.0 EXPLORATION_MIN = 0.1 EXPLORATION_TEST = 0.02 EXPLORATION_STEPS = 425000 EXPLORATION_DECAY = (EXPLORATION_MAX-EXPLORATION_MIN)/EXPLORATION_STEPS wandb.config.batch_size = 32 wandb.config.learning_rate = 0.00025 input_shape = (4, 84, 84)

The CNN is the same. I also used np.sign for the rewards I got.

Can you guide me on what might be possibly going wrong?