Possible memory leakage issue with the '-=' operator.

[chenyang-tao]

Hey developers, here is a possible memory leakage issue that took me a lot of pain to track down...

If I run the following code a few times:

# Params: a minpy array of parameters
# grad_J:  minpy array of gradients wrt Params, returned by grad_and_loss object
# init_params(): an initializer of the parameters
init_params(Params)
print(Params[:3])
print(grad_J[:3])
Params -= grad_J      # This fails
# Params[:] -= grad_J # This will work
print(Params[:3])

the outputs would be something like:

# First run, correct
[-0.30823579 -0.21430817 -0.08100606]
[ -3.63609812e-04  -3.21333297e-04  -1.86065358e-06]
[-0.30787218 -0.21398684 -0.08100419]

# Second run, oops!
[ 0.34216672 -0.62399834  0.29504973]
[ -3.63609812e-04  -3.21333297e-04  -1.86065358e-06]
[-0.30750856 -0.21366552 -0.08100233]

...

So obviously the '-=' operator is being applied to an obsolete copy of the parameters, that should not be in the memory as I have already overwritten it with the re-initialization op. And there are a few times I noticed the Params are simply overwritten by the grad_J with the '-=' op.

However the a very simple snippet of similar code works as expected in the same python session.

a = np.random.rand(5,)
b = np.random.rand(5,)
print(a)
print(b)
a -= b
print(a)

# first run
[ 0.78756934  0.56299415  0.99072079]
[ 0.46403942  0.8988106   0.45598527]
[ 0.32352993 -0.3358165   0.53473556]

# second run
[ 0.56228282  0.84919352  0.05164077]
[ 0.02119064  0.15657083  0.67894815]
[ 0.54109216  0.69262266 -0.62730742]

The version I am using is minpy (0.3.0).

[Taco-W]

@chenyang-tao Sorry for the delay. Can you add the complete code snippet? like the definition of init_params and grad_J. I've made a similar example and run for many times. The obsolete copy thing doesn't happen in my case