Backward Propagation Pointer Bug
When we try to overload operations such that complex operations can be done in a single expression. Something like
Tensor<float> = a;
Tensor<float> = b;
Tensor<float> = c;
Tensor<float> = d;
Tensor<float> e = a*b + c*d;
e.backward();
We get several garbage value Tensor objects when we debug the backOp of e. This is a very puzzling bug.
Also we cannot perform operations where there is a temporary on the right hand side.
Tensor<float> e = a*b + c*d;
is an example of that where a*b is a temporary.
There are bugs in the operation overloading and that is a medium priority item. It would be very good to get fixed but we can proceed using the assemly coding style one op per line approach
Update on this is as follows:
- Added another way to perform operations on tensors by just taking the pointer of two tensors and adding them. This approach removes a lot of bugs and headaches, The only problem is via a user perspective with the API being
tensorOps::add(t1,t2)instead oft1+t2.
Maybe this will change in the future to the latter hopefully.
Changes will be pushed through the CUDA dot product feature