NVIDIA
Added deriv/wrt operators.
The deriv operator will return the first derviative with respect to "X" of the underlying operation.
The derivative of a constant (or tensor) will return 0.
The wrt operator signifies where the variable "X" is in the operation.
Fundamentally this operator has the derivative return 1 instead of 0 at this point. So wrt(tensor) will return 1 instead of 0.
The operator passed into wrt then becomes the points at which the derivative is sampled.
wrt can appear multiple times in an operator but cannot be nested and should have the same sample points passed into it.
Examples:
// df/dx sin(x)*sin(y), sampled at x auto op = deriv(sin(wrt(x)) * sin(y));
// df/dy sin(x)*sin(y), sampled at y auto op = deriv(sin(x) * sin(wrt(y)));
// df/dx sin(x)*sin(x), sampled at x auto op = deriv(sin(wrt(x)) * sin(wrt(x)));
Limiations: deriv cannot be nested. Only the first derivative can be computed. wrt cannot be nested. Not all operations support derivatives.
I could use a good review of this. In particular some of the scalar derivatives have not been tested.