ROCmSoftwarePlatform
The behavior of miopenConvolutionForwardBias
I have tested some cases and found the behavior of this function is:
y = (alpha + beta) * y + alpha * bias
This make me very confused. In cudnn, function "cudnnAddTensor" is used to add bias, and the behavior of that is easy to understand:
C = alpha * A + beta * C
Hello @scarsty the miopenConvolutionForwardBias performs the common Conv + Bias operation.
y = x * w + b
where x is the input, w is the weights, b is bias with as many elements as there are output channels and * represents convolution. Recall that alpha and beta are restricted to 1 and 0 in MIOpen.
In light of the above, I am not sure I understand what you mean by the following:
y = (alpha + beta) * y + alpha * bias
This function only perform adding bias, no conv. The inputs have only y and b, no x.
@scarsty Do you still need assistance with this ticket? If not, please close the ticket. Thanks!
@scarsty Do you still need assistance with this ticket? If not, please close the ticket. Thanks!
I want to hear the reason to design this function like this from the miopen team
@scarsty Hi, the behavior of miopenConvolutionForwardBias is what you observed. As @JehandadKhan mentioned, when alpha = 1 and beta = 0, you have
y = y + bias
which satisfy the functions purpose:"Calculate element-wise scale and shift of a tensor via a bias tensor." https://rocm.docs.amd.com/projects/MIOpen/en/latest/doxygen/html/group__convolutions.html#ga8e445590bd57cd0038324943791a9b4a. If possible, can you provide more context about your question?
